Database Access with database/sql


Progress
0/5

🧩 Prerequisites: You’re comfortable with structs and JSON (lesson 0003), context (lesson 0005), and the context deeper bonus lesson. You’ve got structs for data shapes and context for cancellation β€” database queries need both.

🎯 Your mission: So far, your API data lives in memory β€” it disappears when the server restarts. Real backends persist data to a database. Go’s standard library includes database/sql β€” a lightweight, driver-agnostic package for SQL databases. By the end, you’ll have a server that creates, reads, and lists users from a real SQLite database that survives restarts.

*sql.DB β€” the connection pool

sql.Open(driver, dsn) returns a *sql.DB. This is a pool of connections, not a single connection. Create it once at startup and share it with your handlers. sql.Open doesn’t actually connect β€” it just validates the driver name. Call db.PingContext(ctx) to verify the database is reachable. β†’ sql.Open docs

Three query methods β€” choose the right one

db.QueryContext(ctx, sql, args…) β†’ returns *sql.Rows for SELECT (multiple rows). Must defer rows.Close() and iterate with rows.Next().
db.QueryRowContext(ctx, sql, args…) β†’ returns *sql.Row for single-row SELECT. Call .Scan(&vars…) directly.
db.ExecContext(ctx, sql, args…) β†’ for INSERT, UPDATE, DELETE. Returns sql.Result (last insert ID, rows affected).
β†’ DB.QueryContext docs

Scanning rows into structs

rows.Scan(&field1, &field2, …) copies column values into your variables in column order. Pass pointers so Scan can write into them. Common pattern: create a slice of structs, iterate rows, scan each row into a struct, append to slice. β†’ Rows.Scan docs

SQLite with no CGO: modernc.org/sqlite

We’re using modernc.org/sqlite β€” a pure Go SQLite driver. No C compiler needed, works on all platforms. It implements the standard database/sql driver interface, so your Go code looks the same regardless of which database you’re using. β†’ sqlite driver docs

Where does *sql.DB live? In a struct, not context

This is exactly the design rule from the bonus lesson. The database handle is a dependency β€” it goes in a struct field, not in context.Context. Handlers become methods on that struct, giving them access to s.db. Context is for request-scoped data; structs are for dependencies. β†’ Bonus: Context vs. Arguments

// The database/sql flow β€” from handler to database and back

<span class="hl">Handler</span>
β”‚
β”œβ”€ <span class="dim">s.db.QueryContext(r.Context(), "SELECT ...")</span>
β”‚     β”‚
β”‚     β”œβ”€ <span class="dim">Uses r.Context() for cancellation</span>
β”‚     β”‚   <span class="dim">(client disconnects β†’ query cancelled)</span>
β”‚     β”‚
β”‚     └─ <span class="hl">β†’</span> *sql.Rows
β”‚           β”‚
β”‚           β”œβ”€ <span class="dim">defer rows.Close()    // always close</span>
β”‚           β”œβ”€ <span class="dim">rows.Next()           // advance to next row</span>
β”‚           └─ <span class="dim">rows.Scan(&u.ID, &u.Name)  // copy into struct</span>
β”‚
β”œβ”€ <span class="dim">s.db.QueryRowContext(r.Context(), "SELECT ... WHERE id=?", id)</span>
β”‚     └─ <span class="hl">β†’</span> *sql.Row
β”‚           └─ <span class="dim">.Scan(&u.ID, &u.Name)   // single row, no iteration</span>
β”‚
└─ <span class="dim">s.db.ExecContext(r.Context(), "INSERT INTO ...")</span>
      └─ <span class="hl">β†’</span> sql.Result
            β”œβ”€ <span class="dim">.LastInsertId()</span>
            └─ <span class="dim">.RowsAffected()</span>
MethodSQLReturnsUse when
QueryContextSELECT*sql.Rows (many)Expecting multiple rows
QueryRowContextSELECT*sql.Row (one)Expecting exactly one row
ExecContextINSERT, UPDATE, DELETEsql.ResultModifying data, no rows returned

Follow each step in order. Click Done βœ“ after completing it.

1Set up the lesson directory and dependenciesCurrent
mkdir -p ~/dev/learn/go-lang/lessons/0006-database-sql
cd ~/dev/learn/go-lang/lessons/0006-database-sql
go mod init database-demo
go get modernc.org/sqlite

The go get command downloads the pure-Go SQLite driver and adds it to your go.mod. No C compiler or system library needed β€” it works everywhere.

2Write the full server β€” connect, create table, CRUD endpointsLocked

Create main.go. This brings together everything you’ve learned: structs for data, *sql.DB as a struct dependency, context-aware queries, JSON encoding, and middleware-style patterns.

package main

import (
  "context"
  "database/sql"
  "encoding/json"
  "log"
  "net/http"
  "time"

  _ "modernc.org/sqlite" // driver registers itself with database/sql
)

// --- User struct ---
type User struct {
  ID        int       `json:"id"`
  Name      string    `json:"name"`
  Email     string    `json:"email"`
  CreatedAt time.Time `json:"created_at"`
}

// --- Server holds dependencies ---
type Server struct {
  db *sql.DB
}

// --- Handlers (methods on Server) ---

func (s *Server) listUsers(w http.ResponseWriter, r *http.Request) {
  rows, err := s.db.QueryContext(r.Context(),
      "SELECT id, name, email, created_at FROM users ORDER BY id")
  if err != nil {
      http.Error(w, err.Error(), http.StatusInternalServerError)
      return
  }
  defer rows.Close()

  var users []User
  for rows.Next() {
      var u User
      if err := rows.Scan(&u.ID, &u.Name, &u.Email, &u.CreatedAt); err != nil {
          http.Error(w, err.Error(), http.StatusInternalServerError)
          return
      }
      users = append(users, u)
  }
  if err := rows.Err(); err != nil {
      http.Error(w, err.Error(), http.StatusInternalServerError)
      return
  }

  w.Header().Set("Content-Type", "application/json")
  json.NewEncoder(w).Encode(users)
}

func (s *Server) getUser(w http.ResponseWriter, r *http.Request) {
  id := r.PathValue("id")

  var u User
  err := s.db.QueryRowContext(r.Context(),
      "SELECT id, name, email, created_at FROM users WHERE id = ?", id).
      Scan(&u.ID, &u.Name, &u.Email, &u.CreatedAt)

  if err == sql.ErrNoRows {
      http.Error(w, "user not found", http.StatusNotFound)
      return
  }
  if err != nil {
      http.Error(w, err.Error(), http.StatusInternalServerError)
      return
  }

  w.Header().Set("Content-Type", "application/json")
  json.NewEncoder(w).Encode(u)
}

func (s *Server) createUser(w http.ResponseWriter, r *http.Request) {
  var input struct {
      Name  string `json:"name"`
      Email string `json:"email"`
  }
  if err := json.NewDecoder(r.Body).Decode(&input); err != nil {
      http.Error(w, "invalid JSON", http.StatusBadRequest)
      return
  }
  defer r.Body.Close()

  result, err := s.db.ExecContext(r.Context(),
      "INSERT INTO users (name, email, created_at) VALUES (?, ?, ?)",
      input.Name, input.Email, time.Now())
  if err != nil {
      http.Error(w, err.Error(), http.StatusInternalServerError)
      return
  }

  id, _ := result.LastInsertId()

  // Fetch the full row we just created
  var u User
  s.db.QueryRowContext(r.Context(),
      "SELECT id, name, email, created_at FROM users WHERE id = ?", id).
      Scan(&u.ID, &u.Name, &u.Email, &u.CreatedAt)

  w.Header().Set("Content-Type", "application/json")
  w.WriteHeader(http.StatusCreated)
  json.NewEncoder(w).Encode(u)
}

// --- Main ---

func main() {
  // Open the database (creates the file if it doesn't exist)
  db, err := sql.Open("sqlite", "app.db")
  if err != nil {
      log.Fatal(err)
  }
  defer db.Close()

  // Verify we can actually reach the database
  ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
  defer cancel()
  if err := db.PingContext(ctx); err != nil {
      log.Fatal("cannot ping database:", err)
  }

  // Create the table if it doesn't exist
  _, err = db.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS users (
      id INTEGER PRIMARY KEY AUTOINCREMENT,
      name TEXT NOT NULL,
      email TEXT NOT NULL UNIQUE,
      created_at DATETIME NOT NULL
  )`)
  if err != nil {
      log.Fatal("cannot create table:", err)
  }

  // Seed some data if the table is empty
  var count int
  db.QueryRowContext(ctx, "SELECT COUNT(*) FROM users").Scan(&count)
  if count == 0 {
      db.ExecContext(ctx, "INSERT INTO users (name, email, created_at) VALUES (?, ?, ?)",
          "Alice", "alice@example.com", time.Now())
      db.ExecContext(ctx, "INSERT INTO users (name, email, created_at) VALUES (?, ?, ?)",
          "Bob", "bob@example.com", time.Now())
      log.Println("Seeded 2 users")
  }

  s := &Server{db: db}

  mux := http.NewServeMux()
  mux.HandleFunc("GET /users", s.listUsers)
  mux.HandleFunc("GET /users/{id}", s.getUser)
  mux.HandleFunc("POST /users", s.createUser)

  log.Println("Server starting on http://localhost:8080")
  log.Fatal(http.ListenAndServe(":8080", mux))
}
What’s happening here?
  • _ β€œmodernc.org/sqlite”: The blank import (_) tells Go β€œimport this for side effects only.” The driver’s init() function registers itself with database/sql β€” you never reference it directly.
  • sql.Open(β€œsqlite”, β€œapp.db”): Creates a connection pool to a file-based SQLite database. If app.db doesn’t exist, it’s created automatically.
  • db.PingContext(ctx): Actually verifies the database is reachable. Without this, sql.Open succeeds even if the database is broken.
  • s.db.QueryContext(r.Context(), …): Passes the request’s context β€” if the client disconnects, the query is cancelled.
  • sql.ErrNoRows: A sentinel error returned when QueryRowContext finds zero rows. Not a panic or crash β€” just a value you check with ==.
  • Server struct: Holds the *sql.DB. Handlers are methods, so they get s.db automatically.
3Run the server and test the endpointsLocked

Start the server:

go run main.go

In a second terminal, test the endpoints:

# List all users (comes back with seeded data)
curl http://localhost:8080/users

# Get a single user by ID
curl http://localhost:8080/users/1

# Create a new user
curl -X POST http://localhost:8080/users \
-H "Content-Type: application/json" \
-d '{"name":"Charlie","email":"charlie@example.com"}'

# Verify it persisted β€” list users again
curl http://localhost:8080/users

# Try fetching a user that doesn't exist
curl http://localhost:8080/users/999

Prove it persists: Stop the server (Ctrl+C), restart it with go run main.go, and run curl http://localhost:8080/users again. The seeded users won’t reappear (the table already has data), but Charlie will still be there β€” because the data is in app.db, not in memory.

Press Ctrl+C to stop the server when you’re done testing.

4Explore the database file with the SQLite CLILocked

The app.db file in your lesson directory is a real SQLite database. You can inspect it directly:

# macOS comes with sqlite3 preinstalled. Try it:
sqlite3 app.db ".tables"          # list tables
sqlite3 app.db "SELECT * FROM users;"  # query data
sqlite3 app.db ".schema users"    # see the table schema

This is a great debugging habit β€” being able to query the database directly confirms whether the problem is in your Go code or in the data. The file app.db is a standard SQLite file, so any SQLite tool can open it.

5Add auth middleware β€” protect the endpointsLocked

Now combine this with what you learned in lessons 0005 and the bonus. Add a simple API-key auth middleware that protects POST /users. Add this to your main.go:

// Add above main()

type ctxKey string
const ctxKeyAPIKey ctxKey = "api_key"

func apiKeyMiddleware(next http.Handler) http.Handler {
  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
      key := r.Header.Get("X-API-Key")
      if key != "super-secret" {
          http.Error(w, "unauthorized", http.StatusUnauthorized)
          return
      }
      ctx := context.WithValue(r.Context(), ctxKeyAPIKey, key)
      next.ServeHTTP(w, r.WithContext(ctx))
  })
}

// --- Update your main() to use it: ---

func main() {
  // ... db setup stays the same ...

  mux := http.NewServeMux()
  mux.HandleFunc("GET /users", s.listUsers)
  mux.HandleFunc("GET /users/{id}", s.getUser)

  // Protected routes β€” only POST needs the API key
  protected := http.NewServeMux()
  protected.HandleFunc("POST /users", s.createUser)
  mux.Handle("/", apiKeyMiddleware(protected))

  log.Fatal(http.ListenAndServe(":8080", mux))
}

Test it:

# GET still works without auth (public endpoint)
curl http://localhost:8080/users

# POST without API key β€” blocked with 401
curl -X POST http://localhost:8080/users \
-H "Content-Type: application/json" \
-d '{"name":"Dave","email":"dave@example.com"}'

# POST with valid API key β€” works
curl -X POST http://localhost:8080/users \
-H "Content-Type: application/json" \
-H "X-API-Key: super-secret" \
-d '{"name":"Dave","email":"dave@example.com"}'

This is the sub-mux pattern for route-level middleware: create a separate mux for protected routes, wrap it with middleware, and mount it on the main mux. Clean, idiomatic, no framework needed.

Check your understanding

What does sql.Open actually do β€” does it connect to the database?

Where should *sql.DB live in your application?

What's the difference between QueryContext and QueryRowContext?

Why pass r.Context() to database methods instead of context.Background()?