Database Access with database/sql
π§© Prerequisites: Youβre comfortable with structs and JSON (lesson 0003), context (lesson 0005), and the context deeper bonus lesson. Youβve got structs for data shapes and context for cancellation β database queries need both.
π― Your mission: So far, your API data lives in memory β it disappears when the server restarts. Real backends persist data to a database. Goβs standard library includes database/sql β a lightweight, driver-agnostic package for SQL databases. By the end, youβll have a server that creates, reads, and lists users from a real SQLite database that survives restarts.
*sql.DB β the connection pool
sql.Open(driver, dsn) returns a *sql.DB. This is a pool of connections, not a single connection. Create it once at startup and share it with your handlers. sql.Open doesnβt actually connect β it just validates the driver name. Call db.PingContext(ctx) to verify the database is reachable. β sql.Open docs
Three query methods β choose the right one
db.QueryContext(ctx, sql, argsβ¦) β returns *sql.Rows for SELECT (multiple rows). Must defer rows.Close() and iterate with rows.Next().
db.QueryRowContext(ctx, sql, argsβ¦) β returns *sql.Row for single-row SELECT. Call .Scan(&varsβ¦) directly.
db.ExecContext(ctx, sql, argsβ¦) β for INSERT, UPDATE, DELETE. Returns sql.Result (last insert ID, rows affected).
β DB.QueryContext docs
Scanning rows into structs
rows.Scan(&field1, &field2, β¦) copies column values into your variables in column order. Pass pointers so Scan can write into them. Common pattern: create a slice of structs, iterate rows, scan each row into a struct, append to slice. β Rows.Scan docs
SQLite with no CGO: modernc.org/sqlite
Weβre using modernc.org/sqlite β a pure Go SQLite driver. No C compiler needed, works on all platforms. It implements the standard database/sql driver interface, so your Go code looks the same regardless of which database youβre using. β sqlite driver docs
Where does *sql.DB live? In a struct, not context
This is exactly the design rule from the bonus lesson. The database handle is a dependency β it goes in a struct field, not in context.Context. Handlers become methods on that struct, giving them access to s.db. Context is for request-scoped data; structs are for dependencies. β Bonus: Context vs. Arguments
// The database/sql flow β from handler to database and back
<span class="hl">Handler</span>
β
ββ <span class="dim">s.db.QueryContext(r.Context(), "SELECT ...")</span>
β β
β ββ <span class="dim">Uses r.Context() for cancellation</span>
β β <span class="dim">(client disconnects β query cancelled)</span>
β β
β ββ <span class="hl">β</span> *sql.Rows
β β
β ββ <span class="dim">defer rows.Close() // always close</span>
β ββ <span class="dim">rows.Next() // advance to next row</span>
β ββ <span class="dim">rows.Scan(&u.ID, &u.Name) // copy into struct</span>
β
ββ <span class="dim">s.db.QueryRowContext(r.Context(), "SELECT ... WHERE id=?", id)</span>
β ββ <span class="hl">β</span> *sql.Row
β ββ <span class="dim">.Scan(&u.ID, &u.Name) // single row, no iteration</span>
β
ββ <span class="dim">s.db.ExecContext(r.Context(), "INSERT INTO ...")</span>
ββ <span class="hl">β</span> sql.Result
ββ <span class="dim">.LastInsertId()</span>
ββ <span class="dim">.RowsAffected()</span>| Method | SQL | Returns | Use when |
|---|---|---|---|
QueryContext | SELECT | *sql.Rows (many) | Expecting multiple rows |
QueryRowContext | SELECT | *sql.Row (one) | Expecting exactly one row |
ExecContext | INSERT, UPDATE, DELETE | sql.Result | Modifying data, no rows returned |
Follow each step in order. Click Done β after completing it.
mkdir -p ~/dev/learn/go-lang/lessons/0006-database-sql
cd ~/dev/learn/go-lang/lessons/0006-database-sql
go mod init database-demo
go get modernc.org/sqliteThe go get command downloads the pure-Go SQLite driver and adds it to your go.mod. No C compiler or system library needed β it works everywhere.
Create main.go. This brings together everything youβve learned: structs for data, *sql.DB as a struct dependency, context-aware queries, JSON encoding, and middleware-style patterns.
package main
import (
"context"
"database/sql"
"encoding/json"
"log"
"net/http"
"time"
_ "modernc.org/sqlite" // driver registers itself with database/sql
)
// --- User struct ---
type User struct {
ID int `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
CreatedAt time.Time `json:"created_at"`
}
// --- Server holds dependencies ---
type Server struct {
db *sql.DB
}
// --- Handlers (methods on Server) ---
func (s *Server) listUsers(w http.ResponseWriter, r *http.Request) {
rows, err := s.db.QueryContext(r.Context(),
"SELECT id, name, email, created_at FROM users ORDER BY id")
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
defer rows.Close()
var users []User
for rows.Next() {
var u User
if err := rows.Scan(&u.ID, &u.Name, &u.Email, &u.CreatedAt); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
users = append(users, u)
}
if err := rows.Err(); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(users)
}
func (s *Server) getUser(w http.ResponseWriter, r *http.Request) {
id := r.PathValue("id")
var u User
err := s.db.QueryRowContext(r.Context(),
"SELECT id, name, email, created_at FROM users WHERE id = ?", id).
Scan(&u.ID, &u.Name, &u.Email, &u.CreatedAt)
if err == sql.ErrNoRows {
http.Error(w, "user not found", http.StatusNotFound)
return
}
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(u)
}
func (s *Server) createUser(w http.ResponseWriter, r *http.Request) {
var input struct {
Name string `json:"name"`
Email string `json:"email"`
}
if err := json.NewDecoder(r.Body).Decode(&input); err != nil {
http.Error(w, "invalid JSON", http.StatusBadRequest)
return
}
defer r.Body.Close()
result, err := s.db.ExecContext(r.Context(),
"INSERT INTO users (name, email, created_at) VALUES (?, ?, ?)",
input.Name, input.Email, time.Now())
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
id, _ := result.LastInsertId()
// Fetch the full row we just created
var u User
s.db.QueryRowContext(r.Context(),
"SELECT id, name, email, created_at FROM users WHERE id = ?", id).
Scan(&u.ID, &u.Name, &u.Email, &u.CreatedAt)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusCreated)
json.NewEncoder(w).Encode(u)
}
// --- Main ---
func main() {
// Open the database (creates the file if it doesn't exist)
db, err := sql.Open("sqlite", "app.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Verify we can actually reach the database
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := db.PingContext(ctx); err != nil {
log.Fatal("cannot ping database:", err)
}
// Create the table if it doesn't exist
_, err = db.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
email TEXT NOT NULL UNIQUE,
created_at DATETIME NOT NULL
)`)
if err != nil {
log.Fatal("cannot create table:", err)
}
// Seed some data if the table is empty
var count int
db.QueryRowContext(ctx, "SELECT COUNT(*) FROM users").Scan(&count)
if count == 0 {
db.ExecContext(ctx, "INSERT INTO users (name, email, created_at) VALUES (?, ?, ?)",
"Alice", "alice@example.com", time.Now())
db.ExecContext(ctx, "INSERT INTO users (name, email, created_at) VALUES (?, ?, ?)",
"Bob", "bob@example.com", time.Now())
log.Println("Seeded 2 users")
}
s := &Server{db: db}
mux := http.NewServeMux()
mux.HandleFunc("GET /users", s.listUsers)
mux.HandleFunc("GET /users/{id}", s.getUser)
mux.HandleFunc("POST /users", s.createUser)
log.Println("Server starting on http://localhost:8080")
log.Fatal(http.ListenAndServe(":8080", mux))
}Whatβs happening here?
_ βmodernc.org/sqliteβ: The blank import (_) tells Go βimport this for side effects only.β The driverβsinit()function registers itself withdatabase/sqlβ you never reference it directly.sql.Open(βsqliteβ, βapp.dbβ): Creates a connection pool to a file-based SQLite database. Ifapp.dbdoesnβt exist, itβs created automatically.db.PingContext(ctx): Actually verifies the database is reachable. Without this,sql.Opensucceeds even if the database is broken.s.db.QueryContext(r.Context(), β¦): Passes the requestβs context β if the client disconnects, the query is cancelled.sql.ErrNoRows: A sentinel error returned whenQueryRowContextfinds zero rows. Not a panic or crash β just a value you check with==.Serverstruct: Holds the*sql.DB. Handlers are methods, so they gets.dbautomatically.
Start the server:
go run main.goIn a second terminal, test the endpoints:
# List all users (comes back with seeded data)
curl http://localhost:8080/users
# Get a single user by ID
curl http://localhost:8080/users/1
# Create a new user
curl -X POST http://localhost:8080/users \
-H "Content-Type: application/json" \
-d '{"name":"Charlie","email":"charlie@example.com"}'
# Verify it persisted β list users again
curl http://localhost:8080/users
# Try fetching a user that doesn't exist
curl http://localhost:8080/users/999Prove it persists: Stop the server (Ctrl+C), restart it with go run main.go, and run curl http://localhost:8080/users again. The seeded users wonβt reappear (the table already has data), but Charlie will still be there β because the data is in app.db, not in memory.
Press Ctrl+C to stop the server when youβre done testing.
The app.db file in your lesson directory is a real SQLite database. You can inspect it directly:
# macOS comes with sqlite3 preinstalled. Try it:
sqlite3 app.db ".tables" # list tables
sqlite3 app.db "SELECT * FROM users;" # query data
sqlite3 app.db ".schema users" # see the table schemaThis is a great debugging habit β being able to query the database directly confirms whether the problem is in your Go code or in the data. The file app.db is a standard SQLite file, so any SQLite tool can open it.
Now combine this with what you learned in lessons 0005 and the bonus. Add a simple API-key auth middleware that protects POST /users. Add this to your main.go:
// Add above main()
type ctxKey string
const ctxKeyAPIKey ctxKey = "api_key"
func apiKeyMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
key := r.Header.Get("X-API-Key")
if key != "super-secret" {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
ctx := context.WithValue(r.Context(), ctxKeyAPIKey, key)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
// --- Update your main() to use it: ---
func main() {
// ... db setup stays the same ...
mux := http.NewServeMux()
mux.HandleFunc("GET /users", s.listUsers)
mux.HandleFunc("GET /users/{id}", s.getUser)
// Protected routes β only POST needs the API key
protected := http.NewServeMux()
protected.HandleFunc("POST /users", s.createUser)
mux.Handle("/", apiKeyMiddleware(protected))
log.Fatal(http.ListenAndServe(":8080", mux))
}Test it:
# GET still works without auth (public endpoint)
curl http://localhost:8080/users
# POST without API key β blocked with 401
curl -X POST http://localhost:8080/users \
-H "Content-Type: application/json" \
-d '{"name":"Dave","email":"dave@example.com"}'
# POST with valid API key β works
curl -X POST http://localhost:8080/users \
-H "Content-Type: application/json" \
-H "X-API-Key: super-secret" \
-d '{"name":"Dave","email":"dave@example.com"}'This is the sub-mux pattern for route-level middleware: create a separate mux for protected routes, wrap it with middleware, and mount it on the main mux. Clean, idiomatic, no framework needed.
Check your understanding
What does sql.Open actually do β does it connect to the database?
Where should *sql.DB live in your application?
What's the difference between QueryContext and QueryRowContext?
Why pass r.Context() to database methods instead of context.Background()?