Concurrency and Locking

Running migrations from several processes at once — for example a Kubernetes Job whose pods start together, or every replica of an app migrating on boot — is safe. Since v1.2.0, every operation that changes the database takes a cross-process migration lock before it touches the tracking table, and computes the pending or applied migrations only once it holds the lock. A second process waits, then finds nothing left to do.

The lock is on by default. You don't need to configure anything to get it.

Which Operations Take the Lock

How the Lock Works per Database

<table> is the tracking table name, so migration sets that use different tracking tables (WithTableName) do not block each other.

The lock is taken without blocking and retried with a short backoff — 100 ms, doubling up to 1 s — until it is acquired or the lock timeout elapses. The operation then fails with migrator.ErrLockTimeout without having changed anything:

could not acquire migration lock within 30s (lock "go-migration:migrations" is held by another process)

The lock is held on a dedicated connection taken from the pool for the duration of the operation. If releasing the lock fails, that connection is closed instead of being returned to the pool, which makes the server drop the lock.

Configuring the Lock

package main

import (
    "database/sql"
    "errors"
    "log"
    "time"

    _ "github.com/lib/pq"

    "github.com/gopackx/go-migration/pkg/migrator"
    "github.com/gopackx/go-migration/pkg/schema/grammars"
    _ "your-project/database/migrations"
)

func main() {
    db, err := sql.Open("postgres", "postgres://user:password@localhost:5432/mydb?sslmode=disable")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // One connection holds the lock, another runs the migrations.
    db.SetMaxOpenConns(5)

    m := migrator.New(db,
        migrator.WithGrammar(grammars.NewPostgresGrammar()),
        migrator.WithLockTimeout(2*time.Minute),
        migrator.WithAutoDiscover(),
    )

    if err := m.Up(); err != nil {
        if errors.Is(err, migrator.ErrLockTimeout) {
            log.Fatalf("another process is still migrating: %v", err)
        }
        log.Fatal(err)
    }
}

go-migration migrate --lock-timeout=2m
go-migration db:seed --no-lock

Locking Your Own Work

m.Lock(ctx) acquires the same lock and returns a release function. Use it to serialise your own writers — a data backfill, or seeders run through seeder.Runner — with migrations running in other processes:

func (m *Migrator) Lock(ctx context.Context) (release func() error, err error)

func backfill(ctx context.Context, m *migrator.Migrator, db *sql.DB) error {
    release, err := m.Lock(ctx)
    if err != nil {
        return err // wraps migrator.ErrLockTimeout on timeout
    }
    defer func() {
        if err := release(); err != nil {
            log.Printf("release migration lock: %v", err)
        }
    }()

    _, err = db.ExecContext(ctx, `UPDATE users SET active = true WHERE active IS NULL`)
    return err
}

The lock is polled until it is acquired, the lock timeout elapses, or ctx is cancelled. Calling release more than once is safe. If locking is disabled (WithoutLock) or the database needs no lock (SQLite), Lock returns immediately with a no-op release.

Connection Pool Requirements

On PostgreSQL and MySQL the lock is a session-level lock held on its own connection while the work runs on another. With locking enabled, the pool therefore needs at least two connections: MaxOpenConns >= 2, or 0 for unlimited (the database/sql default). A pool limited to one connection is rejected with an error instead of deadlocking:

migration lock: the connection pool is limited to 1 open connection, but the lock needs its own
connection in addition to the one running migrations; set MaxOpenConns to at least 2 or disable
locking (WithoutLock / --no-lock)

SQLite takes no lock, so SetMaxOpenConns(1) — the recommended setting for SQLite — is fine there.

Deployment Patterns

Migrate on Every Replica

Because the lock serialises concurrent runs, every replica of your app can simply call m.Up() at startup. The first one to get the lock applies the pending migrations; the others wait (up to the lock timeout) and then find nothing pending. Make the lock timeout longer than your slowest migration batch.

Readiness Gates

A cleaner pattern on Kubernetes is to run migrations once, in a Job, and make the application pods wait until the database is up to date. migrate:status --check is built for this: it is read-only, never takes the lock, and exits with status 1 (printing N migration(s) pending to stderr) while anything is pending.

apiVersion: batch/v1
kind: Job
metadata:
  name: myapp-migrate
spec:
  backoffLimit: 3
  template:
    spec:
      restartPolicy: OnFailure
      containers:
        - name: migrate
          image: registry.example.com/myapp-migrator:1.4.0
          args: ["migrate", "--lock-timeout=5m"]
          envFrom:
            - secretRef:
                name: myapp-db

spec:
  template:
    spec:
      initContainers:
        - name: wait-for-migrations
          image: registry.example.com/myapp-migrator:1.4.0
          command: ["sh", "-c", "until migrator migrate:status --check; do sleep 5; done"]
          envFrom:
            - secretRef:
                name: myapp-db
      containers:
        - name: app
          image: registry.example.com/myapp:1.4.0

If the Job's pods are started more than once (retries, or parallelism), the lock makes sure only one of them migrates at a time.

The same command works as a CI gate, for example to fail a pipeline when a deployment forgot to run its migrations:

go-migration migrate:status --check

What's Next?

• Running Migrations — what m.Up() does, step by step
• Migration Status — read-only status and --check
• Pool Configuration — tune MaxOpenConns and friends
• CLI Reference — --lock-timeout and --no-lock