Migrations

Autumn embeds Diesel migrations in the compiled binary and runs them at startup (dev) or via autumn migrate (production). This guide covers the advisory-lock serialisation that prevents schema divergence during rolling deploys, how to monitor contention, and what to expect on non-Postgres backends.

Keep each Diesel table! at 32 columns or fewer. A wider declaration forces diesel's 64-column-tables feature, which makes every crate that uses diesel slower to compile. A wide table is fine when Rust declares it in halves with #[sql_name]. autumn build warns when a migration widens a table past 32 columns and no declaration splits it. See Table width.

Scope: control only, or shards only

AppBuilder::migrations sends a set to the control database and to every shard ([[database.shards]], or each [database.fleet] file). Use migrations_for to keep a set on one side:

use autumn_web::migrate::MigrationScope;

autumn_web::app()
    .migrations_for(MigrationScope::Control, CONTROL_MIGRATIONS)
    .migrations_for(MigrationScope::Shards, TENANT_MIGRATIONS)

Boot and AUTUMN_MIGRATE=1 obey the scope. The autumn migrate and autumn fleet migrate commands read migrations/ and do not know a scope. Do not run autumn migrate on an app with a scoped set: use AUTUMN_MIGRATE=1. See SQLite fleet: migrations.

Advisory lock serialisation

When several replicas boot at the same time or when autumn migrate is called from multiple deployment steps concurrently, every instance would naively race to apply the same pending migrations. Diesel wraps each migration in a transaction, so two processes applying the same migration can deadlock, fail mid-DDL, or leave the schema half-applied.

Autumn prevents this by acquiring a PostgreSQL session-level advisory lock before reading the pending-migration list. Only one process holds the lock at a time; the rest wait (polling every 500 ms). Once the winner finishes, waiters re-read the migration table, find no pending work, and exit successfully.

The lock covers:

• The embedded application migrations run by AppBuilder::migrations(…).
• The Autumn framework migrations run by autumn migrate.

Version collisions

Diesel records applied migrations by version — the leading YYYYMMDDHHMMSS prefix of a migration directory's name — in a single shared table, with no notion of which registered set (the framework, a plugin, the app's own migrations/) a version came from. When two differently-named migrations share one version, the framework detects it automatically at boot and gives the losing one a deterministic substitute version, so both still apply — logged at INFO ("Migration version collision resolved automatically"), never a silently skipped migration. The one exception is a narrow, unrecoverable case on SQLite: adopting a pre-fork database whose already-applied history can't be safely rewritten fails loudly instead of guessing.

Auto-resolution means a collision is never silently lost, but the substitute is still a generated, less-readable version stamp on an otherwise ordinary migration. Prefer to avoid the collision in the first place, most likely when two branches each add a migration around the same time and one picks a round, hand-typed timestamp (midnight, on the hour) instead of the actual current second — two people reaching for 00:00:00 on the same day collide; two people reaching for the real current second, to the second, essentially never do. Two tools help:

• autumn migrate new <name> creates migrations/<version>_<name>/{up,down}.sql with a version guaranteed free across the working tree, every local and remote-tracking git branch this checkout has fetched, and the framework's own compiled-in migrations. Use this instead of hand-creating a migration directory.
• autumn migrate check-collisions is the CI-time backstop for a collision migrate new could not see — a branch pushed after, or a teammate's concurrent PR. It fails when a version this checkout introduces is already claimed by a different directory on the default branch, another pushed branch, or the framework's own migrations, and is silent about a collision between two other branches (that is their gate to fail on, not this checkout's). It needs the full branch history (actions/checkout@v7 with fetch-depth: 0, or a local git fetch origin); without it, it degrades to a working-tree-only check and prints a loud warning rather than failing closed.

Lock key

The advisory lock uses a single bigint key:

MIGRATION_ADVISORY_LOCK_KEY = 0x6175_746E_5F6D_6967  (7 021 124 476 890 851 687)

The value is the big-endian encoding of the ASCII bytes autn_mig. It is stable across framework versions so you can add permanent alerting rules without consulting the source code.

PostgreSQL splits a pg_advisory_lock(bigint) key into two 32-bit halves stored in pg_locks:

Monitoring contention

-- Active migration lock holders and waiters
SELECT
    pid,
    granted,
    mode,
    (SELECT query FROM pg_stat_activity WHERE pid = l.pid) AS query
FROM pg_locks l
WHERE locktype = 'advisory'
  AND classid = 1635087470
  AND objid   = 1601005927
  AND objsubid = 1;

A row with granted = false means a waiter is queued behind the current lock holder. If rows remain indefinitely after a deploy, check whether a migration process crashed mid-run; PostgreSQL will release the lock when the connection closes.

Wait timeout

The default wait is 60 seconds. If the lock is not acquired within that window the process fails with:

migration advisory lock not acquired within 60s;
another process may still be running migrations

The timeout can be overridden per call when using the Rust API:

use autumn_web::migrate::{run_pending_locked, DEFAULT_LOCK_WAIT_TIMEOUT};
use std::time::Duration;

// Use the default (60 s)
run_pending_locked(database_url, MIGRATIONS, None)?;

// Override to 120 s
run_pending_locked(database_url, MIGRATIONS, Some(Duration::from_secs(120)))?;

For the autumn migrate CLI the timeout is always the default.

Table lock timeout and retry

A DDL statement such as ALTER TABLE needs a strong table lock. If a long transaction holds a lock on that table, the DDL waits. Every new query on the table then waits behind the DDL, and the app stalls.

To prevent this, each transactional migration starts with SET LOCAL lock_timeout. A migration that waits longer fails, and its transaction rolls back. The migrator then tries again after a delay. The delay starts at 500ms and doubles each time, to at most 10s. A random factor of 50–100% scales each delay. The retries happen under the advisory lock.

[database]
migration_lock_timeout = "5s"   # default; "0s" = no limit
migration_lock_retries = 5      # default; 0 = no retry

The env vars are AUTUMN_DATABASE__MIGRATION_LOCK_TIMEOUT and AUTUMN_DATABASE__MIGRATION_LOCK_RETRIES. When every attempt times out, the run fails with MigrationError::LockContention. Find the blocking session:

SELECT pid, state, xact_start, query
FROM pg_stat_activity
WHERE pid IN (SELECT pid FROM pg_locks WHERE relation = 'users'::regclass);

Only a lock timeout is retried. A migration with run_in_transaction = false (for example CREATE INDEX CONCURRENTLY) runs with lock_timeout = 0. It must wait for older transactions, and a timeout would leave an INVALID index. The value is always set explicitly, so a role or database default does not apply. "0s" turns such a default off too.

The check reads the English Postgres message. The migrator sets lc_messages = 'C' on its session. autumn migrate adds -c lc_messages=C to PGOPTIONS, but only after a check shows that the role may set it. When the role may not set it, a non-English lock timeout still stops the migration, but it is not retried.

The Rust API takes the policy explicitly:

use autumn_web::migrate::{run_pending_locked_with_policy, MigrationLockPolicy};
use std::time::Duration;

let policy = MigrationLockPolicy::new(Duration::from_secs(2), 3);
run_pending_locked_with_policy(database_url, MIGRATIONS, None, policy)?;

autumn migrate passes the timeout to its diesel subprocess as PGOPTIONS=-c lock_timeout=<ms>. This sets it for the whole session, so:

• When a pending migration has run_in_transaction = false, the CLI runs the pending set in batches of one kind. Transactional batches get the timeout. Non-transactional batches get lock_timeout=0.
• When the server refuses the option (PgBouncer can), the CLI runs again with only the PGOPTIONS it inherited, unchanged. Transactional migrations then run from a copy whose up.sql starts with SET LOCAL lock_timeout = <ms>;, which a transaction pooler keeps. A non-transactional batch runs again only when the server default lock_timeout is already 0; otherwise the run stops, because a role or database default would cancel CREATE INDEX CONCURRENTLY and leave an INVALID index.
• Before it splits the pending set into batches, the CLI reads it with the same lock_timeout and retries, so a lock on __diesel_schema_migrations cannot block it.
• Through a transaction pooler, diesel reads __diesel_schema_migrations before the first migration's SET LOCAL runs, and no session setting reaches that read. A lock held on that table can make this read wait. Run migrations against Postgres directly to keep every step bounded.
• libpq ignores PGOPTIONS when DATABASE_URL has an options parameter, in a postgres:// URI or a keyword/value string (host=db options='…'). So the CLI takes that parameter out of the connection string and uses its value in place of the inherited PGOPTIONS, with the timeout added after it.

autumn migrate down does not use the lock timeout yet: a rollback waits for its locks as before.

Run migrations against Postgres directly.

Wrapping an external migration process

If you invoke an external migration tool (e.g. a raw diesel subprocess) and want it covered by the same advisory lock, use hold_migration_lock:

use autumn_web::migrate::{hold_migration_lock, DEFAULT_LOCK_WAIT_TIMEOUT};

let _guard = hold_migration_lock(database_url, DEFAULT_LOCK_WAIT_TIMEOUT)?;
// Lock is held for the lifetime of `_guard`.
// Run external process here …
// Lock is released when `_guard` drops.

This is exactly what autumn migrate does internally before shelling out to the diesel CLI.

Non-Postgres backends

Advisory locks are a PostgreSQL-specific primitive. SQLite and in-memory test harnesses do not support them.

• SQLite / in-memory — These backends are single-process by nature and do not need cross-process serialisation. Call run_pending directly; no lock is acquired or needed.
• Tests using TestDb — TestDb starts a real Postgres container, so the advisory lock is acquired normally.

If you write tests that call run_pending_locked against a non-Postgres database the connection will fail before the lock query is issued, and the function returns MigrationError::Connection.

Content checksums — never edit an applied migration

Autumn records a SHA-256 of every migration's up.sql in a framework-owned table, autumn_migration_checksums, the first time that migration is applied. Before every subsequent apply (both autumn migrate and startup auto-migrate) each applied migration's on-disk up.sql is re-hashed and compared against the recorded value. A mismatch means the migration was edited after being applied — the deployed schema silently forks from what a fresh build would produce — so the run refuses to continue with a fail-fast error:

migration 20260101000000 checksum mismatch: recorded <hex-a> but on-disk
content hashes to <hex-b>. Migrations must never be edited after being
applied — add a new migration instead, or run the documented re-baseline
command if this change was deliberate.

The rule is simple: never edit an applied migration. Add a new migration that expresses the change instead — that keeps every environment reproducible from the same source tree.

The same guard also catches a migration that was deleted or renamed after being applied: if a version has a recorded checksum but its up.sql is gone from the source tree, a fresh database would no longer run it, so autumn migrate refuses to continue (missing in status). The remedy is the same — never delete or rename an applied migration; add a new migration instead.

Note: startup auto-migrate validation is best-effort and requires the migrations/ directory to be present on disk at runtime (production binaries often ship without the source tree, in which case startup validation is skipped rather than failing); authoritative enforcement is autumn migrate / autumn migrate status in CI or your deploy job, which check against an explicit migrations directory.

Startup auto-migrate only validates on-disk content against recorded hashes; it does not record new checksums. It applies the embedded SQL compiled into the binary, which may differ from the on-disk files, so recording those disk bytes could store a hash for content that was never applied. Authoritative recording happens via autumn migrate and autumn migrate baseline, where the applied bytes are exactly the on-disk up.sql. An app that only ever startup-auto-migrates will therefore leave its migrations unrecorded until one of those CLI commands runs.

Line-ending and trailing-whitespace differences are normalised before hashing (CRLF/CR/LF all collapse to LF, then trim_end()), so a Windows checkout and a Linux one produce identical checksums and an editor that adds or removes a final newline doesn't trip the guard.

Legacy migrations: the unrecorded state

Migrations that were applied before the checksum table existed (or before the app upgraded to a version that tracks them) show up as unrecorded — no recorded checksum, no error. Record their current hashes with:

autumn migrate baseline

The command is additive and idempotent: it records checksums only for applied-but-unrecorded versions, so it is safe to re-run at any time. After a successful baseline, every applied migration is in either the ok or unrecorded state; a subsequent edit to any of them will flip to changed on the next autumn migrate and fail with the message above.

Re-baseline escape hatch (deliberate edits only)

If you have deliberately edited an applied migration and accept that other environments running the previous content will now report a mismatch, use:

autumn migrate baseline --force <version>

This overwrites the stored checksum for that single version with the current on-disk hash. It is logged at WARN so the change is unambiguous in deploy logs. Never the default — the --force <version> flag is the deliberate signal that you understand the consequences.

Rolling a migration back clears its checksum

Rolling a migration back (autumn migrate down) clears its recorded checksum, so it can be re-applied cleanly — including with changed contents. Because the version is no longer applied, editing its up.sql and re-running autumn migrate records the new content's hash from scratch rather than tripping the drift guard on a stale one.

Checking status

autumn migrate status reports each applied migration's checksum state:

• ok — the current on-disk up.sql still matches the recorded hash.
• changed — the migration was edited after being applied; autumn migrate will refuse to run until this is resolved (add a new migration or re-baseline).
• missing — the migration is recorded as applied but its up.sql is gone (deleted or renamed after being applied); autumn migrate will refuse to run until this is resolved — add a new migration instead.
• unrecorded — legacy migration with no stored hash; run autumn migrate baseline.

autumn migrate status is read-only: it never creates the checksum table and needs no write or DDL privileges, so it is safe to run against a database with a read-only role. On a database where the checksum table does not yet exist, every applied migration simply reports unrecorded. The table is created by the framework migration (or lazily on the first autumn migrate / autumn migrate baseline that records a hash), all of which do require write + DDL privileges.

Log output

Set RUST_LOG=autumn_web::migrate=debug to see the full waiting timeline.