Each worker caps its own claims with max_concurrent_claims. A shard can still see workers × max_concurrent_claims claim statements at once. Claim slots cap that number across the whole fleet. The cap is off by default.

Turn it on

Rust
use autumn_harvest::builder::WorkerConfig;

let config = WorkerConfig::default().with_claim_slots_per_shard(8);
  • 0 turns the cap off. The largest value is 1024 (builder::MAX_CLAIM_SLOTS_PER_SHARD). Validation rejects a larger one.
  • Set one value on every worker of a fleet. A worker with no cap takes no slot, so a mixed fleet can run more claims than the cap.
  • GET /admin/config reports the value as claim_slots_per_shard.

How it works

  1. A claim transaction first takes one of N advisory locks with pg_try_advisory_xact_lock. It tries each slot once, from a random start.
  2. A claim that gets no slot sends no claim statement. Its loop tries its next shard in the same pass. A worker on one shard waits for its next wake.
  3. The lock ends with the transaction, at commit or rollback.

The slots belong to the harvest_task_queue table of the shard. Shards that share one database share that table, so they share the slots. Two schemas in one database each have their own table and their own slots.

The cap applies to the claim that scans the backlog. A claim by id from a Redis dispatch channel reads one known row. So does an owner take. Neither takes a slot.

The lock never waits, so it cannot deadlock with the locks of the claim. See lock-order row 20 in docs/architecture.md.

Watch it

harvest.db.claim_slot_miss{shard} counts the claims that got no slot. A steady rate on one shard means that the fleet runs more claim loops than the shard has slots. That is the cap at work. Raise the slot count only if the claim latency of the shard stays low (harvest_db_query_duration{op="claim"}).

A claim that misses still holds a pool connection for one short statement. Size the pool as before.

Choose a value

A cap below the claim loops of the fleet lowers the claim rate of an uncontended shard. See docs/performance.md § "Claim slots" for a measurement. Use the cap when claim latency rises with the number of claimers, for example after a VACUUM or a lock stall.

See also

  • claim-batch.md: one claim statement takes up to 2, 4 or 8 rows. A batch takes one slot per statement.