(motir-core) Decide the queue foundation — Graphile Worker vs pg-boss vs hand-rolled, judged against the step-shim requirement
Everything else in this story is shaped by this answer, so it is settled first, in writing, with evidence.
The question
What does the Postgres job engine build ON — Graphile Worker, pg-boss, or a hand-rolled queue?
The options, and the axis that actually separates them
| Graphile Worker | pg-boss | hand-rolled | |
|---|---|---|---|
| storage | Postgres only | Postgres only | Postgres only |
| wake mechanism | LISTEN/NOTIFY | polling + notify | ours |
| cron | built in | built in (schedule) | ours |
| debounce / singleton | no | yes (singleton / throttle) | ours |
| retries + backoff | built in | built in | ours |
| durable steps | no | no | no |
The last row is the point. No candidate implements memoized steps or a durable sleep — so the step shim is OURS to build under every option, and the choice is only about what sits UNDER it: a claim loop, a schedule tick, and a retry policy. That reframes the decision from "which framework" to "how much of a well-understood loop do we want to own."
What to produce
A decision record under docs/decisions/ naming the choice and the evidence, structured as this corpus's other ADRs are. It must answer:
- The choice, with a rung-1 reading — what mature tools in this space actually do — rather than a preference.
- Whether the candidate's own tables can carry
job_step, or whether we add our own beside them. A library whose schema we must extend is not obviously better than no library. - How its claim loop interacts with
FOR UPDATE SKIP LOCKEDand with more than one worker, since that is what the worker subtask is built on. - Its licence, checked from the installed package rather than the project page — the same discipline that found Inngest's server is SSPL while its SDK is Apache-2.0.
- What it costs to leave. We are migrating off a substrate right now because leaving was expensive; a candidate whose abstractions leak past
lib/jobs/repeats that.
Acceptance criteria
- A
docs/decisions/record names the choice, the rejected options, and the reason each was rejected — with the reason for the winner stated as evidence, not preference. - The durable-step gap is stated explicitly: whichever is chosen, the shim is ours, and the record says so, so no later card assumes the library provides it.
- The chosen package's licence is read from the installed artifact and recorded.
- No candidate that requires Redis or any other new service is admitted — that constraint is the epic's premise and the record must show it was applied, not assumed.
- The record names which subsequent subtasks it settles: the schema, the worker and the shim all read it.
- No production behaviour changes in this card.
Context refs
lib/jobs/defineJob.ts— the option surface any candidate must be able to express (retryPolicy,concurrency,idempotency,debounce,cron)lib/jobs/definitions/codeGraphRefresh.ts— the onlydebounceuser, and the hardest option to reproducedocs/decisions/job-lane-occupancy.md— why this migration is happeningdocs/decisions/— the ADR shape and header convention to follow