Motir

Found by · MOTIR-3963, executing its own measurement · Repo · motir-meta (the rule's home, if one is owed) · Correction · already applied on MOTIR-3963 — this is the RECORD

What went wrong

MOTIR-3963 did an unusually careful job of pre-committing its decision rule, and said why:

"This is written down first on purpose. The failure mode for a measurement like this is picking the rule after seeing the numbers."

The rule it committed to:

Let M12 = median across ≥5 twelve-leg runs of that run's slowest vitest step. Let M8 = the same over the eight-leg-with-packer runs. · M12 ≤ 0.9 × M8 — twelve wins · M12 ≥ 1.1 × M8 — revert · Within ±10% — REVERT TO EIGHT.

The defect: M8 and M12 are measured in different weeks, and nothing in the rule holds the workload constant. M8 was sampled 2026-08-29/30; the M12 sample could only exist days later, by the card's own design — it required five more twelve-leg runs to accumulate. So the time gap is not an accident of execution, it is built into the card's readiness condition.

In that gap the suite grew: 1447 → 1515 unit/integration test files, and +17.6% in measured worker-seconds per run (4348 s → 5112 s median). That is enough on its own to erase the improvement:

So the rule's tie-break fired on suite growth rather than on twelve legs failing. And it fired against its own written justification — the tie-break's stated premise is "a wall-clock gain that, by hypothesis, did not materialise", while the gain plainly did materialise. Obeying the branch would have made CI slower: the worst leg would go ~617 s → ~729 s, an ~18% regression on the exact quantity the card exists to minimise.

Controlling for workload with a within-run dimensionless statistic — worst / (total/legs), immune to drift because it is computed inside each run — gives M12/M8 = 0.865, which fires the KEEP branch. Scaling M8 up to today's workload agrees independently: 0.846.

Why nothing caught it

• The card's own controls were good and were the wrong axis. It demanded a runner-quality control (migrate deploy, ≈3 s healthy vs 6–8 s degraded) and it worked perfectly — every M12 outlier was a degraded runner, and the packer was exonerated on all 15 runs. A workload control is a different axis and was not asked for.
• likely-over-gate-sizing and every advisory tier are silent — the card was 2 points / 40 minutes, well-formed, valid: true, one unrelated subsumption advisory.
• The card pre-committed HARDER than usual, and that is what made it dangerous. A rule announced as un-overridable, with a written warning against rationalising it away, is one a careful runner will follow past the point where its inputs stopped being comparable. The better the pre-commitment prose, the higher the cost of the unexamined premise.

What would have prevented it

A standing check, stated at the point where a card WRITES a comparison rule:

A criterion that compares a BEFORE measurement with an AFTER measurement owes a statement of what is held constant between them — and where the two samples are separated in time, the card must name the drift it is exposed to and carry a control for it (a within-run normalisation, a re-measured baseline, or an explicit same-day A/B). A ratio of two medians taken in different weeks is a measurement of the configuration and of everything else that moved.

Corollary, and it is the sharp half: where the card's own readiness condition forces the gap ("do not run this until N more runs exist"), the drift is guaranteed rather than possible, and the control is mandatory.

Totality measured, not asserted — the check exists in none of its three homes:

grep -rai -e workload -e 'like-for-like' -e 'comparable population' -e 'baseline drift' \
     prompts/plan-rules/ prompts/run.md prompts/plan-procedure.md   -> 0
grep -ai  -e workload -e 'comparable' motir-ai/src/llm/planningRulePacks.ts
     -> 2 hits, both "every comparable tracker", unrelated

Disposition — NOT promoted here, and the close-out settles it

This is ×1, and _shared.md's bar is a pattern that recurs. A closely-related GLOBAL lesson already exists and was RETRIEVED by this run's search_lessons — "Check that your instrument answers the question you asked" (cmk95j5fs0ramupd2odis2kks): "Measurements get adopted as answers to questions they were never able to address, and the substitution is invisible because the number is real." That is this defect at one altitude up, and it has been reinforced against MOTIR-3963 rather than restated as a new rule — per before widening a rule, find the rule.

The W-series stands at W10 (MOTIR-3013); the nearest sibling, W9 (MOTIR-2987) — a defect card's mechanism owes the DISCRIMINATING measurement — is the same family but a different pack (kind-bug.md) and a different check.

So this card's deliverable is to SETTLE the promote question, not to assume it. Evidence for promoting despite ×1 is the totality gap above; evidence against is that one incident is a lesson by definition and the existing global lesson already fired correctly. Decide on the evidence at close-out, and if the answer is SHARPEN, file W11 as ONE task carrying targetRepos: ['motir-meta','motir-ai'] — re-reading MOTIR-1464's not-done children first, since a duplicate W card splits the warrant of the rule it strengthens.

Acceptance criteria

• The correction on MOTIR-3963 is verified as held: its comment carries both the raw and the workload-matched ratios, and .github/workflows/ci.yml on origin/main still reads leg: ['1'…'12'] with a matching --shard=…/12 denominator.
• The promote-vs-lesson question above is SETTLED on stated evidence — re-run the three-home grep, and re-read MOTIR-1464's not-done children for an existing owner — and the answer is recorded on this card either way.
• If SHARPEN: W11 exists as one task under MOTIR-1464 with the repo set, and this card cites it. If LESSON: say so, and cite the reinforced lesson id.
• No move_to_sprint — MOTIR-1465 carries none and this is not build work.

Out of scope

• Re-running the measurement. MOTIR-3963's numbers stand; this card is about the RULE that read them.
• Runner degradation (53% of twelve-leg runs drew at least one degraded runner). Real, larger than the leg-count question, and explicitly out of MOTIR-3963's scope too.