When an agent or user needs something measured before it is changed — test suite runtime, gate throughput, memory pressure, worker concurrency, or a suspected regression. Also use when the user says "why is this slow," "how long does this take," "did that make it faster," "the box is thrashing," "how many can we run in parallel," or "measure this." Use this whenever a number is the deliverable. For deciding what to test, see test-strategy. For a live outage, see incident-response.
Perf Audit
You measure first, and you name the binding constraint rather than the
busy-looking one. Both of the largest wins in this repo came from correcting a
wrong belief about where the bottleneck was.
The measured baseline
The full suite went 518s → 324s → 87.3s on the same tree, all green, from
two changes in one.ie/web/vitest.config.ts:
environment defaults to node, not jsdom. Only ~230 of ~1135 files
touch a DOM; ~900 were each paying ~340ms to build a browser they never used.
One 19-file slice: 7.21s of jsdom for 141ms of assertions.
maxWorkers 4 → 8. The cap existed only because each fork held a
~120MB jsdom. Removing jsdom removed the memory driver.
The second was only possible because of the first. That is the shape of most
real wins here: the constraint moves, and the old tuning becomes wrong.
The loop, in order
Measure the right clock. Use vitest's own Duration, never the elapsed
time of a gate-run.sh call — an 8-fork run once showed a >20-minute wall
clock that was governor queueing, not runtime. Every number you report
carries the clock it came from.
Ask which resource actually runs out. The fleet capped concurrency at
cores - 2, authorising 8 worktrees on a 10-core/24GB box — ~16GB of gates
before editors, sessions and the OS got a byte. Cores were never binding;
memory was. Price the unit that runs out first.
Read swap by direction. Swapins spike during recovery too, as freed
memory lets pages fault back in. Thrash is pages going out while swap
usage grows and free memory is scarce. The swapin counter alone cannot tell
you which you are looking at.
Count what sits outside the governor. Each /do cycle drops a ~115MB
worktree, and editors index gitignored paths — 16 language servers holding
2.3–3.0GB from 3 worktrees, none of it visible to the gate governor. A budget
counting only gates is wrong by more than the gates.
Re-derive the premise before defending the structure. The fast lane was
designed when the suite was 518s. It is 87s now. "Too slow to run" is no
longer the argument — the invariants are.
Hard rules
- Do not narrow
test-cached.sh's key. It hashes git diff HEAD across the
whole repo. An extension filter looks like a free win and is not safe: these
tests readFileSync across pay/, channels/, schema/, .claude/ and
text/ at runtime, so an import graph is not a superset of what a test
reads. Fix the reaching, not the key.
- A broken sensor must never silently serialise the fleet. A probe that
cannot read memory returns a high number, never zero.
- A reused cache hit is reported as reused, never as a fresh pass.
Cannot run
A measurement taken on a box that was already thrashing is not a slow reading —
it is no reading. An 8-fork run once showed a >20-minute wall clock that was
governor queueing, not work, and reporting that as a duration would have sent
someone optimising the wrong thing for a week.
Say cannot-run when there is no baseline to compare against, when the clock
you have is a wrapper's elapsed time rather than the tool's own, or when the
machine was contended enough that the number is about the box. Return
{ ok: false, reason: "<which, and what would make it measurable>" } and close
with warn. Never name a binding constraint from a number you do not trust.
Out of scope
- Changing behaviour to win a number — a faster wrong answer is not a win.
- Declaring a release shippable — see release-gate.
- Triaging a live outage — see incident-response.