← Skills

perf-audit

perf-audit

When an agent or user needs something measured before it is changed — test suite runtime, gate throughput, memory pressure, worker concurrency, or a suspected regression. Also use when the user says "why is this slow," "how long does this take," "did that make it faster," "the box is thrashing," "how many can we run in parallel," or "measure this." Use this whenever a number is the deliverable. For deciding what to test, see test-strategy. For a live outage, see incident-response.

Perf Audit

You measure first, and you name the binding constraint rather than the
busy-looking one. Both of the largest wins in this repo came from correcting a
wrong belief about where the bottleneck was.

The measured baseline

The full suite went 518s → 324s → 87.3s on the same tree, all green, from
two changes in one.ie/web/vitest.config.ts:

  • environment defaults to node, not jsdom. Only ~230 of ~1135 files
    touch a DOM; ~900 were each paying ~340ms to build a browser they never used.
    One 19-file slice: 7.21s of jsdom for 141ms of assertions.
  • maxWorkers 4 → 8. The cap existed only because each fork held a
    ~120MB jsdom. Removing jsdom removed the memory driver.

The second was only possible because of the first. That is the shape of most
real wins here: the constraint moves, and the old tuning becomes wrong.

The loop, in order

  1. Measure the right clock. Use vitest's own Duration, never the elapsed
    time of a gate-run.sh call — an 8-fork run once showed a >20-minute wall
    clock that was governor queueing, not runtime. Every number you report
    carries the clock it came from.

  2. Ask which resource actually runs out. The fleet capped concurrency at
    cores - 2, authorising 8 worktrees on a 10-core/24GB box — ~16GB of gates
    before editors, sessions and the OS got a byte. Cores were never binding;
    memory was.
    Price the unit that runs out first.

  3. Read swap by direction. Swapins spike during recovery too, as freed
    memory lets pages fault back in. Thrash is pages going out while swap
    usage grows and free memory is scarce. The swapin counter alone cannot tell
    you which you are looking at.

  4. Count what sits outside the governor. Each /do cycle drops a ~115MB
    worktree, and editors index gitignored paths — 16 language servers holding
    2.3–3.0GB from 3 worktrees, none of it visible to the gate governor. A budget
    counting only gates is wrong by more than the gates.

  5. Re-derive the premise before defending the structure. The fast lane was
    designed when the suite was 518s. It is 87s now. "Too slow to run" is no
    longer the argument — the invariants are.

Hard rules

  • Do not narrow test-cached.sh's key. It hashes git diff HEAD across the
    whole repo. An extension filter looks like a free win and is not safe: these
    tests readFileSync across pay/, channels/, schema/, .claude/ and
    text/ at runtime, so an import graph is not a superset of what a test
    reads. Fix the reaching, not the key.
  • A broken sensor must never silently serialise the fleet. A probe that
    cannot read memory returns a high number, never zero.
  • A reused cache hit is reported as reused, never as a fresh pass.

Cannot run

A measurement taken on a box that was already thrashing is not a slow reading —
it is no reading. An 8-fork run once showed a >20-minute wall clock that was
governor queueing, not work, and reporting that as a duration would have sent
someone optimising the wrong thing for a week.

Say cannot-run when there is no baseline to compare against, when the clock
you have is a wrapper's elapsed time rather than the tool's own, or when the
machine was contended enough that the number is about the box. Return
{ ok: false, reason: "<which, and what would make it measurable>" } and close
with warn. Never name a binding constraint from a number you do not trust.

Out of scope

  • Changing behaviour to win a number — a faster wrong answer is not a win.
  • Declaring a release shippable — see release-gate.
  • Triaging a live outage — see incident-response.