Record: TEST-CPU-CLASSES
Status: in force
Built: built
Decided: 2026-10-22 by the coordinator

Per-test limits are CPU-work classes plus one hang backstop

A per-test wall timeout judges the machine, not the test: a busy host makes a test wait for a core, it does not make it do more work. On 2026-10-22 the web suite’s required shard 2 went red on pull requests that never touched the failing test (declared-pattern-liveness, 17-30 s of wall against a 30 s wall limit). The cause was CPU work (it re-ran the whole public suite, ~32 s of CPU), and a wall limit could not tell that from a loaded runner. This carries out NO-LOAD-BLOCK (audit L2, row G-05) for per-test limits, under Q-TX2.

The judgement is CPU work. Each test’s CPU (its process, every thread, and every child it reaped) is read by the one in-repo reader (engine/src/shared/run-cpu.ts) and judged against its file’s class: testExecution.testCpuClasses declares defaultCeilingMs and named classes, each listing the files it holds, once, for both suites. A test over its ceiling fails with the work it did and the ceiling it broke. Wall time is recorded beside it and never judged. An unreadable reading is a refusal, never a pass.
The numbers were measured. A full web-suite run (7,270 tests, 2026-10-22, 12-core host) put every test at or under 15.2 s of CPU work except test-isolation-contract’s nested runner launches (up to 30.1 s). The default ceiling is 60 s (hosted runners spent about 1.3 times this host’s CPU on the same test).
One judgement, installed by each runner’s door. engine/scripts/tooling/test-cpu-class.ts holds it; the web suite installs it from tests/setup/test-cpu-class.ts (the reading is kept on each test as meta.cpuMs/meta.wallMs).
Wall time survives only as one hang backstop. testExecution.testHangBackstopMs (600 s) is vitest’s testTimeout and hookTimeout; a test that reaches it is reported as hung, with the CPU work it had done, and is never retried. It sits below the web stall watchdog’s bound (webSuiteWatchdog.stallTimeoutMs, raised to 900 s), so a hung test is named before its run is killed. A nested runner launch’s bound derives from it (tests/test-budget.ts).
A per-test timeout written in a test file is a wall judgement and leaves the tree. The web suite’s 225 (in 35 files) are removed, and any new one is refused AT RUN TIME by the suite’s setup file (tests/setup/test-cpu-class.ts), never by reading sources (a static reader always loses to the next spelling): every bound the runner arms for a test or hook must be the backstop, every test’s effective task.timeout and every hook of its file must be the backstop, and testTimeout and hookTimeout must still be the backstop before and after each test. A refusal fails the run naming the test. tests/prevention/test-cpu-class.test.ts plants each known spelling (aliased and namespace imports, spread, imported and call-built options, a wrapper in a helper, vi["setConfig"], hook bounds, a test with no bound, the DOM pool) in a real nested run.
The engine suite has the same door (NO-LOAD-BLOCK row G-05). Its per-test WALL cap (runtimeBudgetMs, 120 s, node:test’s --test-timeout) is deleted with its 33 per-test overrides: under full-suite contention it failed foundation-cascade, handoff-continuity and repository-path, which spawn three runtimes each and pass alone. The runner (engine/scripts/tooling/inventory-test-run.ts) preloads engine-test-bounds.ts into every test file’s process: the same CPU-class judgement, node:test’s per-test timeout set to testHangBackstopMs, and the same run-time refusal of any other bound. node:test arms each test’s and hook’s timer through timers.setTimeout, which the preload wraps before node:test loads, with a liveness witness on every test. engine/test/workflow/inventory-test-run.test.ts plants a test timeout, a hook timeout and a CPU burner through the real runner. The engine watchdog’s stall bound sits above the backstop (900 s) and its run ceiling is the schema’s largest (4 h): both end hangs, neither judges speed.
A class records a measurement, and its ceiling is derived from it. Each class names its file and measuredCpuMs: the most CPU work any of its tests did on a hosted runner. Its ceiling is that measurement times the one declared headroomFactor (2), because CPU work still varies with the runner’s processor model. One function derives the ceiling (classCeilingMs in engine/src/workflow/workflow-config.ts, which the judgement and the exposure page both call), and no class carries a hand-picked ceiling. The measurements come from #490’s CI run (2026-10-28):
- The engine run (10,800 tests) had 13 tests over the 60 s default: commit-guard-generated-facts 435.2 s, candidate-checkout 258.5 s, S1 b12 195.0 s, S1 b02 176.4 s, foundation-cascade 159.5 s (FC-E002; FC-E001 63.4 s), S1 b15 135.1 s, repository-path 115.7 s (RPA-O003), S1 b08 89.4 s, handoff-continuity 84.5 s (HCF-O003), S1 b13 74.5 s, harness-commands 65.2 s and architecture-baseline-clean 60.0 s.
- The web shards’ heaviest tests were test-cpu-class 40.8 s and test-isolation-contract 39.6 s. Both run nested runners, and they replace the hand-set 180 s nested-runner class. The next heaviest web test did 25.4 s.
- Every test over about 120 s of CPU work has a performance issue for its owning lane. The class unblocks the suite; the issue makes the test cheaper.
Every engine runner runs under the same bounds. The targeted door (engine/scripts/tooling/test-guards.ts: the engine’s test:unit script, and harness flake reproduce, which runs there) installs the suite’s own bounds, built by the one construction (engineTestBoundsFrom in engine-test-bounds.ts), for the files it runs, with the backstop as node:test’s per-test timeout. A run naming no test file is refused (node would otherwise discover files the bounds do not cover), and so is a folder (node does not expand one). Before, a targeted run had no CPU judgement and no backstop, so a CPU-class failure could never be seen or reproduced there (#615).
Every test has the headroom, not only the classed ones (#615). A test with no class sits under the default ceiling with whatever headroom it happens to have: S1-B09-HOSTILE-REAL did 60,207 ms against the default’s 60,000 ms on one hosted run. So the run records each test’s CPU work, and the engine worker prints a [cpu-headroom] line for every test over its class ceiling divided by headroomFactor (30 s for the default). Each line is answered by a measured class for that file, from the reading in the line. The line is a signal and never fails the run: a failing line at half the ceiling would only move the flake to that line. B09 now has its class (s1-b09, 60,207 ms).
Concurrency and threads. The judgement keeps one reading per test context, never one per process. A nested or concurrent test therefore never clears another test’s window: a parent counts its subtests’ work, and concurrent tests each count the work done in their window. Both over-count and never under-count. The engine preload installs only in the dispatched test file’s own main thread. A worker thread or a child node started with the inherited options is the test’s own work.