Machine-dependent tests — annotate, measure, re-check
Record each machine-dependent failure: test, machine, evidence, disposition and re-check date. Keep assertions; a missing prerequisite means unverified, not green. Record rerun timing beside the first failure. Fix the class and prove a reverted fix fails. Fixtures own scratch directories, HOME, config and children; see hermetic gates.
Register
| Test / check | Machine class | Evidence (log / SHA) | Disposition | Re-check |
|---|---|---|---|---|
openapi:check:dry | Concurrent worktrees | #3037: a passing check restored another tree’s global /tmp backups | Generate privately and compare with the index; never write or restore checkout files. Pipe-ordered worktrees and original symlink/hardlink witnesses preserve bytes and inode identity during generation and on failure/cancellation. Full gates still run singly. | 2026-11-11, and on generator changes |
| Background-session no-wait witnesses | Loaded ARC runner; fake CLI startup counted in the session duration | #3008: correct result at 1.509 s against 1.5 s; a startup-only delay reproduces it at 1.661 s | Twelve no-wait ceilings become explicit-clock close/idle decisions while retaining the real CLI scenarios. Inventory and proof boundary; the fourteen duration-budget assertions remain separate. | 2026-11-10, and on background lifecycle decisions |
TestConnectWaitsForABrokerThatIsNotUpYet, TestWaitForBrokerIsBounded… (pkg/queue/nats) | Loaded ARC merge-group runner | run 41687370688, #3010: calls = 1; 31 attempts, last an i/o timeout | Corrected, verified 2026-11-04: loop, Connect and store twin in synctest; Connect needs natsgo's verdict, timeouts past the attempt, exact budget. Mutants redden them | 2026-11-11 |
TestEngineRun_IsolatedSubbotSiblingsOverlap | Concurrent isolated children registering worktrees in one repository | #2670; an empty commondir in a pending registration reproduces Git's exact refusal | Runtime setup, reconstruction and cleanup serialize their Git registry commands in-process, keyed by the physical common directory. A virtual-time witness holds an incomplete registration, checks main/linked/symlink spellings and an independent repository, then releases it; removing the gate or making it global fails it. The three-child overlap witness is unchanged. | 2026-11-04, and on runtime worktree lifecycle changes |
TestRunSpreadCoverageStrictCheck/a_Forbidden_plus_an_expired_deadline_still_names_the_Role | Loaded ARC test-unit runner; shell startup races a 200 ms probe | #2571, PR run | spreadCoverageFailureCause decides from the explicit cluster and probe errors; the witness stages both, other rows keep real kubectl shims (Forbidden, timeout, node coverage). Reversing the precedence or dropping the probe error at the caller fails it. Budget and Loki cases already repaired by 20e4aeb4c, 5ee792110; ARC tracked in #2564. | 2026-11-04, and on doctor diagnostic changes |
TestShutdown_DrainDoesNotLeakAgainstASerializingTracker | Loaded ARC merge-queue runner under -race | #2072, merge-group failure: 1/32 claims leaked despite the deadline-bucket assertion | Virtual time drives the serial tracker; the native store's kernel watcher stays outside the bubble. Refresh and release must each enter with their whole budget, every call must have a deadline, and all 32 real cards must return to Ready unclaimed. Moving the revert budget before admission, dropping its bound, or skipping release fails it. | 2026-11-04, and on drain budget changes |
TestDeliveryProbeOnlyAWellFormedAnswerCanHold | Parallel ARC socket fixtures | #2745 merge-group failure: a non-redirecting row's target received a request; the reservation witness proves the old refused address can be reclaimed | The shared deadAddress fixture owns a bound, non-listening socket until cleanup: no other fixture can claim it, the kernel still refuses connections. Every verdict, retry-count and zero-redirect assertion stays. | 2026-11-03, and on probe fixture changes |
TestBackground_NoWrapUpInsideATurnTheCLIStarted | Loaded ARC merge-queue runner; fake CLI and stream reader scheduled independently | #2745 merge-group failure, commit b7f7f0ca8; a local 800ms reader delay reproduces it | Corrected, verified 2026-10-27: keeping the CLI round trip, the explicit-clock lifecycle fixture tests the expired timer before and after assistant text and at the turn's close; removing either in-turn guard fails it. Ten race repetitions and the delayed-reader scenario pass. | 2026-11-03, and on lifecycle timer changes |
TestLaunchDetached_NamesTheDroppedOverrides | Operator workstation with an installed iterion outside PATH | e40def071, including its mutation record | Corrected, verified 2026-10-24: proc.SetInstallPathCandidates makes the fixed install tier injectable; the witness empties that tier and pins HOME and ITERION_BIN. | 2026-10-31, and on binary resolver changes |
TestRepoRootFindsTheNearestRealGitEntry | Operator machine with a foreign, empty /tmp/.git, recreated by an unidentified process | Operator observation, #2585 mission, baseline e40def071; no archived log. Re-check 2026-10-24 (+ #2585 diff): reproduced in the jailed task check; empty, owner jo, mode 0555; rmdir: Device or resource busy, still red | Environment parasite. Inspect ownership/type/content; before a verdict remove only the confirmed empty directory with rmdir /tmp/.git, record the purge and rerun; an rmdir refusal stops the purge. Never delete it recursively, force-unmount it or weaken repository-discovery assertions. | 2026-10-31, and before the next affected verdict |
internal/ciguard's script guards — every helper that hands a script path to a child shell | Any machine with a warm GOCACHE, including a CI job that restored one | Measured 2026-10-29 mutating scripts/ci/baked-actions-manifest.sh for #2435: two mutants deleting a generator rule came back ok (cached) | go test caches on the files the TEST process opened, not a script a child shell reads: every such helper reads it through scriptUnderTest (refusing an empty one) before exec.Command, and both mutants then redden on their own subtests with a warm cache. -count=1 hides the class for one run only | 2026-11-05, and whenever a new guard shells out |
🧪 Tests workflow, queued jobs | Saturated ARC runner pool | Operator observation, #2585 mission: runs queued for hours; no run URL | Infrastructure delay, no test verdict. On recurrence attach run URL, SHA, queued/start times and runner class; measure queue wait apart from execution (queue runbook). Missing/cancelled jobs never count as passing. | 2026-10-31, and at the next landing |
TestRun_aLeaderWhosePassesFindNothingToDoKeepsItsLease (pkg/lease) | Loaded ARC runner | 41236299236 (63aac5608), 0.93 s; also on #2732's mongo job; green locally ×5 | Corrected, verified 2026-11-02: the scenario runs in a synctest bubble — the 150 ms TTL and four idle TTLs are virtual, so the stall that reddened CI cannot starve the holder. Removing the renewal's deadline reset reddens it on its own "a peer ran N passes" assertion (#2854). | 2026-11-07, and on the renewal and step-down timers |
TestBrokenListSplicesStayLinear (pkg/dsl/parser) | Loaded ARC runner under -race | 41233597165, 3.42 s; 41236299236, 24.14 s; local ~1.9x | T41: the ratio reads min/min — load only inflates; a quadratic rescue scores ≥ 4x even at its best. | 2026-11-07, and on parser splices |
TestEvalWorkBudgetCountsNodeEvaluations (pkg/dsl/expr) | Loaded ARC runner under -race | 41236299236: refusal at 20.06 s vs 10 s; 1.45 s locally | T41: tripwire at 2 min — its target is the budget not biting (~1 h). The refusal stays asserted | 2026-11-07 |
TestAnInstallTakesTheLockOfEveryFileItWritesOrRemoves (pkg/botinstall) | Loaded ARC runner under -race | 41236299236: the lock sweep blew its 300 ms on a stalled shard; green locally ×3 | T41: test installLockWait at 2 s (production unchanged); the bounded wait and the 10 s watchdog are untouched. | 2026-11-07 |
TestSiblingBrokenListsParseInLinearTime (pkg/dsl/parser) | Loaded ARC runner under -race | 41243004752 (5058166bc): 6.43 s vs 5 s; ~150 ms locally | T42: bound at a minute — the quadratic regression it prices takes minutes at this size | 2026-11-07 |
TestBackground_ADeliveredResultNoLongerHoldsTheGrace (pkg/backend/delegate) | Loaded ARC runner under -race | main witness on the T43 tip dfd8c8db4: 4.15 s against the 4 s ceiling; green locally ×3 | T44: ceiling at 10 s — it prices the grace HELD past a delivered result, which parks the run for the 20 s result wait; the margin stays decisive. | 2026-11-08, and on the background-session grace |
signOutMachineTransitions (studio) — "holds the start while the named session is held" | Loaded ARC studio job | T43 witness: the OIDC landing listed 4 elements where 2 were expected (a 500-answering logout retried under load); the file passes 3× locally | Load-induced duplication in the landing's pending path, unreproduced. One occurrence; a recurrence gets a ticket and a query-dedup fix — the assertion stays. | 2026-11-08, and on the sign-out machine |
TestARefusedPropertyIsNotRecorded (pkg/dsl/parser) | Host running the full task check suite | T48 gate (2026-11-02): the quadrupling ratio measured 8.6x against the 8x bound with the table perfectly correct; green ×3 standalone | T49: min-of-3 on both sides, the repair T41 gave TestBrokenListSplicesStayLinear — load only inflates a min; a quadratic Set scores ~16x at best, so the 8x bound stays decisive. | 2026-11-09, and on parser property tables |
TestFanOutStopsOnUndecidedEffects (pkg/runtime) | Loaded ARC test-unit and race-shard legs | #2852: reds pre-fix 2026-10-29: fb16b094a (multi/deadline/late=false, test-unit leg), 7384a1494 (each/deadline/late=true, race-shard (0)); zero re-flakes in the 16 failed main runs since e9df14815, 14 tip SHAs (measured 2026-11-02) | e9df14815 (#2843): 2 s budget, a guard names a pre-branch loss; the machine-derived deadline the ticket asks for lands only if this re-flakes | 2026-11-09, and on fan-out deadline changes |
TestGuardFoundRefusesALiveIntruder (pkg/internal/proc) | Loaded ARC race shard; an exec in progress hides /proc/<pid>/environ from a scan for an instant, longer under load | run 40908455782 (7384a1494, 2026-10-29): guardFound accepted a live process this test did not start | Corrected, verified 2026-10-29 by 8d51e9dc6: the verdict reads a scan that provably held the intruder (afterGuardScan, premiseWait) and must name the pid the helper printed; both mutants redden on their own assertions. Zero re-flakes in the 15 failed main runs since it landed (via T34; measured 2026-11-02) | 2026-11-05, and on the proc scan guard |
TestSweepStopsAProcessThatShowsAfterAnEmptyScan, …ANonEmptyScan (pkg/internal/proc) | Loaded host, crowded PID namespace: a sh-program scan without grep -z takes 4.5–8 s | Local, load 45/102: premise: … [0 1 0 1 0]; 3/3 under 300 extra processes + 48 busy loops, sibling 2/2, f5de1b70f | Corrected, verified 2026-11-03: no deadline in a window's phase, waits count scans, empty-scan members must die of TERM; green 3/3 under that load. Drop mutants redden all arms | 2026-11-10, and on the sweep witnesses |
Process capture, checkout-root and installed-binary witnesses: process fixtures.