E2E coverage matrix
The single feature×coverage inventory for iterion. One row per operator-observable promise, at the granularity a regression report would name it. Every row is terminal (covered-deterministic / covered-live / unit-only / excluded) or an honest uncovered gap with the plan.
A citation must be the test that would FAIL if the promise broke. The deterministic gate greps every reference, so it catches a citation that does not exist — it cannot catch one that exists but tests something else. Two adversarial audits sampled 70 rows and found 11 such mis-citations (mostly: a row citing the mechanics of a helper while the WIRING that invokes it went untested). When you touch a row, re-read the cited test and ask what its failure would actually mean.
This file supersedes the three partial coverage docs it reconciles:
- docs/e2e_coverage.md — the iterion×claw-code-go live matrix + the deliberate claw-side gaps (folded into the
backends,tools-mcpandobservabilityfamilies below). - docs/live-e2e-coverage.md — the opt-in real-LLM layer (how it works, the quality-judge panel, the per-bot targets). Still the reference for running the live layer; its bot/feature tables are mirrored here as
covered-liverows. - e2e/SCENARIOS.md — the flagship-workflow stub-executor scenarios (folded into
runtimeanddsl).
Legend
| Status | Meaning |
|---|---|
covered-deterministic | A CI-runnable, credential-free test drives the real seams and asserts observable outcomes |
covered-live | Only exercised in the opt-in live-tagged layer. Its Notes open with the gate (#2794): gate: paid (a real credential / model turn — money) or gate: tool (tool-gated: a real external tool, no credential, no money) |
unit-only | Deliberately terminal at unit level — an e2e would only re-test the harness |
excluded | Not exercisable in this repo's harness (needs a third-party tenant / cloud control plane) |
uncovered | Real gap — backlog |
What "the front door" means per family
- dsl — the front door is
.botsource text: parse → compile → the diagnostic or IR an operator sees fromiterion validate. Rows whose behaviour only shows at execution time cite a runtime/e2e test instead. - runtime — the public engine API (
runtime.Engine.Run/Resume) against a realstore.RunStore, asserting persisted status, checkpoint, artifacts and the event stream.e2e/additionally enters through.botfixtures. - cli —
cli.Run*entry points with a real store and (where an LLM would be involved) a stubruntime.NodeExecutorinjected at the documented seam. - server-api / cloud —
httptestagainst the real wired handler. - reliability — the invariants of #1006, each entered through the layer that OWNS it: the engine's admission gate and correction loop, the producer's repair ladder, the rollout resolver and the baseline reader. A row is
covered-deterministiconly on a test that refuses the invariant's violation; see ADR-152 for the ownership map and the slices the open rows owe. - studio-ui — Playwright (
studio/e2e/,task test:e2e:ui) drives a real Chromium against the REAL server: the builtiterionbinary serving the embedded SPA over a throwaway store thatstudio/e2e/serve.mjsseeds with genuine artifacts — runs the engine actually executed (tool + compute fixtures, no LLM credential) and a board card created through the CLI. The suite asserts rendered content and interactions, never HTTP status codes. Its CI execution lane is studio-e2e-advisory.yml: Mondays at 05:30 UTC and manual dispatch, on the same non-blocking pattern as a11y-advisory. The job installs Chromium, builds the embedded SPA and binary, runs the whole spec directory, and uploads the HTML report and failure traces. A failure makes this advisory red; it does not block a merge. The local browser download remains opt-in. Without it the task prints SKIPPED: no Playwright Chromium browser installed; the Studio UI e2e suite did not run. CI treats that absence as an error. A successful local task exit after that message is not a browser pass.
Studio browser coverage and evidence (#1733, #2356)
covered-deterministic describes the test layer, not the latest execution result. Previously these browser citations had no CI lane, and a local skip could be mistaken for coverage. The advisory above now supplies the lane; its run report is the execution evidence. Until the workflow is landed and has run, no scheduled green result is claimed.
At this revision there are 13 spec files, covering 12 browser-backed matrix rows, not 13 executed rows. assistant-demo.spec.ts is the thirteenth file: it explicitly skips unless ITERION_COPI_DEMO=1 (see COPI_DEMO.md). The separate studio-ui.dsl-vocabulary row is deterministic unit/Go coverage, not a Playwright row. All 13 files are discovered by the advisory; the opt-in demo skip remains visible in its report.
The #1733 product decision keeps the per-node ModelPicker compact: the registry options carry context and price, and the inherit option names the authored model. The launch spec checks that model's actual option and selects it. It no longer asks for ModelCapsCaption, which belongs to the fallback input, or restores a second fetch/caption per node after the registry-picker change in 6eea0ce899. The fixture server pins ITERION_MODEL_SPECS=off and CLAW_DISABLE_LIVE_REGISTRY=1 so neither registry's network/cache changes the curated context or introduces a price. Unknown prices remain unprinted. The run-console spec matches the exact log event Run finished (the conversation's closing card has a longer message) and the log search field's accessible name Search log.
To run locally, use devbox run -- task test:e2e:ui:install, then devbox run -- task test:e2e:ui. Browser installation needs access to the Playwright CDN; execution needs permission to launch Chromium and bind the loopback test server. The 2026-10-25 implementation session verifies discovery (39 tests in 13 files) and the named-skip path, but cannot claim a browser pass: Chromium is absent, its download fails with ENOTFOUND cdn.playwright.dev, and a system-Chrome probe cannot bind 127.0.0.1:4899 (socket: operation not permitted).
The three design-system drifts are corrected in design-system.md: the loading example, the loading entry in the Don'ts table, and the Settings dialog described as a route. #2356(c), the contrast-token mirror, is already enforced by studio/src/__tests__/a11y/tokens-mirror.test.ts; no new mirror is needed.
Regression witnesses and how to falsify them:
run-console.spec.ts: restore the broadRun finishedlocator to expose the log/card strict-mode collision; change the exact log event text to prove the card cannot satisfy the assertion. RestoreSearch log…as the accessible name to fail when filling the log search field.launch.spec.ts: remove the authored model from the registry options or change its context to128K; its unique option/context assertions must fail. Print a$0price for unknown data to fail the no-price assertion. Re-enable either registry with a differing cached context/price to expose why the fixture pins both offline switches.guard.test.mjs: return zero for a missing browser, remove the named no-run message, or suppress the CI annotation/summary; each breaks the missing-browser witness. ForcePLAYWRIGHT_BROWSERS_PATHto an empty directory to exercise the real task's local skip and its CI failure.- Advisory lane: on manual dispatch, break a Playwright assertion to get a red job plus retained report/trace; remove the browser installation on a clean runner to get the explicit no-run error. These CI probes require a published workflow and are not claimed as executed locally.
- Existing design-system nets: render an
EmptyStatewith raw loading text to failsource-discipline.test.ts; change a mirrored token inapp.csswithout updating_tokens.tsto failtokens-mirror.test.ts.
Scope of the current campaign
Pass 1 built this inventory and closed the highest-value CLI quick wins (run --preset, run --max-*, secret, runs questions|answer, report). Pass 2 closed the remaining CLI quick wins: issue, diagram, skill, memory, run --recipe. Pass 3 closed the four CLI rows earlier passes had written off as needing a seam or a live model — the cli family is now terminal end to end. Each turned out to have a deterministic front door:
run --model/--backend— drive the REAL executor at fixtures whose backend/model cannot resolve; the failure text quotes whichever value won the resolution chain, offline.run --auto-resume— the loop's only wait is the reset-aware USAGE_LIMIT_BLOCKED delay, and the run's own retry policy clamps it, so a 1ms horizon removes the sleep without a clock seam.bench asymptote— the command re-runs nothing; stub-driven runs of a judge-in-a-loop fixture supply the exact events it parses.dispatch— boot the daemon in-process on a loopback port, assert the served surface, stop it with SIGTERM.
Everything still uncovered is deliberate backlog for later scoped runs, family by family. What is left, and why it is not a quick win:
Pass 5 introduced the browser harness the studio-ui family was waiting on and closed its rows (see the family note above).
Pass 4 closed the three remaining cloud rows — dlq, migrate-blobs, valkey-state — which earlier passes had written off as needing a live Mongo/S3/Valkey. Each had a deterministic front door after all:
valkey-state— two servers built through the realNew()wiring over one miniredis; each flow starts on replica A and finishes on replica B.migrate-blobs— the walker between two real stores, plus the real AWS SDK against an in-process path-style S3 gateway.dlq— the admin REST surface behind a narrowQueueBackendseam; the JetStream-side semantics are covered separately by the live-brokernats-conformancejob (TestSchemaRolloutMixedFleet).
| ID | Feature | Family | Status | Tests | Notes |
|---|---|---|---|---|---|
| dsl.node-agent | agent node: LLM node with structured I/O executes and publishes | dsl | covered-deterministic | TestSingleModel_HappyPath (e2e/e2e_test.go) | |
| dsl.node-judge | judge node: verdict-producing LLM node | dsl | covered-deterministic | TestSingleModel_HappyPath (e2e/e2e_test.go) | |
| dsl.node-tool | tool node: direct shell command, no LLM | dsl | covered-deterministic | TestExecutorToolNodeShellCommand (pkg/backend/model/executor_test.go), TestToolNodeShellMaterializesSecret (pkg/backend/model/secrets_materialize_test.go) | the shell path needs the REAL executor: the e2e scenario stub replaces every node — agent and tool alike — so a tool node rerouted through the LLM path would still pass there |
| dsl.node-compute | compute node: deterministic expression output | dsl | covered-deterministic | TestToolAndComputePublishArtifact (pkg/runtime/engine_test.go) | |
| dsl.node-human | human node: pause/resume with interaction record | dsl | covered-deterministic | TestCompliance_HumanGate (e2e/e2e_test.go) | |
| dsl.node-subbot | subbot node: nested child run, outputs read back | dsl | covered-deterministic | TestRunSubbotsPersistNestedLineage (pkg/cli/run_subbot_test.go) | |
| dsl.node-emit-wait | emit/wait: in-bot event pair with mandatory timeout | dsl | covered-deterministic | TestEventsEmitWait (e2e/events_emit_wait_test.go) | |
| dsl.node-await-answers | await_answers node parks its branch until answers land | dsl | covered-deterministic | TestAwaitAnswersReleasedByAnswer (e2e/async_interaction_test.go) | |
| dsl.node-done | done terminal node ends the run finished | dsl | covered-deterministic | TestSingleModel_HappyPath (e2e/e2e_test.go) | |
| dsl.node-fail | fail terminal node ends the run non-resumable failed | dsl | covered-deterministic | TestFailNode (pkg/runtime/engine_test.go) | |
| dsl.router-fan-out-all | router fan_out_all spawns parallel branches | dsl | covered-deterministic | TestDualParallel_HappyPath (e2e/e2e_test.go) | |
| dsl.router-fan-out-each | router fan_out_each: per-item branches with a dep DAG | dsl | covered-deterministic | TestFanOutEach_DAG_DiamondOrderingAndParallelism (pkg/runtime/fan_out_each_test.go) | |
| dsl.router-condition | router condition mode picks the matching edge | dsl | covered-deterministic | pkg/dsl/ir/validate_test.go, TestElseEdge_Routing (pkg/runtime/else_edge_test.go) | |
| dsl.router-round-robin | router round_robin alternates targets across iterations | dsl | covered-deterministic | pkg/runtime/round_robin_test.go | |
| dsl.router-llm | router llm mode: model picks the route | dsl | covered-deterministic | TestLLMRouterSelectsOtherRoute (pkg/runtime/llm_router_test.go) | |
| dsl.edges-conditional | edge when / when not conditions on a boolean output field | dsl | covered-deterministic | TestSingleModel_RefineLoop (e2e/e2e_test.go) | |
| dsl.edges-else | edge else fires only when no sibling when matched (and no back-edge with work left) | dsl | covered-deterministic | TestElseEdge_PreferredOverStrayUnconditional (pkg/runtime/else_edge_test.go) | |
| dsl.edges-loop | bounded loop edge as name(n) | dsl | covered-deterministic | TestBoundedLoop (pkg/runtime/engine_test.go) | |
| dsl.edges-loop-templated-cap | loop cap templated from an upstream output | dsl | covered-deterministic | TestLoopTemplatedCap_FromOutput (pkg/runtime/engine_test.go) | |
| dsl.edges-loop-in-parallel | bounded as loop / as foreach wholly owned by one fan_out_all, fan_out_each, or llm multi branch is accepted; boundary-crossing cycles remain C244 | dsl | covered-deterministic | TestValidateLoopInFanOutAllBody_Allowed (pkg/dsl/ir/validate_exec_branch_loops_test.go) | also fan_out_each, llm-multi, foreach, and template-head acceptance; TestValidateLoopReenteringBodyFromJoin_Rejected and TestValidateLoopOnFanOutRouterEdge_Rejected pin boundary diagnostics; TestValidateLoopStraddlingLLMMultiPerEdgeCollector_Rejected / TestValidateLoopBeforeNonElectedAwaitInFanOutAll_Allowed pin the llm-multi rule that each branch is bounded by the collector its OWN edge can elect, since the model may select that edge alone |
| dsl.edges-data-mapping | edge with {…} data mapping and reference interpolation | dsl | covered-deterministic | TestResolveMapping_InterpolatesSurroundingLiterals (pkg/runtime/engine_resolve_mapping_test.go) | |
| dsl.refs | reference syntax: input/vars/outputs/artifacts substitution | dsl | covered-deterministic | pkg/dsl/ir/ref_test.go | |
| dsl.refs-edge-input | edge with {{input.x}} is the source output (C034 and the resolver agree; no run-input fallback; router pass-through preserved, including llm; mid-graph routers warn C032 when the field is not incoming) | dsl | covered-deterministic | TestEdgeInputRef_CompileAndRuntimeAgree (pkg/runtime/edge_input_ref_test.go) | also TestEdgeInputRef_RouterPassThrough, TestEdgeInputRef_LLMRouterPassThrough, TestEdgeInputRef_SchemalessSourceWarnsAndDoesNotFallBack, TestEdgeInputRef_MidGraphRouterDoesNotFallBack, TestEdgeInputRef_MidGraphRouterPassThrough; TestValidateRefInputOnEdge_SourceInputOnly_C034 / TestValidateRefInputOnEdge_RunInputOnly_C034VarsHint / TestValidateRefInputOnEdge_RouterPassThrough_NoSchemaSkip / TestValidateRefInputOnEdge_SchemalessSource_C032Warn / TestValidateRefInputOnEdge_MidGraphRouter_NoIncomingWith_C032 / TestValidateRefInputOnEdge_MidGraphRouter_IncomingWith_OK (pkg/dsl/ir/validate_test.go) |
| dsl.vars-defaults | vars: defaults applied when no override is supplied | dsl | covered-deterministic | pkg/dsl/ir/validate_var_default_test.go | |
| dsl.vars-enum | vars: enum constraint rejects an out-of-set value at launch | dsl | covered-deterministic | TestRunRejectsInvalidEnumVar (pkg/runtime/engine_var_enum_test.go) | |
| dsl.presets | in-source presets: block resolves named value sets | dsl | covered-deterministic | pkg/dsl/ir/presets_test.go | CLI wiring covered by cli.run-preset |
| dsl.prompts | prompt <name>: blocks and prompt includes | dsl | covered-deterministic | pkg/dsl/ir/prompt_include_test.go | |
| dsl.schemas | schema <name>: typed node output contracts | dsl | covered-deterministic | pkg/backend/model/schema_test.go | |
| dsl.cursors | cursor <name>: calibration fragments reach the system prompt | dsl | covered-deterministic | TestBuildSystemPromptCursorsOnly (pkg/backend/delegate/cursors_test.go), pkg/backend/model/cursors_test.go, pkg/dsl/ir/cursor_resolve_test.go | |
| dsl.attachments | attachments: block: files resolved and passed to nodes | dsl | covered-deterministic | pkg/dsl/ir/attachments_test.go, pkg/runtime/attachment_path_test.go | |
| dsl.skills-field | skills: field pulls library skills into the run mirror | dsl | covered-deterministic | pkg/dsl/ir/validate_skills_test.go, pkg/runtime/library_skills_test.go | |
| dsl.mcp-server-block | mcp_server: block declares stdio/http/sse MCP servers | dsl | covered-deterministic | TestMCPServer_SSETransport (pkg/dsl/parser/parser_mcp_test.go) | |
| dsl.capabilities | capabilities: list opens the board tool surface | dsl | covered-deterministic | pkg/dsl/ir/validate_capabilities_test.go, TestBoardDispatcher_E2E_CapabilityGate (e2e/board_dispatcher_test.go) | |
| dsl.supervisor-block | supervisor <name>: declaration compiles and spawns a coordinator | dsl | covered-deterministic | pkg/dsl/ir/compile_supervisors_test.go, pkg/supervise/coordinator_test.go | |
| dsl.compress-field | compress: precedence (CLI → node → workflow → env → default) | dsl | covered-deterministic | TestResolveWithDefault (pkg/backend/rewrite/rewrite_test.go), TestResolveWithDefaultSourced (pkg/backend/rewrite/rewrite_test.go), pkg/dsl/ir/compress_test.go | |
| dsl.auto-memory-field | auto_memory: per-node MEMORY.md switch, off by default | dsl | covered-deterministic | pkg/dsl/ir/auto_memory_test.go, pkg/backend/model/executor_auto_memory_test.go | |
| dsl.permission-field | permission: mode + allow/ask/deny rule lists | dsl | covered-deterministic | pkg/dsl/ir/permission_test.go, pkg/backend/permission/permission_test.go | |
| dsl.budget-block | budget: block fields compile onto the workflow | dsl | covered-deterministic | pkg/dsl/ir/budget_ceiling_test.go | |
| dsl.sandbox-block | sandbox: block (image/build/network/host_state) compiles | dsl | covered-deterministic | pkg/dsl/ir/sandbox_test.go, pkg/dsl/parser/parser_sandbox_test.go | |
| dsl.worktree-field | worktree: mode default resolution | dsl | covered-deterministic | pkg/dsl/ir/worktree_default_test.go | |
| dsl.session-modes | session modes (fresh / inherit / artifacts_only) | dsl | covered-deterministic | TestSessionInherit (pkg/runtime/session_test.go), TestSessionFork (pkg/runtime/session_test.go), TestSessionInheritIfAvailable_FallsBackToFreshWhenNoSession (pkg/runtime/session_test.go) | the modes are a RUNTIME behaviour; pkg/backend/model/session_test.go is the session store (persistence + compaction), a different contract |
| dsl.secrets-block | secrets: declarations + optional-secret semantics | dsl | covered-deterministic | pkg/dsl/ir/secrets_test.go, pkg/dsl/ir/optional_secret_test.go | |
| dsl.memory-block | memory: block scopes/visibility validation | dsl | covered-deterministic | pkg/dsl/ir/memory_visibility_test.go | |
| dsl.verified-action | Verified Action quad (goal/postcondition/policy/recovery) | dsl | covered-deterministic | TestVerifiedActionEngineEmitsAndStrips (e2e/verified_action_test.go) | |
| dsl.groups-iteration | group: expansion / iteration sugar | dsl | covered-deterministic | pkg/dsl/ir/expand_groups_test.go, pkg/dsl/ir/group_expansion_test.go, pkg/dsl/ir/foreach_test.go, TestTransportCarriesAFullKindGroup (pkg/dsl/ast/jsonenc_transport_test.go), TestGroupLocalOutputsExecuteIndependently (pkg/runtime/group_expansion_test.go) | Two compiled instances resolve local compute expressions, edge conditions and mappings at runtime; all six node kinds, prompt specialization, nested-copy isolation and repeat compilation are checked. |
| dsl.loop-cap-expression | per-crossing expression and template bounds, fresh resume with raised vars, typed invalid-cap failures | dsl | covered-deterministic | pkg/dsl/ir/loop_cap_expression_test.go, pkg/runtime/loop_cap_expression_test.go | Same evaluator on trunk and branch edge selection; the v15 schema FLOOR (not the current version) preserves old-runner refusal; a string expression result is refused rather than coerced. |
| dsl.diagnostics | compile diagnostics C001–C2xx codes and severities | dsl | covered-deterministic | pkg/dsl/ir/diag_codes_test.go, TestValidate_Invalid (pkg/cli/cli_test.go) | |
| dsl.unparse-roundtrip | IR → .bot serialization round-trips | dsl | covered-deterministic | pkg/dsl/unparse/roundtrip_test.go | |
| dsl.ast-json | AST JSON encode/decode (MarshalFile/UnmarshalFile) | dsl | covered-deterministic | pkg/dsl/ast/jsonenc_test.go | |
| dsl.expr | expression evaluator for compute nodes and when conditions | dsl | unit-only | pkg/dsl/expr/expr_test.go, pkg/dsl/expr/overflow_test.go | pure evaluator, exhaustively asserted at unit level; both engine effects ride a deterministic carrier — compute output via dsl.node-compute (TestToolAndComputePublishArtifact asserts the computed value through the engine), when: expressions via the review-topology e2e (TestReviewTopology_DualAlternates drives if(vars.review_mode == 'mono', …, loop.review_loop.iteration % 2 == 1) through edgeWhenHolds); a further pipeline test would re-assert a pure function (#2792) |
| dsl.parser-fuzz | lexer/parser robustness on malformed input | dsl | unit-only | pkg/dsl/parser/fuzz_test.go | fuzzing is by nature a unit-level property test; the operator-visible surface (a diagnostic, not a panic) is dsl.diagnostics — TestValidate_Invalid feeds malformed source through the real CLI and a parser panic would kill the test process; the fuzz functions also replay their seed corpus (every examples/*.bot) as ordinary tests on each go test (#2792) |
| runtime.linear-execution | sequential node execution to a terminal node | runtime | covered-deterministic | TestLinearPath (pkg/runtime/engine_test.go) | |
| runtime.selected-incoming-edges | a node input is built only from incoming edges routing selected for this visit; unselected conditional siblings do not clobber, fan-out joins still merge every selected branch, and the set survives pause/resume (issue #484) | runtime | covered-deterministic | TestSelectedIncoming_ExclusiveSiblingsIgnoreUnselectedElse (pkg/runtime/incoming_test.go) | also TestSelectedIncoming_ExclusiveSiblingsIgnoreUnselectedElseRegardlessOfOrder, TestSelectedIncoming_ElsePathStillApplies, TestSelectedIncoming_UntrackedFallbackMergesAllWithOutput, TestSelectedIncoming_FanOutJoinMergesBothSelected, TestSelectedIncoming_SurvivesFailedResume; loop-head back-edge precedence remains TestLoopHeadSelfRefBackEdgeAdvances (pkg/runtime/loop_head_selfref_test.go) |
| runtime.await-wait-all | convergence await: wait_all waits for every branch | runtime | covered-deterministic | TestDualParallel_HappyPath (e2e/e2e_test.go) | |
| runtime.await-best-effort | convergence await: best_effort proceeds on partial branches | runtime | covered-deterministic | TestChaos_FailMidFanOut_BestEffort (pkg/runtime/chaos_test.go) | also TestSharedTargetFanOut_BranchFailure (pkg/runtime/convergence_fires_once_test.go): the survivor's output reaches the collector and the failure is named on join_ready, while wait_all fails the run before the collector |
| runtime.each-settled-evidence | a failed fan_out_each item's unresolved routes cannot be pruned by another item's decision; agreeing fallback mappings survive, conflicts stay absent, and live mappings win, including fresh-engine resume | runtime | covered-deterministic | TestSettledFloorEachEvidence (pkg/runtime/settled_each_evidence_test.go) | compiled DSL; negative control where every item rejects the alternative; TestSettledTemplateEvidenceUsesEachStart checks independent resume cursor/declaration provenance and deterministic deduplication |
| runtime.await-fires-once | the collector of a fan-out fires exactly once, after every branch has settled, under both await modes — including when a fan-out target is ALSO reachable from outside the fan-out (the mono/dual review topology, issue #741), across a branch pause + resume, and across a failure downstream of the collector | runtime | covered-deterministic | TestSharedTargetFanOut_CollectorFiresOnce (pkg/runtime/convergence_fires_once_test.go), TestReviewPRDual_CollectorChainFiresOnce (e2e/review_pr_convergence_test.go) | also TestSharedTargetFanOut_ResumeAfterBranchPauseFiresCollectorOnce, TestSharedTargetFanOut_ResumeAfterTailFailureDoesNotRefireCollector, TestSharedTemplateHeadFanOutEach_ReplaysEveryItemAndFiresCollectorOnce (fan_out_each, same class); the election itself is TestFindConvergencePoint_TargetFedFromOutsideTheFanOutIsNotTheCollector (pkg/runtime/fan_out_internal_test.go) and TestShippedDualTopologies_ElectTheDeclaredCollector (e2e/review_pr_convergence_test.go) for review-pr + evolve |
| runtime.local-loop | local loop re-executes and versions artifacts | runtime | covered-deterministic | TestSingleModel_RefineLoop (e2e/e2e_test.go) | |
| runtime.global-reloop | global reloop restarts the recipe from an upstream node | runtime | covered-deterministic | TestSingleModel_GlobalReloop (e2e/e2e_test.go) | |
| runtime.loop-exhaustion | loop cap exhaustion fails the run with LOOP_EXHAUSTED | runtime | covered-deterministic | TestCIFix_LoopExhaustion (e2e/e2e_test.go), TestLoopExhaustionRuntimeError (pkg/runtime/hardening_test.go) | |
| runtime.budget-cost | max_cost_usd exceeded past the graced ceiling stops the run and emits budget_exceeded | runtime | covered-deterministic | TestBudgetCostExceeded (pkg/runtime/budget_test.go) | with the default grace on, 100% alone no longer stops a run — see runtime.budget-exit-grace |
| runtime.budget-tokens | max_tokens exceeded past the graced ceiling stops the run | runtime | covered-deterministic | TestBudgetTokensExceeded (pkg/runtime/budget_test.go) | |
| runtime.budget-duration | max_duration exceeded past the graced ceiling stops the run | runtime | covered-deterministic | TestBudgetDurationExceeded (pkg/runtime/budget_test.go) | |
| runtime.budget-warning | budget warning event at the soft threshold, advisory only | runtime | covered-deterministic | TestBudgetWarningEmitted (pkg/runtime/budget_test.go), TestWarnTokensAdvisoryNeverBlocks (pkg/runtime/budget_test.go) | |
| runtime.budget-shared | budget accounting shared across parallel branches | runtime | covered-deterministic | TestBudgetSharedFirstComeFirstServed (pkg/runtime/budget_test.go) | |
| runtime.budget-exit-grace | a spent cap still walks FORWARD to a terminal node, inside a proportional ceiling, so banked work is delivered | runtime | covered-deterministic | TestBudgetGraceDeliversBankedWork (pkg/runtime/budget_test.go), TestBudgetGraceIsBounded (pkg/runtime/budget_test.go), TestBudgetGraceCoversDuration (pkg/runtime/budget_test.go), TestBudgetGraceCoversImmediateRecordPath (pkg/runtime/budget_test.go), TestBudgetGraceEdgeErrorStillDiesOnBudget (pkg/runtime/budget_test.go) | the delivery promise AND its bound: past cap × (1+ratio) the run fails as before, the duration axis gets a real graced deadline rather than none, and an in-grace overrun with no matching edge still dies as BUDGET_EXCEEDED (the sentinel the runner acks terminal). The fan-out path is its own row below |
| runtime.budget-exit-grace-refusals | the grace is refused when the loop guard is off, when the ratio is 0, and on an externally-imposed cap; a bad ratio fails closed | runtime | covered-deterministic | TestBudgetGraceRefusedWhenLoopGuardOff (pkg/runtime/budget_test.go), TestBudgetGraceAbsoluteWhenZero (pkg/runtime/budget_test.go), TestBudgetGraceRefusedOnImposedCap (pkg/runtime/budget_test.go), TestBudgetGraceInvalidEnvFailsClosed (pkg/runtime/budget_test.go), TestBudgetClampToCeilingMarksImposedCap (pkg/dsl/ir/budget_ceiling_test.go) | the safety half: the grace borrows "no further iteration" from the loop guard, and a cap clamped by the platform ceiling or a pool donor's allowance is absolute — the clamp choke point marking it is covered too |
| runtime.budget-exit-grace-event | every graced node is auditable: one budget_exit_grace event naming the exceeded axis and its own used/limit | runtime | covered-deterministic | TestBudgetGraceEventIsCoherentAndSingular (pkg/runtime/budget_test.go) | a deliberate overspend must be visible in the events, not discovered on the invoice — and the triple must not mix one dimension's ratio under another's name |
| runtime.budget-exit-grace-outranks-hard-limit | on the SEQUENTIAL pre-exec path a granted grace decides and returns: the 90% hard limit on another axis gets no second opinion | runtime | covered-deterministic | TestBudgetGraceSurvivesHardLimitOnAnotherAxis (pkg/runtime/budget_test.go) | the max_cost_usd + max_duration pairing late in a long run; refusing a node at 90% of axis B while permitting 110% of axis A would defeat the grace exactly where it matters. checkBudgetBeforeExec returns the grace verdict (pkg/runtime/budget.go:600-602) |
| runtime.budget-exit-grace-refused-in-branches | a parallel branch never receives the exit grace, and the wait_all death that follows still carries the BUDGET_EXCEEDED sentinel | runtime | covered-deterministic | TestBudgetGraceStopsBeforeFanOutBranches (pkg/runtime/budget_test.go), TestBudgetGraceFanOutWaitAllKeepsBudgetSentinel (pkg/runtime/budget_test.go) | sibling spend lands on one shared budget concurrently, so no branch can price its own next node — the same reason predictive loop pricing is off there. checkPreExecBudget (pkg/runtime/branch.go) now refuses on exceeded and returns, so the earlier grace/hard-limit fall-through that could write a contradicting event pair is gone by construction. The stop-path shape matters as much as the refusal: a naked convergence error would go back to JetStream as retryable and loop resume/refail against the same spent cap |
| runtime.loop-budget-guard | a loop back-edge the remaining budget cannot fund is declined, so the run leaves through its own exit path with the work it banked | runtime | covered-deterministic | TestLoopBudgetGuard_FallsThroughToDeliveryTail (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_OffRestoresTheStrandingFailure (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_ConditionalBackEdgeIsNotPricedWhenItDoesNotMatch (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_IgnoresUnenforcedAxes (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_DeclineNamesItsEscapeHatches (pkg/runtime/loop_budget_test.go) | the guard the exit grace borrows its "no further iteration" half from; emits budget_warning with reason loop_budget_guard, whose detail says the price is the last iteration's and names the escape hatches (#2677) |
| runtime.loop-budget-guard-pricing | an iteration is priced from the loop's own ENTRY and re-priced on re-entry, and the prices ride the checkpoint | runtime | covered-deterministic | TestLoopBudgetGuard_LateEnteredLoopIsPricedFromItsEntry (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_ReEnteredLoopReBasesItsPrice (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_MarksSurviveResume (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_UnpricedLoopIsNotDeclined (pkg/runtime/loop_budget_test.go) | so a second-phase or nested loop is never charged for the work that preceded it |
| runtime.loop-budget-guard-sized-by | a loop declaring sized_by "outputs…" is priced by its real traversal — the previous crossing's cost scaled by size_now / size_at_previous_crossing — while an undeclared loop keeps the pessimistic price, a prorated decline still fires when even the scaled cost does not fit, and the samples ride the checkpoint | runtime | covered-deterministic | TestLoopBudgetGuard_SizedByPricesTheTraversalNotTheFirstPass (pkg/runtime/loop_sized_by_test.go), TestLoopBudgetGuard_UnsignedLoopKeepsThePessimisticPrice (pkg/runtime/loop_sized_by_test.go), TestLoopBudgetGuard_SizedByStillDeclinesWhatDoesNotFit (pkg/runtime/loop_sized_by_test.go), TestLoopBudgetGuard_SizedBySamplesSurviveResume (pkg/runtime/loop_sized_by_test.go), TestLoopSizedBy_CompilesToTheLoop (pkg/dsl/ir/loop_sized_by_test.go), TestLoopSizedBy_InvalidSignalIsC326 (pkg/dsl/ir/loop_sized_by_test.go), TestLoopSizedBy_ConflictingClauseAcrossEdges (pkg/dsl/ir/loop_sized_by_test.go), TestEdgeLoopSizedByClause (pkg/dsl/parser/parser_test.go), TestUnparseKeepsTheLoopSizedByClause (pkg/dsl/unparse/lossless_test.go) | the shrinking-iteration half of #2677; the first crossing of a loop whose signal appears inside its own body has no earlier reading and prices full — guard: off is that shape's clause |
| runtime.loop-budget-guard-precedence | --loop-budget-guard → loop clause guard: → workflow loop_budget_guard: → ITERION_LOOP_BUDGET_GUARD → on, and the run-level override travels onto the cloud queue | runtime | covered-deterministic | TestLoopBudgetGuard_PrecedenceChain (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_WorkflowOffIsHonouredEndToEnd (pkg/runtime/loop_budget_test.go), TestValidateLoopBudgetGuardMode (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_LoopClauseOptsOut (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_LoopClauseOnBeatsWorkflowOff (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_LaunchOverrideBeatsTheClause (pkg/runtime/loop_budget_test.go), TestLoopBudgetGuard_OneOptedOutLoopRefusesTheExitGrace (pkg/runtime/loop_budget_test.go), TestLoopGuard_CompilesToTheLoop (pkg/dsl/ir/loop_guard_test.go), TestLoopGuard_InvalidValueIsC178 (pkg/dsl/ir/loop_guard_test.go), TestEdgeLoopGuardClause (pkg/dsl/parser/parser_test.go), TestUnparseKeepsTheLoopGuardClause (pkg/dsl/unparse/lossless_test.go) | C133 rejects an invalid workflow mode, C178 an invalid clause value; one opted-out loop refuses the exit grace for the run |
| runtime.budget-unpriced-cost | a delegate that reports no cost is advertised as a FLOOR, not a total, rather than silently counting as $0 | runtime | covered-deterministic | TestSharedBudget_UnpricedSpend (pkg/runtime/budget_test.go) | the cost_usd_unpriced advisory documented in docs/dsl.md; warns at most once per run and never blocks |
| runtime.max-parallel-branches | max_parallel_branches semaphore bounds concurrency | runtime | covered-deterministic | TestFanOutEach_DAG_BoundedParallelism (pkg/runtime/fan_out_each_test.go) | |
| runtime.branch-local-loops | loops/foreach inside parallel branches use independent durable counters, outputs, artifact allocations, human-resume cursors, and rewind invalidation | runtime | covered-deterministic | TestFanOutEachBranchLocalLoopsKeepIndependentCounters (pkg/runtime/branch_local_loop_test.go) | also TestFanOutAllBranchLocalLoopRunsBeforeWaitAll, TestFanOutEachBranchLocalForeachUsesPrivateIndex, TestResolveLoopAndEachSeeEnclosingTrunkNamespaces, TestFanOutInsideTrunkLoopResolvesEnclosingPreviousOutput, TestFanOutInsideTrunkForeachResolvesEnclosingItem, TestFanOutEachBranchLocalLoopHumanResumeKeepsScope, TestFanOutEachSiblingHumanGatesResumeOneScopeAtATime, TestFanOutEachBranchCanPauseAtSequentialHumanGates, and TestRewind_PromotesFanOutBodyToRouter; race-covered |
| runtime.daily-cap-branch-resume | a branch that pauses and resumes contributes BOTH passes' spend to the per-day cap ledger | runtime | covered-deterministic | TestBranchDailyCapLedgerSurvivesResume (pkg/runtime/branch_local_loop_test.go), TestBranchDailyCapLedgerKeyStableAcrossSiblingResume (pkg/runtime/branch_local_loop_test.go), TestConformance_FilesystemShared (pkg/store/storetest_run_test.go) | the ledger key is <run>#<branch>#<loop-path> and AddSpend keeps the monotonic MAX per key; the loop path is restored from the checkpoint, so a resumed branch re-uses its pre-pause key even when sibling goroutine order changes. With branch-local cursors the resumed pass runs only the nodes after the gate, so an accumulator restarting at zero would be max()'d away and every post-resume dollar would vanish from the cap |
| runtime.workspace-safety | only one mutating branch may run concurrently | runtime | covered-deterministic | TestWorkspaceSafetyRejectsDualMutation (pkg/runtime/budget_test.go) | |
| runtime.checkpoint | a checkpoint is saved after every successful node | runtime | covered-deterministic | TestCheckpointPreservesUpstreamOutputs (pkg/runtime/engine_test.go) | |
| runtime.resume-failed | resume from failed_resumable restarts at the failing node | runtime | covered-deterministic | TestResumeFromFailed (pkg/runtime/engine_test.go), TestResumeDoesNotReplayUpstream (pkg/runtime/engine_test.go) | |
| runtime.resume-hash-guard | resume refuses a changed .bot unless --force | runtime | covered-deterministic | TestForceResumeBypassesHashCheck (pkg/runtime/engine_test.go), pkg/runview/service_resume_hash_test.go | |
| runtime.resume-settled-contract | a forced source edit removes a floor-supplied artifact dependency from the next publication while preserving the historical contract | runtime | covered-deterministic | TestSettledArtifactContractAfterForcedEdgeRemoval (pkg/runtime/settled_contract_resume_test.go) | Two compiled sources with different hashes, enforce, real publication/pause/resume, unforced refusal and an ordinary subsequent resume. |
| runtime.resume-human | resume a paused_waiting_human run with answers | runtime | covered-deterministic | TestHumanPauseAndResume (pkg/runtime/engine_test.go), TestResume_Success (pkg/cli/cli_test.go) | |
| runtime.resume-launch-overrides | launch-time --sandbox / --sandbox-default-image / --sandbox-host-state / --merge-into / --branch-name / --merge-strategy / --auto-merge are persisted on the run at launch and replayed by CLI + studio + cloud-runner resume paths; explicit resume flags layer over them (#1435, #1366) | runtime | covered-deterministic | TestRunResolveDoc_PersistsLaunchSandboxAndMergeOverrides (pkg/runtime/launch_overrides_persist_test.go), TestBundleResourceDir_AbsolutisesRelative (pkg/runtime/launch_overrides_persist_test.go), TestPickString_ExplicitWinsElseInherits (pkg/cli/resume_launch_overrides_test.go) | Cloud runner reads its own env for the sandbox tier (cfg.SandboxOverride/Default/DefaultImage/HostState, msg.SandboxImage) — the pod IS the isolation boundary — but replays MergeInto/BranchName/MergeStrategy/AutoMerge off the run doc (Philosophy #3, no feature ships local-only). WithFilePath stores the launcher's path verbatim; the mount source is absolutised at bundleResourceDir alone. See docs/resume.md. |
| runtime.resume-answered-human-advance | a resume from failed_resumable whose failed node is a human node with a previously-answered non-retired interaction on it REPLAYS those answers through resumeFromPause itself (the dispatch predicate answeredHumanGateReplay flips the status back to paused_waiting_human; there is no second transition to drift), which publishes the declared artifact, closes the parked-window boundary at the human node (markPreNodeBoundary — files the operator wrote while parked do NOT land in the next node's rewind scope) and advances. A rewind stamps Run.LastRewindAt and retires the pivot's blocking-pause interactions (twin refusals). A non-empty --answer on the failed resume is what gets recorded through the pause path's own write (#1435 third defect + PR #1490 gate findings Rac891d & R60aa7e & R62a836) | runtime | covered-deterministic | TestResumeFromFailure_AdvancesPastAnsweredHumanNode (pkg/runtime/answered_human_advance_test.go), TestRecordHumanAnswers_EmptyDoesNotWipe (pkg/runtime/answered_human_advance_test.go), TestResumeFromFailure_RefusesStaleAnswersAfterRewind (pkg/runtime/answered_human_advance_test.go), TestResumeFromFailure_CallerAnswersOverwriteStored (pkg/runtime/answered_human_advance_test.go), TestGateReplay_ClosesParkedWindow (pkg/runtime/answered_human_advance_test.go) | Freshness proof: the predicate matches the exact interactionIDForPause(runID, nodeID, loopCounters) from the CHECKPOINT's counters AND refuses a stored answer whose AnsweredAt is at or before Run.LastRewindAt. Rewind's twin layer retires blocking-pause interactions (RetireInteractions with includeBlocking=true). The replay witnesses the pause path's own resumed-boundary label (resume:gate:0 on the workspace tracker), which only the resumed boundary writes. |
| sandbox.mode-split-auto-vs-inline | sandbox: auto degrades to an UNSANDBOXED run (nil sandbox, so the host devbox and iterion PATH shim are provisioned) with a visible sandbox_skipped event when the host has no container-runtime driver (and on the resolver's own obstacles, reachable from a library caller), whatever tier named the mode; an author-wired build: block is as explicit as an image: one, so --sandbox auto does not rewrite it; sandbox: { mode: inline, image/build: … } refuses with RuntimeError{Code: SANDBOX_DRIVER_UNAVAILABLE} wrapping sandbox.ErrDriverUnavailable, the persisted Run.FailureCode carries it, and the retrypolicy classifier keys it Deterministic so a redelivery does not spin (#1425 + #1426) | sandbox | covered-deterministic | TestDriverForSpecReportsHostCapabilityNotRunPolicy (pkg/sandbox/factory_test.go), TestFactoryDriverWrapsErrDriverUnavailable (pkg/sandbox/factory_preferred_wrap_test.go), TestResolveAndStartSandbox_AutoDegradesToHostWithVisibleEvent, TestResolveAndStartSandbox_EveryAutoDegradeSaysFileSecretsAreDropped, TestResolveAndStartSandbox_BuildOnlyInlineBlockIsNotRewrittenByAutoFlag, TestResolveAndStartSandbox_InlineRefusesWithTypedCode, TestRun_ExplicitSandboxRefusalParksTheTypedCodeOnTheRun, TestRetryPolicy_SandboxDriverUnavailableIsDeterministic (pkg/runtime/sandbox_auto_vs_inline_test.go), TestResolveSandboxSpecAutoDegradesWhateverItsSource (pkg/runtime/sandbox_internal_test.go) | The factory test pins that pkg/sandbox holds no mode policy (the doctor and the pre-flight read the same capability answer); the runtime tests exercise both halves through resolveAndStartSandbox with the REAL noop driver in the registry; TestRun_…ParksTheTypedCodeOnTheRun drives a whole Engine.Run so the classification chain (setupFailureStatus → setupFailureCode → the persisted document) is proven, not just the call site. |
| runtime.mission-receipt-cas | durable mission resume claims one exact status and stamps its host receipt on run_resumed; rewind resolves a non-mutating final pivot, guards it before mutation, and stamps run_rewound | runtime | covered-deterministic | TestResumeFromFailure_MissionClaimUsesExactStatusAndStampsReceipt (pkg/runtime/resume_failure_char_test.go), TestResolveRewindPivotIsNonMutatingAndGuardedReceiptIsAuditable (pkg/runview/rewind_test.go), TestSubmitResume_MissionExpectedStatusAndReceiptCrossPublisherCAS (pkg/server/cloudpublisher/publisher_resume_test.go) | cloud publisher consumes expected status before queued; runner keeps receipt identity |
| runtime.cancel | cancellation produces a cancelled status with a checkpoint | runtime | covered-deterministic | TestCancelProducesCancelledStatus (pkg/runtime/hardening_test.go) | |
| runtime.timeout | outer deadline produces a TIMEOUT failure | runtime | covered-deterministic | TestTimeoutProducesFailedStatus (pkg/runtime/hardening_test.go) | |
| runtime.interaction-modes | human interaction: llm / llm_or_human / async escalation | runtime | covered-deterministic | TestInteractionLLMOrHumanEscalation (pkg/runtime/engine_test.go), TestInteractionLLMAutoRespond (pkg/runtime/engine_test.go) | |
| runtime.async-backend-capability | async nodes refuse incapable primary/fallback routes and Pi print transport before dispatch, without retries | runtime | covered-deterministic | pkg/dsl/ir/async_backend_test.go, pkg/backend/model/async_backend_test.go, TestCapabilityUnsupportedDoesNotRetry (pkg/runtime/recovery/capability_test.go) | |
| runtime.ask-user-conversation | ask_user relays prior Q/A and persists the conversation | runtime | covered-deterministic | TestInteractionAskUserPersistsConversation (pkg/runtime/engine_test.go) | |
| runtime.worktree-finalize | worktree: auto creates a branch and fast-forwards the checkout | runtime | covered-deterministic | pkg/runtime/worktree_test.go | |
| runtime.worktree-early-refusal | terminal compute/fail refusal releases a pristine checkout and rewind restores its original baseline | runtime | covered-deterministic | pkg/runtime/worktree_refusal_test.go, pkg/runview/rewind_reclaimed_test.go | |
| runtime.rewind | iterion rewind re-anchors a run and invalidates downstream state | runtime | covered-deterministic | TestRewindThenResume_SkipsUpstreamNodes (e2e/rewind_resume_test.go) | |
| runtime.run-document-concurrency | rename and rewind refuse stale document writes instead of overwriting a concurrent resume | runtime | covered-deterministic | TestRenameDoesNotUndoConcurrentResume (pkg/runview/run_version_test.go), TestRewindDoesNotOverwriteConcurrentResume (pkg/runview/run_version_test.go), TestRewindDoesNotOverwriteCheckpointAfterFinalSave (pkg/runview/run_version_test.go) | Shared FS/Mongo SaveRunVersionConflicts conformance and cross-store FS contention pin the persistence boundary. A rewind conflict after file restoration does not roll files back. |
| runtime.fanout-output-freshness | re-invoked fan-outs discard obsolete branch outputs at convergence, including all-failed, partial, unreached-tail and empty invocations | runtime | covered-deterministic | TestConvergenceReinvocationUsesCurrentOutputs, TestConvergenceFreshnessRespectsInvocationScope (pkg/runtime/convergence_freshness_test.go) | Collector and direct downstream references, fresh-engine pause/resume, branch cursor, unselected targets and bounded-edge boundaries. Published artifacts retain their history. |
| runtime.rewind-workspace | rewind restores workspace files for non-worktree runs, scoped to what the run recorded changing (issue #380) | runtime | covered-deterministic | TestRewindRestoresWorkspaceEndToEnd (e2e/rewind_workspace_test.go) | |
| runtime.rewind-workspace-failed-node | a node that dies mid-execution still has its debris undone, via the fail: boundary | runtime | covered-deterministic | TestRewindScopeCoversAFailedNodesDebris (e2e/rewind_workspace_test.go) | |
| runtime.rewind-workspace-stop-window | edits made while a run is stopped (fail → triage → resume, a human gate, a delegate ask_user pause) are not attributed to the run | runtime | covered-deterministic | TestRewindAfterResumeKeepsTriageEdits, TestRewindScopeAfterHumanGatePause, TestRewindScopeAfterDelegatePause (e2e/rewind_workspace_test.go) | |
| runtime.fork | fork a run at a prior LLM turn into a resumable child run | runtime | covered-deterministic | pkg/runview/fork_test.go | |
| runtime.event-stream | event sequence coherence (ordering, pairing, monotonic seq) | runtime | covered-deterministic | TestEventSequenceCoherence (e2e/e2e_test.go) | |
| runtime.artifact-versioning | repeated node executions version their artifacts | runtime | covered-deterministic | TestSingleModel_GlobalReloop (e2e/e2e_test.go) | |
| runtime.publish-gate | only a publish:-declared node leaves an artifact | runtime | covered-deterministic | TestOnlyAPublishedNodeLeavesAnArtifact (e2e/handoff_publish_test.go) | |
| runtime.skills-mirror | bundle/plugin/library skills mirrored into .claude/skills/; orphans (renamed / removed / disabled upstream) pruned at pass end in run-owned worktrees, operator-edited copies preserved | runtime | covered-deterministic | TestMirrorBundleSkills_CopiesIntoClaudeSkills (pkg/runtime/bundle_test.go), TestPruneWorkspaceMirror_OrphanIsPrunedAcrossBothShapes, TestPruneWorkspaceMirror_FreshMirrorSurvives, TestPruneWorkspaceMirror_OperatorEditedFileIsPreserved, TestPruneWorkspaceMirror_MarkerlessFileIsPreserved, TestPruneWorkspaceMirror_StaleMarkerWithoutDestIsCleaned, TestPruneWorkspaceMirror_AllThreeKindDirs, TestPruneWorkspaceMirror_InPlaceWorkspaceSkipsByDefault, TestPruneWorkspaceMirror_InPlaceWorkspacePrunesWithEnvOptIn, TestClearMirroredTierMarkers_WipesEveryKind, TestPruneWorkspaceMirror_EndToEndRename (pkg/runtime/prune_mirror_test.go) | the pruner tests beside the mirror test carry the class: both mirror shapes, operator-edited and markerless files preserved, a stale marker without its destination cleaned, the three kind dirs, in-place workspaces pruned only on opt-in, and an end-to-end rename |
| runtime.devbox-provision | a bot's/target's devbox.json is installed and put on PATH | runtime | covered-deterministic | TestEngineRun_HostDevbox_RepoProjectInstallsInPlace (pkg/runtime/devbox_host_test.go) | |
| runtime.subbot-depth-guard | nested subbot recursion depth guard | runtime | covered-deterministic | TestSubbotRunnerForCLI_RecursionDepthGuard (pkg/cli/subbot_nested_test.go) | |
| runtime.supervisor-steer-seam | supervisor steers a watched node via the message inbox — the composed path with the steer decision stubbed (#2793): a watched node's evidence wakes the coordinator through the production event transport, the stubbed decision's injection leaves through the real runview queue node-scoped and supervisor-framed, lands in the file-backed inbox, waits while another node executes, is delivered once and consumed by the watched node's drain, the steering's own inbox echo wakes nobody, and the durable cursor ends on one settled intervention | runtime | covered-deterministic | TestCoordinatorSteersWatchedNodeThroughTheInbox (pkg/supervise/coordinator_steer_inject_test.go), pkg/supervise/coordinator_test.go, pkg/backend/model/inbox_test.go | the only stand-in is the Decision itself; a mute of the injection branch reddens the composed test on its own inbox assertion |
| runtime.supervisor-steer-decision | a real model, watching a supervised run, composes a SENSIBLE steer — the decision itself (Coordinator → GenerateObjectDirect), not the delivery | runtime | covered-live | TestLive_Feat_Supervisor (e2e/live_feat_supervisor_test.go) | gate: paid — paid target task test:live:feat:supervisor; its ledger row test:live:feat:supervisor (e2e/testdata/live/ledger.json) reads never, so under #1424's clause 2 this row is an honest gap, not a green line. The opencode tool-gated-free model does NOT apply: a CLI bench proves a tool boundary without a model turn, while what a stub cannot judge here is whether the composed steer is sensible — only a real model turn carries that. Delivery seam split off to runtime.supervisor-steer-seam (#2793) |
| runtime.recovery-dispatch | adaptive recovery ladder for verified action nodes | runtime | covered-deterministic | TestVerifiedActionEngineEmitsAndStrips (e2e/verified_action_test.go), pkg/backend/model/executor_verified_action_test.go | |
| runtime.privacy-redaction | secret/PII redaction across the run pipeline | runtime | covered-deterministic | TestE2E_PrivacyPipeline (e2e/privacy_test.go) | |
| persistence.run-json | run.json metadata, status transitions, format version | persistence | covered-deterministic | TestFormatVersionPersisted (pkg/runtime/hardening_test.go), pkg/store/store_test.go | |
| persistence.events-jsonl | events.jsonl append + monotonic seq + replay | persistence | covered-deterministic | pkg/store/store_test.go | |
| persistence.artifacts | versioned per-node artifacts under artifacts/ | persistence | covered-deterministic | pkg/store/store_test.go | |
| persistence.interactions | interaction records (questions/answers) persisted per run | persistence | covered-deterministic | TestAwaitAnswersAlreadyAnswered (e2e/async_interaction_test.go), pkg/store/store_test.go | |
| persistence.child-runs | parent/child run lineage for subbots | persistence | covered-deterministic | TestRunSubbotsPersistNestedLineage (pkg/cli/run_subbot_test.go) | |
| persistence.workspace-versioning | content-addressed workspace snapshots + restore | persistence | covered-deterministic | pkg/workspacetrack/native_test.go | |
| persistence.store-anchoring | store dir resolution (project .iterion vs $ITERION_HOME/projects) | persistence | covered-deterministic | TestStoreAnchorDir_BotInsideProjectResolvesProjectStore (pkg/cli/storeanchor_test.go) | |
| persistence.mongo-store | cloud Mongo-backed run store conformance | persistence | covered-deterministic | TestConformance_Mongo (pkg/store/mongo/conformance_test.go) | CI job mongo-conformance; skips without a Mongo endpoint. The board (kanban) Mongo store is a different contract — see dispatcher.native-tracker |
| persistence.mongo-future-fields | an older replica renames a run without truncating additive run/checkpoint/branch fields; intentional deletes, stale saves and future schema versions remain guarded | persistence | covered-deterministic | TestSaveRunPreservesFutureStateAcrossOldReaderRename, TestSaveRunRefusesStaleCopyWithFutureState, TestSaveRunRefusesFutureSchema (pkg/store/mongo/runs_unknown_test.go) | Real Mongo under mongo-conformance; includes BSON type preservation and reordered edge records. Skips without a Mongo endpoint. |
| cli.validate | iterion validate parses, compiles and reports diagnostics | cli | covered-deterministic | TestValidate_Valid (pkg/cli/cli_test.go), TestValidate_Invalid (pkg/cli/cli_test.go) | |
| cli.connector-identities | connectors gen preserves operation/auth identities across regeneration and leaves the existing package unchanged on refusal | cli | covered-deterministic | TestConnectorRegenerationPreservesExistingIdentities (pkg/cli/connectors_identity_test.go), TestConnectorIdentityRefusalsPreservePackage (pkg/cli/connectors_identity_test.go), TestForgejoIdentityLockMatchesAllExistingOperations (pkg/connector/identity/lock_test.go) | Actual two-generation fixtures, retired names, auth references, overlay conflicts, future locks, concurrency and the 503 shipped Forgejo identities; no queue or runtime format change |
| cli.validate-bundle | iterion validate cross-checks a bundle manifest (C2xx) | cli | covered-deterministic | TestRunValidate_BundleVarTypoWarns (pkg/cli/validate_bundle_test.go) | |
| cli.run | iterion run executes a .bot and persists the run | cli | covered-deterministic | TestRun_Success (pkg/cli/cli_test.go) | |
| cli.run-vars | iterion run --var key=value overrides workflow vars | cli | covered-deterministic | TestRun_WithVars (pkg/cli/cli_test.go) | |
| cli.run-preset | iterion run --preset applies an in-source preset, --var wins over it | cli | covered-deterministic | TestRunPresetAppliesValuesAndVarWins (e2e/cli_launch_overrides_test.go) | |
| cli.run-budget-override | iterion run --max-* re-budgets the workflow for this run | cli | covered-deterministic | TestRunBudgetOverrideCapsTheRun (e2e/cli_budget_override_test.go) | |
| cli.run-model-backend-override | iterion run --model/--backend selector=… re-target nodes | cli | covered-deterministic | TestRunBackendOverrideRetargetsNodes (e2e/cli_model_backend_override_test.go), TestRunModelOverrideRetargetsNodes (e2e/cli_model_backend_override_test.go) | drives the REAL ClawExecutor (no stub) against fixtures whose backend/model cannot resolve, so the resolved value is readable from the failure text — credential-free and offline (unknown backend dies at registry lookup, unknown provider inside claw's spec parse) |
| cli.run-human-pause | iterion run returns at a human pause (--no-interactive) | cli | covered-deterministic | TestRun_HumanPause (pkg/cli/cli_test.go) | |
| cli.run-json | --json machine output mode for run | cli | covered-deterministic | TestRun_SuccessJSON (pkg/cli/cli_test.go) | |
| cli.run-recipe | iterion run --recipe <file> applies a recipe overlay | cli | covered-deterministic | TestRunRecipeAppliesPresetVarsAndVarStillWins (e2e/cli_recipe_test.go), TestRunRecipeBudgetOverridesTheWorkflowBudget (e2e/cli_recipe_test.go), TestRunRecipeRefusesAMismatchedWorkflow (e2e/cli_recipe_test.go), TestRunRecipeRefusesAnUnloadableFile (e2e/cli_recipe_test.go) | preset vars reaching the run, --var precedence over the recipe, workflow_ref.path resolving the .bot, the recipe budget changing the run outcome, and the mismatch/unloadable refusals |
| cli.run-auto-resume | --auto-resume N re-drives a retryable failed_resumable run | cli | covered-deterministic | TestRunAutoResumeRecoversRetryableFailure (e2e/cli_auto_resume_test.go), TestRunAutoResumeStaysOffByDefault (e2e/cli_auto_resume_test.go), TestRunAutoResumeRefusesNonRetryableCause (e2e/cli_auto_resume_test.go), TestRunAutoResumeStopsAtTheAttemptBudget (e2e/cli_auto_resume_test.go) | no clock seam needed: a USAGE_LIMIT_BLOCKED cause takes the reset-aware wait, which the run's own Retry policy (the field iterion schedule sets) clamps — a 1ms horizon collapses it, so the whole suite runs in <0.5s |
| cli.resume | iterion resume continues a paused/failed/cancelled run | cli | covered-deterministic | TestResume_Success (pkg/cli/cli_test.go) | |
| cli.resume-answers-file | iterion resume --answers-file / --answer @file | cli | covered-deterministic | TestResolveFileAnswerFlags_AttachesLocalFile (pkg/cli/resume_file_answers_test.go), TestParseAnswersFile (pkg/cli/cli_test.go) | |
| cli.resume-subbot | resume a run that owns subbot children | cli | covered-deterministic | TestResume_RunWithSubbot (pkg/cli/resume_subbot_test.go) | |
| cli.inspect | iterion inspect lists runs and shows a run's state | cli | covered-deterministic | TestInspect_ListRuns (pkg/cli/cli_test.go), TestInspect_SingleRun (pkg/cli/cli_test.go) | |
| cli.inspect-events | iterion inspect --events renders the stored event stream | cli | covered-deterministic | TestInspect_WithEvents (pkg/cli/cli_test.go) | |
| cli.inspect-node | iterion inspect --node per-node trace/artifacts/log sections | cli | covered-deterministic | TestInspect_SectionTrace (pkg/cli/cli_test.go), TestInspect_SectionArtifactsIncludesBody (pkg/cli/cli_test.go) | |
| cli.report | iterion report renders a run's chronological markdown report | cli | covered-deterministic | TestReportRendersChronologicalRunReport (e2e/cli_report_test.go), TestReportHonoursOutputPathAndJSON (e2e/cli_report_test.go) | |
| cli.diagram | iterion diagram emits a Mermaid graph for a .bot | cli | covered-deterministic | TestDiagramRendersEveryNodeAndEdgeOfTheWorkflow (e2e/cli_diagram_test.go), TestDiagramViewsDifferAndUnknownViewIsRefused (e2e/cli_diagram_test.go), TestDiagramJSONCarriesTheRenderedGraph (e2e/cli_diagram_test.go) | every node + edge of e2e/testdata/diagram_mini.bot, the condition/loop/mapping labels, the compact↔detailed↔full deltas, and the typo'd --view refusal |
| cli.runs-prune | iterion runs prune age/status/keep-last retention + dry-run | cli | covered-deterministic | TestRunPrune_AgeFiltering (pkg/cli/runs_prune_test.go), TestRunPrune_DryRunDeletesNothing (pkg/cli/runs_prune_test.go) | |
| cli.runs-async-questions | iterion runs questions / runs answer drive an async question to delivery | cli | covered-deterministic | TestRunsQuestionsThenAnswerReleasesAwaitGate (e2e/cli_async_questions_test.go), TestRunsAnswerRejectsBadInput (e2e/cli_async_questions_test.go) | |
| cli.fork | iterion fork creates a resumable fork at a prior turn | cli | covered-deterministic | pkg/runview/fork_test.go | CLI layer is a thin wrapper over runview.Service.Fork |
| cli.rewind | iterion rewind (incl. --auto bot-diff targeting) | cli | covered-deterministic | TestRewind_RefusesRunningRun_E2E (e2e/rewind_resume_test.go) | |
| cli.import | iterion import lowers a Claude-Code workflow script to a draft .bot | cli | covered-deterministic | TestRunImport_WritesDraft (pkg/cli/import_test.go) | |
| cli.bots-list | iterion bots list discovers .bot/.botz bundles | cli | covered-deterministic | TestBotsList_Bundle (pkg/cli/bots_test.go) | |
| cli.bots-create | iterion bots create scaffolds a discoverable bundle | cli | covered-deterministic | TestBotsCreate_ProducesDiscoverableBundle (pkg/cli/bots_create_test.go) | |
| cli.bots-create-shapes | --template <shape> renders a complete workflow of that form (bounded loop, reviewer fan-out, human gate, verified action, …) with its annex files, compiled through the bundle loader and held to its form | cli | covered-deterministic | TestGalleryShapes (pkg/botscaffold/shapes_test.go) | |
| cli.bots-regen-catalog | iterion bots regen-catalog regenerates Nexie's catalog skill | cli | covered-deterministic | bots/catalog_freshness_test.go | |
| cli.bundle-pack | iterion bundle pack produces a loadable .botz | cli | covered-deterministic | TestBundle_SecAuditSource_PackOpenCompile (e2e/bundle_sec_audit_source_test.go) | |
| cli.marketplace | iterion marketplace submit/install/uninstall (bot + plugin kinds) | cli | covered-deterministic | TestMarketplaceCLI_SubmitInstallUninstall_KindAware (pkg/cli/marketplace_test.go) | |
| cli.plugin | iterion plugin list/enable/disable/install/uninstall/config | cli | covered-deterministic | pkg/plugin/install_test.go, pkg/plugin/config_test.go | |
| cli.skill-library | iterion skill list/show/add/rm/export (layered global/project) | cli | covered-deterministic | TestSkillLibraryAddListShowExportRemove (e2e/cli_skill_test.go), TestSkillLibraryProjectScopeShadowsGlobal (e2e/cli_skill_test.go) | round-trip against an isolated ITERION_HOME, cross-checked through the same skilllib.LocalStoreForProject().Resolve the runtime mirror calls. skill import (git URL) is covered as plugins.install |
| cli.secret | iterion secret set/list/rm local sealed-secret lifecycle | cli | covered-deterministic | TestSecretSetListRemoveRoundTrip (e2e/cli_secret_test.go), TestSecretProjectScopeOverridesGlobal (e2e/cli_secret_test.go) | |
| cli.memory | iterion memory export/import/du | cli | covered-deterministic | TestMemoryExportImportRoundTripsASpace (cmd/iterion/memory_test.go), TestMemoryImportStrategyDecidesWhoWinsOnConflict (cmd/iterion/memory_test.go), TestMemoryDuReportsTheSpaceUsageAndQuota (cmd/iterion/memory_test.go), TestMemoryRefusesAnUnaddressableSpace (cmd/iterion/memory_test.go) | archive round-trip into a different space, the skip/overwrite conflict strategies, usage/quota reporting, and the unaddressable-space refusal — all through the real rootCmd against an isolated ITERION_HOME |
| cli.models | iterion models resolves capabilities and their source | cli | covered-deterministic | TestRunModels_JSONSingleModel (pkg/cli/models_test.go) | |
| cli.openapi | iterion openapi emits this build's OpenAPI 3.1 spec offline | cli | covered-deterministic | pkg/server/openapi_test.go | |
| cli.schedule | iterion schedule add/list/remove/install/uninstall crontab manifest | cli | covered-deterministic | TestRunScheduleAddListRemove (pkg/cli/schedule_test.go), TestRunScheduleInstallUninstall_SeamRoundTrip (pkg/cli/schedule_test.go) | |
| cli.schedule-gate | schedule overlap policy + pre-launch guard + tick audit | cli | covered-deterministic | TestScheduleRun_OverlapSkipsAndAuditsBlockingRun (pkg/cli/schedule_gate_test.go), TestScheduleRun_GuardNonZeroBlocks (pkg/cli/schedule_gate_test.go) | |
| cli.issue | iterion issue create/list/show/move/update/close/board | cli | covered-deterministic | TestIssueCLILifecycleCreateMoveUpdateClose (e2e/cli_issue_test.go), TestIssueCloseRefusesABoardWithNoTerminalState (e2e/cli_issue_test.go) | create → list/show → move → update → close read back through a fresh native.Store, plus the events.jsonl audit trail |
| cli.issue-import | iterion issue import pulls forge issues onto the board | cli | covered-deterministic | TestRunIssueImport_CreatesCardsAndIsIdempotent (e2e/cli_issue_import_test.go), TestRunIssueImport_RejectsUnsupportedForge (e2e/cli_issue_import_test.go) | drives the real CLI against a fake forge, then re-opens the board through a FRESH store: the cards that landed are the oracle, and a re-import proves idempotence |
| security.disable-auth-switch | ITERION_DISABLE_AUTH bypasses authentication on every /api/* endpoint | server-api | covered-deterministic | TestDisableAuthSwitchGovernsEveryProtectedRoute (pkg/server/disable_auth_switch_test.go) | both positions asserted as one contract — only the PAIR is meaningful, since pinning one alone passes for a switch welded shut |
| security.origin-gate | every state-changing /api route refuses a foreign Origin (403), while same-origin, loopback, wails and no-Origin callers pass | server-api | covered-deterministic | TestEveryStateChangingAPIRouteRefusesForeignOrigin (pkg/server/origin_gate_sweep_test.go), TestOriginGateAdmitsLegitimateCallers (pkg/server/origin_gate_sweep_test.go), TestOriginGateSweepBites (pkg/server/origin_gate_sweep_test.go), TestKillSwitchReachesHandlerLevelCallsToo (pkg/server/origin_gate_observability_test.go), TestSameWSOrigin (pkg/server/hub_origin_test.go) | the CSRF boundary, and the promise is "every route", so the sweep reads the LIVE routing table (174 rows) instead of a hand-kept list — a route added tomorrow is asserted with no edit here. Driven through the composed srv.handler, so it proves the gate is WIRED and not merely correct. Two properties keep it honest: it names the endpoints it exists for (a refactor that stops registering them fails rather than silently shrinking the sweep), and …SweepBites flips ITERION_REQUIRE_ORIGIN=0 to prove the 403s come from the gate rather than from some other refusal. The kill switch is asserted on BOTH callers, because it is read inside requireSafeOrigin rather than in the middleware: ~70 handlers call that directly, and a switch reaching only the middleware left the documented emergency rollback working for some routes and silently not for run cancel/merge, project writes, platform settings, bot sources and marketplace writes |
| security.origin-gate-diagnostics | a refusal is logged naming method/path/Origin (at info, so it cannot evict Sentry breadcrumbs), an admitted request logs nothing, and ITERION_ALLOWED_ORIGINS names extra hosts — normalised the way a browser serialises an Origin, malformed entries reported rather than dropped | server-api | covered-deterministic | TestOriginGateNamesWhatItRefused (pkg/server/origin_gate_observability_test.go), TestOriginGateStaysQuietWhenItAdmits (pkg/server/origin_gate_observability_test.go), TestRefusalLogCannotForgeARecord (pkg/server/origin_gate_observability_test.go), TestRefusalLogIsBounded (pkg/server/origin_gate_observability_test.go), TestExtraAllowedOriginsAdmitsASecondPublicHost (pkg/server/origin_gate_observability_test.go), TestExtraAllowedOriginsIsOffByDefault (pkg/server/origin_gate_observability_test.go), TestExtraAllowedOriginsReachesTheGateOnTheCloudServerToo (pkg/server/origin_gate_observability_test.go), TestMalformedAllowedOriginIsReportedNotSwallowed (pkg/server/origin_gate_observability_test.go), TestRefusalDoesNotFireTheLogHook (pkg/server/origin_gate_observability_test.go), TestRefusalIsLoggedAtInfoExactly (pkg/server/origin_gate_observability_test.go), TestAllowlistEntriesAreNormalisedTheWayABrowserSerialisesAnOrigin (pkg/server/origin_gate_observability_test.go), TestWildcardOriginIsRefusedLoudly (pkg/server/origin_gate_observability_test.go), TestTrailingSlashIsAcceptedNotScolded (pkg/server/origin_gate_observability_test.go) | the gate answers the caller and nothing else, so "nothing legitimate is refused" and "we cannot see one" were the same empty grep — which is how the board-MCP no-Origin claim stayed an inference. The env-var half drives the WHOLE chain from BOTH constructors (env → New / BrowserGuard → allowlist → verdict on a real request): asserting the parser alone stays green with the wiring deleted, and pinning only BrowserGuard leaves the cloud server — the surface the variable exists for — free to regress to inert while green. Both log values go through logSafe and NOT %q, whose escaping would neutralise a CRLF on its own and mask a removed sanitiser — the first version of the forging test did exactly that and passed inert. Normalisation is asserted through the GATE and over PublicURL as well as the env var, since both build an entry from a URL and matching is ==: an entry that merely parses is not an entry that works. The LEVEL is pinned in both directions rather than left to the call site, because it is written down in docs an operator greps by: at warn the refusal would evict Sentry breadcrumbs, below info it is invisible in production, and a runbook naming the wrong level rebuilds the false negative the line exists to kill |
| security.cookie-host-prefix | session cookies carry __Host-, so a sibling host under a shared registrable domain cannot toss one; legacy bare-named sessions still authenticate through the migration | server-api | covered-deterministic | TestHostPrefixOnProductionCookieShape (pkg/server/auth_cookie_prefix_test.go), TestHostPrefixWithheldWhenItsTermsCannotBeMet (pkg/server/auth_cookie_prefix_test.go), TestPrefixedCookieWinsOverATossedBareCookie (pkg/server/auth_cookie_prefix_test.go), TestHostPrefixedSessionRoundTrip (pkg/server/auth_cookie_prefix_test.go), TestClearAuthCookiesExpiresBothSpellings (pkg/server/auth_cookie_prefix_test.go) | covers the shape production runs (Secure + no Domain), which the older TestSetAuthCookiesAttributes never exercised — both its cases fall outside the prefix's terms, so they passed whichever way this resolved. The round trip is the composition guard: register through the real handler, replay the minted cookie on /api/auth/me. Write-only and read-only tests would both stay green if the two disagreed on the name |
| security.response-headers | the studio serves nosniff / Referrer-Policy / X-Frame-Options / Permissions-Policy and an enforced CSP, on served AND short-circuited responses | server-api | covered-deterministic | TestSecurityHeadersOnDocuments (pkg/server/middleware_security_headers_test.go), TestSecurityHeadersRideShortCircuitedResponses (pkg/server/middleware_security_headers_test.go), TestSecurityHeadersKillSwitch (pkg/server/middleware_security_headers_test.go), studio/e2e/specs/security-headers.spec.ts | script-src is asserted by exact source list, not substring: 'unsafe-inline' there would turn an injected string back into running code. The kill-switch test proves the headers come from THIS middleware and not from a layer that happens to set them. Browser half in studio-ui.csp |
| studio-ui.dsl-vocabulary | Monaco paints every generated lexer keyword and registered property, list markers and arrow chains while preserving prompt text | studio-ui | covered-deterministic | studio/src/lib/iterLanguage.test.ts, TestGeneratedDSLDocsAreFresh (pkg/dsl/spec/docs_fresh_test.go) | The real Monarch engine consumes the generated module; dsl:check compares it with the parser registry. |
| studio-ui.csp | the SPA boots and mounts Monaco under the enforced CSP with zero violations and zero third-party-CDN requests | studio-ui | covered-deterministic | studio/e2e/specs/security-headers.spec.ts | the only honest oracle for a CSP is a browser: a policy is worth its directives only if the app still works under it. Real chromium against the real binary. It also pins the editor's self-hosting — Monaco used to be fetched from cdn.jsdelivr.net at runtime — and asserts the editor actually rendered the fixture's source, so a blank page (what a too-strict CSP produces) cannot pass |
| security.origin-gate-dispatcher | iterion dispatch's own mux refuses a cross-origin state-changing POST while still serving its board UI and no-Origin callers | cli | covered-deterministic | TestDispatchDaemonRefusesCrossOriginWrites (e2e/cli_dispatch_daemon_test.go) | the daemon builds its HTTP surface by hand rather than through Server.routes(), so the studio's protections reached none of it — a drive-by page could create a board card, move it into the dispatcher-eligible state and force the poll, which runs a workflow with tools on the host. Drives the real daemon over loopback; reverting the BrowserGuard wrap reproduces the 201/202 an adversarial review measured live |
| security.agent-binding-cookies | the per-flow OIDC / forge binding cookies (RFC 9700 §4.7.1 login-CSRF guard) carry __Host-, refuse a tossed bare cookie outright, and clear both spellings | server-api | covered-deterministic | TestAgentBindingCookiesCarryTheHostPrefix (pkg/server/agent_binding_cookie_test.go), TestTossedAgentBindingCookieIsRefused (pkg/server/agent_binding_cookie_test.go), TestAgentBindingCookiesClearBothSpellings (pkg/server/agent_binding_cookie_test.go) | these were left bare on the in-code reasoning that a cross-site script "can't set a cookie for iterion's origin" — the claim a shared registrable domain falsifies. No legacy fallback here, deliberately: an interrupted flow is a re-login, a defeated CSRF guard is an account takeover |
| security.desktop-cookie-handoff | the desktop harvests a rotated refresh token, and strips the cloud's session cookies from its webview, under either cookie spelling | cli | covered-deterministic | TestCloudLoginHarvestsEitherCookieSpelling (cmd/iterion-desktop/cloud_cookie_prefix_test.go), TestHarvestRefreshCookieMatchesBothSpellings (cmd/iterion-desktop/cloud_cookie_prefix_test.go), TestProxyStripsBothSpellingsFromTheWebview (cmd/iterion-desktop/cloud_cookie_prefix_test.go) | the row exists because a private copy of the cookie literal drifted from the server's: the harvest silently returned "", the jar kept the PREVIOUS token, and replaying it made the server revoke every session that user held, on every device, on a ~13-minute loop. The fake cloud is parameterised over both spellings — it only ever emitted the bare one, which is why nothing caught it |
| studio-ui.first-paint | the editor stays off the critical path: the runs list does not fetch Monaco, and opening the editor does | studio-ui | covered-deterministic | studio/e2e/specs/first-paint.spec.ts | measured in the browser, not read off the bundle — a chunk name in the entry file may be a lazy import() specifier. Both halves are asserted so "lazy" cannot degrade into "never": one static import from app chrome had put 4.2 MB of JS plus a render-blocking 158 KB stylesheet on every page |
| cli.sandbox-doctor | iterion sandbox doctor [--strict] host/run pre-flight diagnosis | cli | covered-deterministic | TestRunSandboxDoctorStrictNoSandbox (pkg/cli/sandbox_strict_test.go), TestRunNetworkStrictChecks (pkg/cli/sandbox_strict_test.go) | |
| cli.studio | iterion studio boots the server and reports its port | cli | covered-deterministic | TestRunStudio_OnReady_RandomPort (pkg/cli/studio_test.go), TestIsLoopbackBindHost (pkg/cli/studio_bind_test.go) | |
| cli.server | iterion server boots the HTTP server without the studio launcher | cli | covered-deterministic | TestServerCommandBootsLocalModeAndShutsDownOnSignal (e2e/cli_server_boot_test.go), TestServerCommandFailsLoudlyOnBusyPort (e2e/cli_server_boot_test.go) | spawns the built binary as a subprocess and signals THAT child (not the test binary), so the daemon-SIGTERM aliasing of cli.dispatch is not reproduced |
| cli.runner | iterion runner boots a cloud runner pod from its config | cli | covered-deterministic | TestRunnerCommandRefusesLocalMode (e2e/cli_server_boot_test.go), TestRunnerCommandRefusesBrokenConfig (e2e/cli_server_boot_test.go) | the entry-point posture (mode gate + config loader) is what is deterministic; the claim/execute loop IS a NATS/Mongo/S3 consumer and is covered by cloud.runner-pod |
| cli.bots-templates | iterion bots templates lists the templates bots create accepts | cli | covered-deterministic | TestBotsTemplates_MatchesStudioGallery (pkg/cli/bots_create_test.go) | pins the CLI list against botscaffold.Templates(), the same source the studio gallery renders |
| bots.install | bot bundle install from a git URL or local path (layout validation, name override, existing-needs-force) | bots | covered-deterministic | TestInstall_SingleBundleRoot (pkg/botinstall/install_test.go), TestInstall_MalformedRejected (pkg/botinstall/install_test.go), TestInstall_ExistingNeedsForce (pkg/botinstall/install_test.go) | iterion bots install and the studio install endpoint are both thin wrappers over this core |
| plugins.lifecycle-run | iterion plugin run <name> <phase> executes a plugin's lifecycle command | plugins | covered-deterministic | TestRunLifecycle (pkg/plugin/lifecycle_test.go), pkg/server/plugins_routes_test.go | |
| cli.dispatch | iterion dispatch daemon boots from a config with the bot catalogue | cli | covered-deterministic | TestDispatchDaemonBootsServesAndStopsOnSignal (e2e/cli_dispatch_daemon_test.go), TestDispatchDaemonFailsLoudlyOnABusyPort (e2e/cli_dispatch_daemon_test.go), TestDispatchDaemonRejectsAnInvalidConfig (e2e/cli_dispatch_daemon_test.go) | in-process boot on a temp store + loopback port: asserts /healthz, the server-info advertisement, the started actor's state and the YAML it runs, the mounted native-board routes, then the SIGTERM clean exit and the non-zero exit on a bind failure |
| cli.migrate-to-cloud | iterion migrate to-cloud local store → Mongo/S3 | cli | covered-deterministic | TestMigrateToCloud_TransfersEveryVersionAndIsIdempotent (cmd/iterion/migrate_to_cloud_test.go), TestMigrateToCloud_DryRunWritesNothing (cmd/iterion/migrate_to_cloud_test.go), TestMigrateToCloud_MissingRunIsAnError (cmd/iterion/migrate_to_cloud_test.go), pkg/cli/migrate_orgs_test.go, pkg/cli/migrate_run_paths_test.go | the command's own walker between two real stores (every artifact version, --dry-run writes nothing, re-runs don't duplicate, an unknown run is an error), plus the sibling migrate orgs backfill and run-path rewrite. The S3 upload half is cloud.migrate-blobs |
| cli.mcp-server | iterion mcp operator MCP server (local_/remote_ tools) | cli | covered-deterministic | TestMCPServer_DetachedRunSurvivesServerExit (e2e/mcp_server_test.go), pkg/operatormcp/tools_local_test.go | |
| cli.supervise | iterion supervise attaches to a managed run or a raw claude session | cli | covered-deterministic | pkg/supervise/coordinator_test.go, pkg/supervise/transcript_test.go | |
| cli.remote | iterion remote typed subcommands against a cloud instance | cli | covered-deterministic | TestRemoteRunsLaunch_SendsSourceAndVars (pkg/cli/remote_test.go), TestRemoteRunsFollow_CursorAndTerminal (pkg/cli/remote_test.go) | |
| cli.remote-assistant-mission | iterion remote runs mission start / get / list / stop exposes the durable target-scoped mission API and JSON output | cli | covered-deterministic | TestRemoteRunsMissionCommandsUseTargetScopedAPI (pkg/cli/remote_test.go), TestAssistantMissionCreateReattachesTerminalInvocationWithoutRenewal (pkg/server/assistant_mission_test.go) | typed remote client plus real authenticated API handler |
| cli.remote-login | browser loopback CLI-auth token mint + persistence | cli | covered-deterministic | TestResolveRemoteConfig_EnvMode (pkg/cli/remote_test.go), pkg/server/auth_routes_test.go | |
| cli.version | iterion version [--commit] | cli | covered-deterministic | cmd/iterion/version_test.go | |
| cli.bench-asymptote | iterion bench asymptote convergence benchmark | cli | covered-deterministic | TestBenchAsymptoteMeasuresTheConvergenceCurve (e2e/cli_bench_asymptote_test.go), TestBenchAsymptoteComparesTwoGroups (e2e/cli_bench_asymptote_test.go), TestBenchAsymptoteWritesTheMarkdownReport (e2e/cli_bench_asymptote_test.go), TestBenchAsymptoteRefusesAnIncompleteRequest (e2e/cli_bench_asymptote_test.go) | the command re-runs nothing — it derives the curve from what past runs persisted — so real stub-driven runs of a judge-in-a-loop fixture supply the same events an operator's runs would |
| backends.selection-explicit | node backend: / workflow default_backend: selects the delegate | backends | covered-deterministic | pkg/backend/model/resolve_backend_test.go | |
| backends.autodetect | credential probing picks a backend when none is declared | backends | covered-deterministic | pkg/backend/detect/detect_test.go, pkg/backend/model/resolve_backend_test.go | live variant: TestLive_Feat_BackendAutodetect |
| backends.claw | claw in-process client: generation, retry, cache observability | backends | covered-deterministic | pkg/backend/model/claw_backend_test.go, pkg/backend/model/generation_test.go | real-provider behaviour: covered-live by TestLive_Lite_ClawComprehensive |
| backends.claude-code | claude_code CLI delegate: append-system-prompt, setting sources | backends | covered-deterministic | pkg/backend/delegate/claude_code_cred_test.go | |
| backends.pi | pi delegate incl. the RPC session + embedded extension | backends | covered-deterministic | pkg/backend/delegate/pi_rpc_test.go, pkg/backend/delegate/pi_mcp_test.go | |
| backends.kimi | kimi CLI delegate through the generic CLI-agent seam | backends | covered-deterministic | pkg/backend/delegate/cliagent_test.go | |
| backends.grok | grok CLI delegate through the generic CLI-agent seam | backends | covered-deterministic | pkg/backend/delegate/grok_test.go | |
| backends.opencode | opencode CLI delegate through the same seam: argv, stream parsing, session resume/fork, the untrusted-checkout refusal, and the per-backend diagnostics | backends | covered-live | pkg/backend/delegate/opencode_test.go, pkg/backend/delegate/opencode_session_test.go, pkg/backend/detect/opencode_test.go, pkg/dsl/ir/opencode_backend_test.go, pkg/server/opencode_backend_test.go, pkg/backend/delegate/opencode_live_test.go, TestLive_Feat_OpenCodeProjectScreen, TestCLILiveOpenCodeSessionResume (pkg/backend/delegate/opencode_session_clilive_test.go) | gate: tool — NO ledger row is owed: no money is spent anywhere on this row. the live half is two conformance benches against the REAL CLI. One asks which directories it loads resources from and requires iterion never to accept a layout the CLI loads from — it needs no credential (plugins load before any model turn) and every row asserts the CLI actually loaded, so a stub on PATH fails instead of passing vacuously. The other (task test:clilive) resumes and forks sessions through iterion's own Execute against a scripted model on 127.0.0.1 and reads the rehydration off what the CLI sends. The permission-gate cell still needs a live denial |
| backends.codex | Codex CLI delegate and supported DSL selection, including explicit native web_search | backends | covered-live | pkg/dsl/ir/compile_test.go, pkg/backend/delegate/delegate_test.go, pkg/backend/delegate/codex_web_search_test.go, TestLive_Feat_CodexWebSearch | gate: paid — its ledger row test:live:feat:codex-web-search (e2e/testdata/live/ledger.json) reads never — an honest gap, not a green line. a real hosted search cannot be exercised deterministically without a provider credential and network; the live test proves current URLs + readonly write refusal + WebSearch lifecycle, while deterministic tests pin live/disabled config, distinct WebSearch/Bash names, and incompatible-version refusal |
| backends.system-prompt-composition | per-backend SystemPromptMode (Standalone/Append/AuthoredBase) | backends | covered-deterministic | pkg/backend/delegate/delegate_test.go | |
| backends.reasoning-effort | reasoning_effort propagation and wire remapping | backends | covered-deterministic | pkg/backend/model/effort_test.go | |
| backends.ultracode | ultracode mode: xhigh + orchestration prerogative + C089 | backends | covered-deterministic | pkg/dsl/ir/ultracode_test.go, pkg/backend/model/effort_test.go | live behaviour on 4.8: TestLive_Feat_Ultracode |
| backends.retry-classification | transient vs fatal backend errors, retry + feedback | backends | covered-deterministic | pkg/backend/model/executor_retry_classification_test.go, pkg/backend/model/network_retry_test.go | |
| backends.cost-accounting | per-call cost/token accounting reaching the run totals | backends | covered-deterministic | TestMetricsEmitter_delegateFinished_costReachesRunTotals (pkg/runner/metrics_test.go), TestMetricsEmitter_clawCostIsNotCountedTwice (pkg/runner/metrics_test.go), TestRunBudgetOverrideCapsTheRun (e2e/cli_budget_override_test.go), pkg/backend/cost/cost_test.go, pkg/backend/model/executor_hooks_cost_test.go | the whole money path: the delegate prices a call, the executor carries it on the event, the emitter accumulates it into RunTotals (and does NOT double-charge claw, which emits both a step and a delegation total), and the accumulated cost is what trips the run's own budget cap |
| backends.oauth-forfait | subscription OAuth paths (Anthropic auth token, ChatGPT codex) | backends | covered-deterministic | pkg/backend/model/openai_forfait_ctx_test.go, pkg/secrets/subscription_oauth_test.go | credential resolution is deterministic; a real forfait call is excluded (needs a live subscription) |
| backends.host-chatgpt-forfait-in-sandbox | a host-wide ITERION_OPENAI_USE_OAUTH=1 makes the host's ChatGPT forfait claw's instrument over OPENAI_API_KEY for a run holding no OpenAI credential of its own, and a sandboxed openai/… node spends it too: each dispatch writes the host's login, reduced to its access token and account (no refresh token, id token or API key), into a container dir of its own, points that node's runner at it with the OAuth path forced, and removes it after (on a cancelled node too, and after a failed write, which fails the node); nothing is delivered without the operator's exact 1, under the refusal 0, an OPENAI_BASE_URL, a switch or codex home a project .env planted (a warning says so, and only where the forfait would otherwise have gone), the run's own key, pinned key or forfait, a host with no ChatGPT login, a node of another provider, or a sandbox whose commands run on the host (noop, however wrapped) — the same instrument in process and in the container for a process resolving afresh (a long-lived one keeps the in-process client it resolved first), cloud lookups and local | backends | covered-deterministic | TestSandboxedClawSpendsWhatTheHostSpends (pkg/backend/model/claw_host_forfait_test.go), TestSandboxedClawSpendsWhatTheHostSpendsLocally (pkg/backend/model/claw_host_forfait_test.go), TestSandboxedClawNodeGetsTheHostForfaitTheSwitchOrders (pkg/backend/model/claw_host_forfait_test.go), TestTheRunsOwnOpenAIInstrumentKeepsTheHostForfaitHome (pkg/backend/model/claw_host_forfait_test.go), TestTheKillSwitchKeepsTheHostForfaitHomeWhateverTheRunHolds (pkg/backend/model/claw_host_forfait_test.go), TestNoHostForfaitNothingIsDelivered (pkg/backend/model/claw_host_forfait_test.go), TestAHostRunGetsNoForfaitDelivery (pkg/backend/model/claw_host_forfait_test.go), TestOnlyAnOpenAINodeGetsTheHostForfait (pkg/backend/model/claw_host_forfait_test.go), TestAFailedDeliveryFailsTheNode (pkg/backend/model/claw_host_forfait_test.go), TestEachDispatchGetsItsOwnCopyRemovedAfterIt (pkg/backend/model/claw_host_forfait_test.go), TestACancelledNodeStillRemovesItsCopy (pkg/backend/model/claw_host_forfait_test.go), TestAGatewayKeepsTheHostForfaitHome (pkg/backend/model/claw_host_forfait_test.go), TestWithoutTheOrderTheHostForfaitStaysHome (pkg/backend/model/claw_host_forfait_test.go), TestAProjectDotEnvCannotShipTheHostForfait (pkg/backend/model/claw_host_forfait_test.go), TestAPlantedSwitchIsNoAlarmWhereNothingWouldHaveGone (pkg/backend/model/claw_host_forfait_test.go), TestTheCopyIsTheLoginTheHostJudged (pkg/backend/model/claw_host_forfait_test.go), TestDefaultRulesAllowEveryOpenAIWireHost (pkg/sandbox/netproxy/matcher_test.go) | the container is a fake driven through the real dispatch (executeViaSandboxRunner) whose filesystem is a test directory where the real write and removal scripts run; a real ChatGPT-forfait call and a real docker/kubernetes exec are not re-proven here |
| backends.glm-routes-to-zai | a GLM model on the anthropic wire (bare glm-* on claude_code or pi, anthropic/glm-* on claw) is a z.ai route: it pins zai in the funding walk, its task carries the zai hint when it has none and a z.ai key is reachable, claw in process and in the sandbox's env factory spends the z.ai key (and its subscription guard spares it), pi receives the pinned z.ai key of its own route; a GLM call honours the operator's ANTHROPIC_BASE_URL on claw as on claude_code; a GLM spec naming another provider (openrouter/z-ai/glm-4.6) keeps that provider | backends | covered-deterministic | TestGLMRouteIsFundedWithTheZAIKey (pkg/backend/model/glm_routes_to_zai_test.go), TestGLMOnAnotherProviderKeepsItsOwnRoute (pkg/backend/model/glm_routes_to_zai_test.go), TestGLMTaskWithoutHintIsRoutedToZAI (pkg/backend/model/glm_routes_to_zai_test.go), TestClawPinnedProviderForGLMIsZAI (pkg/backend/model/glm_routes_to_zai_test.go), TestClawInProcessGLMSpendsTheZAIKey (pkg/backend/model/glm_routes_to_zai_test.go), TestClawEnvFactoryServesGLMThroughZAI (pkg/backend/model/glm_routes_to_zai_test.go), TestClawSubscriptionGuardSparesAGLMNode (pkg/backend/model/glm_routes_to_zai_test.go), TestForwardableProviderEnv_OtherProviderIgnoresTheForfait (pkg/backend/model/claw_sandbox_env_test.go), TestPiRouteEnvCarriesThePinnedKeyOfItsOwnRoute (pkg/backend/delegate/pi_test.go), TestPiExecuteHandsTheRouteKeyToTheTransport (pkg/backend/delegate/pi_test.go), TestRouteProviderHintSendsGLMToZAIOnTheHintedBackends (pkg/backend/model/glm_routes_to_zai_test.go), TestGLMHonoursTheOperatorsBaseURL (pkg/backend/model/glm_routes_to_zai_test.go) | the Claude forfait crosses into a claw sandbox only for a claude model on claw's anthropic provider; a real z.ai call is covered-live only through the review-pr GLM route |
| backends.bedrock-vertex-foundry | AWS Bedrock / GCP Vertex / Azure Foundry providers | backends | excluded | needs real cloud credentials for AWS/GCP/Azure; claw's SDK paths are unit-tested with mocked clients and iterion adds no logic of its own | |
| backends.model-quality | real model output quality / value-for-money grading | backends | covered-live | pkg/benchmark/quality/quality_test.go, TestLive_Bot_RgaaAudit (e2e/live_bot_rgaa_audit_test.go) | gate: paid — the receipt: the ledger row test:live:bot:rgaa-audit reads pass (2026-10-13). the essence of the feature IS the live model; the judge panel is report-only by default. The snapshot engine itself is free (task test:live:quality:unit, no credential); what needs a real model is the grading of its output, which rides a live bot run — e2e/live_quality_test.go holds assessQuality and declares no test of its own, so it is the harness, not the witness |
| tools-mcp.tool-registry | tool registry + per-node allow lists (claw-native names) | tools-mcp | covered-deterministic | pkg/backend/tool/registry_test.go | |
| tools-mcp.tool-aliases | manifest-gated Read/Bash/Grep aliases after exact/MCP resolution | tools-mcp | covered-deterministic | TestToolAliasesThroughRealEngineAndClawTools (pkg/runtime/tool_aliases_test.go), TestToolAliasesReachClawFallbackAndRecoveryAgent (pkg/runtime/tool_aliases_test.go), TestToolAliasesAreRebuiltOnFreshEngineResume (pkg/runtime/tool_aliases_test.go), TestToolAliasesAndPolicyUseTheSameMCPResolution (pkg/backend/model/tool_aliases_test.go) | real Claw tools on a temporary proof file; only provider call choice scripted; old-runner refusal proof in docs/tool-name-aliases.md |
| tools-mcp.claw-builtins | claw built-in tools (read/write/bash/glob/grep/edit/web_fetch) | tools-mcp | covered-deterministic | pkg/backend/tool/claw_builtins_test.go | live variant: TestLive_Lite_ClawBuiltinTools |
| tools-mcp.mcp-lifecycle | MCP server startup, health check and --skip-mcp-health | tools-mcp | covered-deterministic | pkg/backend/mcp/config_test.go, TestSkipMCPHealthFromEnv (pkg/cli/run_skipmcp_test.go) | |
| tools-mcp.mcp-transports | MCP stdio / streamable-http / legacy-sse transports | tools-mcp | covered-deterministic | pkg/backend/mcp/config_test.go, pkg/backend/delegate/pi_mcp_test.go | |
| tools-mcp.mcp-oauth | MCP OAuth broker / PKCE wiring | tools-mcp | covered-deterministic | pkg/backend/mcp/oauth_test.go | a real third-party OAuth consent flow is excluded (see integrations.third-party-oauth) |
| tools-mcp.strict-mcp-isolation | claude_code nodes get ONLY the resolved MCP set (--strict-mcp-config); host ~/.claude.json servers excluded, ITERION_CLAUDE_CODE_STRICT_MCP=0 opts back in | tools-mcp | covered-deterministic | TestClaudeCodeSpawn_StrictMCPConfigByDefault (pkg/backend/delegate/claude_code_strict_mcp_test.go), TestBuildArgs_StrictMCPConfig (pkg/backend/delegate/claudesdk/buildargs_test.go) | |
| tools-mcp.board-tools | board capability tools over stdio, HTTP and in-process claw | tools-mcp | covered-deterministic | TestBoardDispatcher_E2E_BotCreatesAndDispatches (e2e/board_dispatcher_test.go), pkg/backend/tool/claw_board_tools_test.go | |
| tools-mcp.ask-user | ask_user / ask_user_async / await_answers MCP surface | tools-mcp | covered-deterministic | pkg/askusermcp/http_test.go, TestAwaitAnswersReleasedByAnswer (e2e/async_interaction_test.go) | |
| tools-mcp.permission-gate | permission gate blocks/asks on a non-allow-listed tool call | tools-mcp | covered-deterministic | pkg/backend/model/permission_gate_test.go, pkg/backend/permission/permission_test.go | live variants: TestLive_Feat_Permission_Deny / _Ask |
| tools-mcp.permission-gate-external-hook | the gate holds on a CLI backend through its native PreToolUse hook (grok, kimi), deny-only | tools-mcp | covered-deterministic | pkg/backend/permissionhook/hook_test.go, pkg/backend/delegate/cliagent_test.go, pkg/dsl/ir/validate_fallbacks_test.go | live proof (filesystem sentinel, not model prose): TestLive_Feat_Permission_Deny_Grok / _Kimi |
| tools-mcp.secret-guard | secret placeholders never leak into tool input/output | tools-mcp | covered-deterministic | pkg/backend/model/hooks_secretguard_test.go, pkg/backend/model/secretguard_binding_hosts_test.go | |
| tools-mcp.computer-use | read_image / screenshot / computer_use dispatch | tools-mcp | covered-deterministic | pkg/backend/tool/claw_builtins_test.go | headless-unavailable propagation is the deterministic half; live use is TestLive_Lite_ClawReadImage |
| tools-mcp.tool-display | human-readable rendering of tool calls in console/report | tools-mcp | unit-only | pkg/backend/tooldisplay/display_test.go | read again 2026-10-31 (#2798): the premise holds — HeaderDetail / BlockBody / ResultDisplay are pure functions over (tool name, JSON payload), their string shape pinned by the cited suite, and the stack around them (delegate stream, model executor, hook paths) is already exercised by the deterministic rows the rendered lines reach — observability.report-generation renders a real run's events into report.md, server-api.run-console serves the same stream to the console — so an e2e here would re-assert string shape without adding risk coverage. Done being read |
| observability.report-generation | report.md chronological rendering from events + artifacts | observability | covered-deterministic | TestReportRendersChronologicalRunReport (e2e/cli_report_test.go) | the artifact table lifts each artifact's conventional summary: field; node outputs without one are intentionally not inlined |
| observability.metrics | benchmark.CollectMetrics over a finished run | observability | covered-deterministic | TestCIFix_HappyPath (e2e/e2e_test.go), pkg/benchmark/benchmark_test.go | |
| observability.alerts | stall/budget/failure alerting + liveness heartbeat | observability | covered-deterministic | pkg/alert/manager_test.go | |
| observability.completion-webhooks | run-completion webhooks behind the SSRF guard | observability | covered-deterministic | pkg/notify/completion_test.go, pkg/secure/httpdial/httpdial_test.go | |
| observability.user-notifications | run-outcome web-push notifications with per-episode dedup | observability | covered-deterministic | pkg/usernotify/dispatcher_test.go | |
| observability.otlp-tracing | OTLP exporter wiring for runs/server | observability | covered-deterministic | pkg/cloud/tracing/tracing_test.go, pkg/benchmark/otlp_test.go | an end-to-end span assertion needs a real collector; setup/shutdown/no-endpoint paths are asserted here |
| observability.effective-model | the model that actually ran is recorded on delegate_started/delegate_finished, run.json nodes_served, and a model_drift event when it differs from the declared model: | observability | covered-deterministic | TestEffectiveModelRecordThroughEngine (pkg/runtime/delegate_observability_engine_test.go), TestDelegateModelReachesStore (pkg/backend/model/delegate_observability_test.go), TestRecordNodeServedRoundTrips (pkg/store/nodes_served_test.go) | promoted from unit-only (#2797): the engine path produces the record for free — a scripted delegate backend stamps what a CLI parses off its stream, and muting emitModelDrift or recordServed reddens the engine test on its own assertion. CLI backends never emit llm_request.model, so these surfaces ARE the durable record; claw still also emits llm_request |
| observability.facade-routing | a node served through an Anthropic-shaped facade says so: nodes_served[<node>].fingerprint plus one model_served_via_facade event per node and facade, raised from the success path only, with no credential from the operator-supplied base URL in the recorded label | observability | covered-deterministic | TestFacadeRoutingRecordThroughEngine (pkg/runtime/delegate_observability_engine_test.go), TestDelegateFacadeRoutingReachesStore (pkg/backend/model/delegate_observability_test.go), TestDelegateFacadeRoutingSilentOnFailure (pkg/backend/model/delegate_observability_test.go), TestProviderFingerprint_FacadeBaseURLCarriesNoCredential (pkg/backend/delegate/claude_code_cred_test.go) | promoted from unit-only (#2797): the record needs no real facade — a scripted backend stamps the fingerprint a facade session leaves, and muting emitFacadeRouting reddens the engine test; what a real facade credential would still buy is only the facade's own alias semantics, which iterion does not implement. Declared and effective AGREE under a facade, so model_drift cannot fire — the fingerprint is the only evidence |
| sandbox.docker-driver | docker driver: container lifecycle, workspace bind-mount | sandbox | covered-deterministic | TestTheStandInDaemonSeesTheContainersLifecycleAndItsWorkspaceBindMount (pkg/sandbox/docker/fake_daemon_test.go), TestAStaleContainerNameConflictIsForceRemovedAndTheStartRetriedOnce (pkg/sandbox/docker/fake_daemon_test.go), TestAThoroughlyConflictedNameFailsTheStartWithoutSilentRecovery (pkg/sandbox/docker/fake_daemon_test.go), TestPrepareProbesTheImageAndPullsOnlyWhenTheDaemonLacksIt (pkg/sandbox/docker/fake_daemon_test.go), e2e/live_feat_sandbox_net_test.go | the driver shells out to the container CLI, so a stand-in daemon binary set as the driver's Runtime — the fault-injection seam image_env_test.go's fakeInspector already uses — drives the PUBLIC driver path with no daemon at all: the full docker run argv (detached, auto-removed, named after the run, both managed labels, workdir at the workspace folder, the WORKSPACE BIND-MOUNT type=bind,source=<host tree>,target=<container folder>, the spec's extra mounts, user, tmpfs, sorted env pairs, the image ending in sleep infinity), the process-namespace probe read back from the daemon, exec against the started container (workdir defaulting to the bind-mount target), the stop-then-force-remove cleanup and its idempotence, Prepare's probe-before-pull ordering (an image the daemon holds is never pulled), and the stale-name conflict — force-remove by name, then ONE retry of the IDENTICAL argv; a conflict that survives the retry fails the Start with the daemon's own cause. A mutant that drops the workspace bind-mount from Start turns the first test red, and a muted conflict predicate turns the retry test red (#2796). Only the daemon's half — a real bind-mount sharing the host tree, a real in-container process — needs a container runtime: TOOL-GATED, NEVER PAID — no credential anywhere on this row, and its live target (task test:live:feat:sandbox-net: docker + iterion-sandbox-full, no LLM) reads never in the ledger. The noop driver is a host passthrough and asserts none of this (pkg/sandbox/noop/noop_test.go) |
| sandbox.spec-resolution | sandbox spec resolution (auto / devcontainer / image / none) | sandbox | covered-deterministic | pkg/sandbox/spec_test.go, pkg/sandbox/factory_test.go | |
| sandbox.network-policy | network: allowlist/denylist CONNECT proxy enforcement | sandbox | covered-deterministic | TestSandboxNetworkAllowlist_DeclarationStartsAnEnforcingProxy (pkg/runtime/sandbox_network_policy_test.go), TestSandboxNetworkDenylist_BlocksOnlyTheListedHosts (pkg/runtime/sandbox_network_policy_test.go), TestSandboxNetworkOpen_StartsNoProxy (pkg/runtime/sandbox_network_policy_test.go), pkg/sandbox/netproxy/proxy_test.go, e2e/live_feat_sandbox_net_test.go | the proxy runs on the HOST, so the whole wire is deterministic after all: the declaration compiled from real .bot source starts an enforcing proxy, an allowed host tunnels to a loopback echo server (through the host alias — the proxy refuses any other way to its own loopback, pkg/sandbox/netproxy/proxy_loopback_test.go), a host outside the policy is refused AND emits the network_blocked event the operator sees, an un-tokened client is refused (not an open relay), and network: open / no block at all start no proxy. Forcing ResolveNetworkPolicy to open turns both policy tests red. Only the container-side half — the sandboxed process actually routing its egress through the injected HTTPS_PROXY — still needs the live layer |
| sandbox.host-state | host_state: auto / none mounts of ~/.iterion and ~/.claude | sandbox | covered-deterministic | TestApplyHostStateMounts_ClaudeConfigFile (pkg/runtime/sandbox_mounts_test.go), TestApplyHostStateMounts_HomeNestedBindParentsWritable (pkg/runtime/sandbox_mounts_test.go) | pkg/sandbox/spec_test.go only validates the enum string; the mounts themselves are asserted in the runtime tests cited here |
| sandbox.image-env-screen | the env a repository-authored IMAGE bakes in is screened once the image is local (after the pull, after a build: materialised it) and before anything starts: the credential and routing sub-class of the deny class is NEUTRALISED with --env NAME= (set and empty, which those consumers read as absent), the rest of the class is REPORTED and keeps its value (an emptied PATH, HOME, loader path or KUBECONFIG breaks an image doing its job), a driver that cannot read an image's config says NOT screened with the reason rather than passing as clean, the host-side ITERION_SANDBOX_IMAGE_ENV_ALLOW lets one named variable through and no other (never from the workspace, so the repository under review cannot allowlist itself), a malformed planted name is reported instead of disarming the batch, neither a planted name nor the repository's image reference nor a driver's error can write a line of its own into the operator's log (all three are quoted), and the whole class travels to the operator as one sandbox_image_env_planted event whose dispositions are DEDUCED from the container that started (the image's ENV compared to the .Config.Env the driver reads back, so the record cannot contradict docker inspect; a driver that cannot read a container publishes undeduced and not one disposition) — planted whenever the class was read, neutralized / reported whenever the container was, the four conditional ones when they hold — plus one warning per disposition in the run's log (#2317) | sandbox | covered-deterministic | TestTheImageEnvScreenPartitionsWhatAnImagePlants (pkg/runtime/sandbox_image_env_screen_test.go), TestTheImageEnvScreenHonoursTheHostAllowKnob (pkg/runtime/sandbox_image_env_screen_test.go), TestAKnobAllowedNameTheDriverOverridesIsFiledWhereTheContainerSaysSo (pkg/runtime/sandbox_image_env_screen_test.go), TestNoDispositionTravelsWhenTheContainerCannotBeRead (pkg/runtime/sandbox_image_env_screen_test.go), TestAContainerValueNothingExplainsIsPublishedAsUnexplained (pkg/runtime/sandbox_image_env_screen_test.go), TestTheDeductionReadsTheLastEntryTheWayDockerResolvesIt (pkg/runtime/sandbox_image_env_screen_test.go), TestTheImageEnvScreenSaysWhenItScreenedNothing (pkg/runtime/sandbox_image_env_screen_test.go), TestARefusedNeutralisationNeverPassesAsDone (pkg/runtime/sandbox_image_env_screen_test.go), TestAMalformedPlantedNameDoesNotDisarmTheRest (pkg/runtime/sandbox_image_env_screen_test.go), TestTheScreensWarningTellsTheTruthAfterARefusal (pkg/runtime/sandbox_image_env_screen_test.go), TestTheScreensWarningDoesNotLetAnImageWriteLinesInTheLog (pkg/runtime/sandbox_image_env_screen_test.go), TestTheImageEnvEventCarriesBothHalvesOfTheClass (pkg/runtime/sandbox_image_env_event_test.go), TestTheScreenedSpecAndTheEventComeFromTheSameScreen (pkg/runtime/sandbox_image_env_event_test.go), TestARefusedNeutralisationTravelsUnderItsOwnKeyAndStopsTheBoot (pkg/runtime/sandbox_image_env_event_test.go), TestADoubledPlantedNameIsOneFinding (pkg/runtime/sandbox_image_env_event_test.go), TestEveryDriverStartScreensTheImagesEnv (pkg/runtime/sandbox_image_env_wiring_test.go), TestTheContainerStartsWithTheSpecTheScreenReturned (pkg/runtime/sandbox_image_env_boot_test.go), TestNeutralizedImageEnvReachesTheRunArgv (pkg/sandbox/docker/image_env_test.go), TestEveryJavaTrustPairCarriesItsOwnValue (pkg/sandbox/docker/image_env_test.go), TestStartedContainerEnvRefusesWhatItCannotRead (pkg/sandbox/docker/image_env_test.go), TestScreenImageEnvNameAnswersItsRows (pkg/sandbox/devcontainer/image_env_disposition_test.go), e2e/live_feat_sandbox_image_env_test.go | The whole seam is deterministic except the last link: the screen's partition, the host knob, the refusal paths, the event's payload, the deduction (a double that MODELS the container the way the daemon resolves it: the image's ENV, then the spec's --env pairs, then the driver's own) and the WIRING (a go/ast guard: every driver.Start of pkg/runtime starts the spec the screen seam returned AND hands that screen's record to the publication seam, or is declared exempt with its reason) are proven without a container; TestNeutralizedImageEnvReachesTheRunArgv proves the docker driver turns the neutralisation into --env NAME= argv. Only the in-container READING — a process started from the planted image finding the name empty — needs a daemon, so it lives in the live layer (task test:live:feat:sandbox-image-env, docker and a local base image, no LLM and no credential); the deduction's own end-to-end witness runs beside it under the dockerlive tag (TestLiveTheImageEnvEventAgreesWithDockerInspect, pkg/runtime/sandbox_image_env_dockerlive_test.go: a throwaway image baking one name per disposition, the five dispositions compared to docker inspect of the container the REAL driver created, the host knob naming both a name nothing touches and one the driver's own Start overrides). |
| sandbox.kubernetes-driver | kubernetes driver: capabilities, pod/network manifests, mounts, GC, orphan sweep | sandbox | covered-deterministic | TestPodManifestStructure (pkg/sandbox/kubernetes/driver_test.go), TestBuildNetworkPolicyShape (pkg/sandbox/kubernetes/driver_test.go), TestValidateSpecMatchesPrepare (pkg/sandbox/kubernetes/validate_test.go), pkg/sandbox/kubernetes/gc_test.go, pkg/sandbox/kubernetes/mounts_test.go | the manifest/spec/GC half is deterministic (57 unit tests, same shape as the docker driver); only the real pod lifecycle needs a cluster — see sandbox.kubernetes-pod-lifecycle |
| sandbox.kubernetes-pod-lifecycle | a k8s sandbox pod is actually created, executed against and torn down | sandbox | excluded | re-measured in the tree 2026-10-31 (#2796): no fake apiserver harness exists — go.mod carries no k8s client or envtest (the driver shells out to the kubectl binary), pkg/sandbox/kubernetes holds no httptest apiserver, and the kind job (.github/workflows/arc-kind-probe.yml, scripts/ci/arc-kind-probe.sh) captures facts about the ARC runner's daemon reachability and never drives this driver. A stand-in kubectl could only re-assert iterion's argv — the manifest/apply/GC half already covered, see sandbox.kubernetes-driver — never the ACTUALITY this row promises. The artefacts the promise needs: a live cluster + a registry for the image | |
| sandbox.buildkit | sandbox.build: via docker buildx on the local driver | sandbox | covered-deterministic | TestBuild_ArgvShape (pkg/sandbox/docker/build_test.go), TestBuild_SurfacesStderrTail (pkg/sandbox/docker/build_test.go), TestBuild_NoOpWhenSpecHasNoBuild (pkg/sandbox/docker/build_test.go), TestBuild_RequiresWorkspacePath (pkg/sandbox/docker/build_test.go) | re-measured in the tree 2026-10-31 (#2796): iterion's half of the promise IS deterministic — the driver's Builder refuses inputs escaping the workspace, hands the operator's Dockerfile, context and build-args to buildx in a stable sorted order, tags the image iterion-sandbox-build:<run-id> and --loads it into the local store, consumes the fresh ref into the returned spec (Build nil-ed once spent), surfaces buildx's failed to solve stderr tail, and stays a no-op without a build: block — all through the runBuildx fault-injection seam (a mutant that drops --load turns TestBuild_ArgvShape red). What still needs the daemon is BuildKit's half — layers actually materialising from the Dockerfile: a Docker daemon with BuildKit, TOOL-GATED NEVER PAID, and no live target drives sandbox.build: through the driver today (the live image-env target builds its fixture with plain docker build outside the driver). The kubernetes driver still refuses build: by design (SupportsBuild: false; V2-6: cloud workflows reference pre-built images pinned by digest) |
| plugins.registry | plugin discovery, enable state, builtin embedding | plugins | covered-deterministic | pkg/plugin/plugin_test.go, pkg/plugin/inspect_test.go | |
| plugins.install | plugin install from a git URL or path (incl. bare skills repos) | plugins | covered-deterministic | pkg/plugin/install_test.go, pkg/plugin/skilllib_test.go | |
| plugins.rewriters | rewriter chain rewrites shell commands (rtk compression) | plugins | covered-deterministic | pkg/plugin/plugin_test.go, pkg/dsl/ir/compress_test.go | live end-to-end with the rtk binary: TestLive_Feat_Compress |
| plugins.contributed-skills | plugin skills/commands/agents mirrored into the workspace | plugins | covered-deterministic | TestMirrorInjectedPluginFiles_WritesEachKind (pkg/runtime/contributions_test.go), TestMirrorPluginContributions_SkillLandsInBothForms, TestMirrorInjectedPluginFiles_SkillLandsInBothForms, TestMirrorInjectedPluginFiles_CommandsAndAgentsAreFlat (pkg/runtime/plugin_skills_test.go), TestResolveContributionsFor_DedupsMultipleTeamSourcesSharingAName, TestResolveContributionsFor_LocalPluginStillShadowsTeamSource (pkg/server/cloudpublisher/contributions_dedup_test.go) | the plugin-skills tests pin BOTH shapes (directory <name>/SKILL.md and the flat alias) and keep commands/agents flat; the cloudpublisher pair pins cloud dedup of same-name team sources and a local plugin shadowing a team source |
| plugins.workspace-commands | a prompt opening with /<name> resolves to <workspace>/.claude/commands/<name>.md on claude_code AND claw; on claw the MODEL can also invoke a command mid-conversation through the command tool (#1716) | plugins | covered-live | TestLive_Feat_WorkspaceCommands (e2e/live_feat_workspace_commands_test.go), pkg/backend/model/slash_commands_test.go, pkg/backend/tool/claw_command_test.go | gate: paid — read 2026-10-31 (#2798): KEPT covered-live, not a duplicate — the cited deterministic suites prove everything iterion owns (the claw expansion, the claude_code passthrough, the command tool), while the live run alone witnesses what iterion only consumes: the real claude CLI resolving a project command iterion deliberately passes through untouched, and the two model halves no stub can produce (a real model answering the expanded body; a real model choosing to invoke the command tool mid-conversation). Both parity nodes run tools: [], so resolving the command is the only path to the token its file carries; the third node (via_claw_model_tool) grants tools: [command] on a prompt that does NOT open with the invocation, so only the model-invoked tool carries the token; paid target task test:live:feat:workspace-commands reads never in the ledger — an honest gap, not a green line |
| plugins.workspace-agents | a claw node's agent call with subagent_type: <name> resolves <workspace>/.claude/agents/<name>.md through iterion's resolver (pkg/backend/model/workspace_agents.go, os.Root at the node's workspace, 256 KiB ceiling): the file's body is the child's system prompt, its frontmatter description is reported in the tool result, and tools: intersects the node's effective set — never widens it (#1714) | plugins | covered-live | TestLive_Feat_ClawWorkspaceAgents (e2e/live_feat_workspace_agents_test.go), pkg/backend/model/workspace_agents_test.go, TestTheBackendLoopHandsTheSubagentRunnerTheNodesWorkspace (pkg/backend/model/claw_workspace_agent_wiring_test.go) | gate: paid — the receipt: the ledger row test:live:claw-workspace-agents reads pass (2026-11-02). the live witness keys on an unguessable token and a marker that exists only in the agent file's body — a generic child finds the token but never the marker, and blind-scribe's declared tools: [] (under a node holding no read_file) must yield NO path to the token; the wiring witness drives the real ClawBackend.Execute and reddens when ParentLoop.WorkDir is dropped (the T40 train repair once removed it silently); paid target task test:live:claw-workspace-agents |
| plugins.hooks-merge | plugin hook fragments idempotently merged into settings.json | plugins | covered-deterministic | TestMergePluginHooks_InjectIdempotentRemove (pkg/runtime/plugin_hooks_test.go), pkg/backend/model/settings_hooks_test.go | |
| plugins.private-source | team-scoped private plugin source binding (git + secret ref) | plugins | covered-deterministic | pkg/pluginsource/pluginsource_test.go | |
| server-api.run-console | run console REST: run detail, events, log, node sections | server-api | covered-deterministic | pkg/server/runs_test.go, pkg/runview/service_test.go | |
| server-api.run-launch | launch a run over HTTP with vars/overrides | server-api | covered-deterministic | pkg/runview/service_launch_budget_test.go, pkg/runview/service_launch_dispatch_fields_test.go, pkg/runview/service_launch_pause_test.go, pkg/runview/service_launch_operator_resume_test.go, pkg/server/runs_test.go | budget overrides (local + forwarded on the cloud path) and invalid durations, dispatch fields, the paused-launch and operator-resume shapes, and the HTTP surface itself |
| server-api.run-control | pause / cancel / resume / rewind / bump-loop / raise-budget | server-api | covered-deterministic | pkg/runview/service_commands_test.go | |
| server-api.assistant-missions | authenticated target-scoped mission create/reattach/list/get/stop; generic proposal allowlist, durable FS claims, one-shot rewind execution and receipt reconciliation | server-api | covered-deterministic | TestAssistantMissionCreateReattachesTerminalInvocationWithoutRenewal (pkg/server/assistant_mission_test.go), TestAssistantMissionCoordinatorExecutesOnlyPersistedRewindProposal (pkg/server/assistant_mission_test.go), TestFSStoreRejectsConcurrentTargetAndFencesClaims (pkg/assistantmission/fs_test.go), TestValidateMissionProposalRejectsWatchAndSourceMutation (pkg/server/assistant_mission_test.go) | Mongo shares the same store contract and adds unique partial indexes for invocation/active target |
| server-api.run-ws | live run WebSocket stream | server-api | covered-deterministic | pkg/server/runs_ws_test.go | |
| server-api.review-scope | review scope + diff since the previous human gate | server-api | covered-deterministic | pkg/server/runs_review_scope_test.go | |
| server-api.run-files | run workspace file browse/read/write with path containment | server-api | covered-deterministic | pkg/server/runs_files_test.go | |
| server-api.run-commits | per-run commit list + commit detail | server-api | covered-deterministic | pkg/server/runs_commits_test.go | |
| server-api.preview-proxy | run preview proxy behind the SSRF guard | server-api | covered-deterministic | pkg/server/runs_preview_test.go | |
| server-api.answer-human | answer a human gate / async question over HTTP | server-api | covered-deterministic | pkg/server/runs_answer_uploads_test.go | |
| server-api.queued-messages | operator chat messages queued into a running node's inbox | server-api | covered-deterministic | pkg/backend/model/inbox_test.go, pkg/server/runs_steer_test.go | |
| server-api.bots | bot catalog listing + per-bot metadata over HTTP | server-api | covered-deterministic | pkg/server/bots_routes_test.go, pkg/botregistry/registry_test.go | |
| server-api.board | native kanban REST (CRUD, transitions, labels, views) | server-api | covered-deterministic | pkg/dispatcher/native/http_test.go, pkg/dispatcher/native/store_test.go | |
| server-api.dispatcher-dashboard | dispatcher state/refresh/cancel HTTP surface | server-api | covered-deterministic | TestDispatcherE2E_HTTPSurface (e2e/dispatcher_test.go), pkg/dispatcher/http_test.go | |
| server-api.server-info | /api/server/info capability flags gate the SPA features | server-api | covered-deterministic | TestServerInfo (pkg/server/runs_uploads_test.go), TestServerInfo_SkillsEnabledLocal (pkg/server/local_skills_routes_test.go), TestMarketplace_ServerInfoFlag (pkg/server/marketplace_routes_test.go), TestMarketplace_ServerInfoFlagFalseWhenNil (pkg/server/marketplace_routes_test.go), pkg/server/backends_routes_test.go | each flag is asserted against the wiring that decides it — present when the store/feature is wired, false when it is nil |
| server-api.openapi | generated OpenAPI 3.1 spec matches the wired routes | server-api | covered-deterministic | pkg/server/openapi_test.go | |
| server-api.pipeline-board | /api/v1/pipeline-board/* control centre (columns, folding, actions, bulk ops) | server-api | covered-deterministic | TestPipelineBoardColumnBucketing (pkg/server/pipeline_boards_test.go), TestPipelineBoardFoldsDescendantsAndCollectsReviews (pkg/server/pipeline_boards_test.go), pkg/server/pipeline_board_actions_test.go | the REST layer behind studio-ui.pipelines |
| server-api.limits-cost | /api/v1/limits/cost + /override admin cost-cap surface | server-api | covered-deterministic | TestCostCapStatusDisabledByDefault (pkg/server/limits_test.go), TestCostCapStatusAndOverride (pkg/server/limits_test.go) | |
| server-api.backends-detect | /api/backends/detect credential auto-detection the toolbar reads | server-api | covered-deterministic | TestBackendsDetectRouteShape (pkg/server/backends_routes_test.go), TestBackendsDetect_NoSensitiveLeak (pkg/server/backends_routes_test.go) | the no-leak assertion is the one that matters: the endpoint reports availability, never a credential |
| server-api.effort-capabilities | /api/effort-capabilities per-backend/model effort lookup | server-api | covered-deterministic | TestEffortCapabilities_ClawOpus48 (pkg/server/effort_test.go), TestEffortCapabilities_UltracodeOnlyOnOpus48 (pkg/server/effort_test.go), TestEffortCapabilities_UnknownModel (pkg/server/effort_test.go), TestEffortCapabilities_CompilerParity (pkg/server/effort_test.go), TestGrokEffortCapabilities (pkg/server/grok_backend_test.go) | also covers the sibling /api/resolve-effort. The ultracode-only-on-4.8 case is the one that would catch a capability table drifting; the parity test catches the picker and the compiler's C177 dial set disagreeing |
| server-api.resolve-model | /api/resolve-model env-expands a node model: literal for the studio canvas | server-api | covered-deterministic | TestResolveModel_EnvSubstitutionUsesSetValue (pkg/server/effort_test.go), TestResolveModel_InvalidExpansionYieldsEmpty (pkg/server/effort_test.go), TestLooksLikeModelSpec (pkg/dsl/ir/validate_test.go) | the canvas used to replace ${VAR:-openai-codex/gpt-5.6-sol} with the word "env"; this endpoint returns the live spec (sol/terra/luna) and refuses expansions that don't look like a model so it cannot leak process env |
| server-api.editor-endpoints | /api/parse, /api/unparse, /api/validate DSL round-trip surface | server-api | covered-deterministic | TestUnparse_RoundTripIsSemanticallyStable (pkg/server/server_dsl_test.go), TestValidate_UnknownNodeReferenceIsC001 (pkg/server/server_dsl_test.go), TestParse_UnparseableSource (pkg/server/server_dsl_test.go) | round-trip fidelity (parse -> unparse -> parse is byte-identical) and the diagnostic CODE, not the message text |
| server-api.local-secrets | /api/local/secrets desktop/CLI secret surface | server-api | covered-deterministic | TestLocalSecrets_CreateListDeleteSealsAndNeverEchoesTheValue (pkg/server/local_secrets_routes_test.go), TestLocalSecrets_RenameOntoAnExistingNameIsRefused (pkg/server/local_secrets_routes_test.go) | the real routes on a New()-built local-mode server with the sealed store FILE as the oracle: create → the plaintext is absent from disk and from every response (only name/last4/fingerprint), list, re-post rotates in place without duplicating or widening the egress lock, delete removes it from both API and file; rename collision 409 leaves both records intact, unknown id 404. The previous citation (local_skills_routes_test.go) was a mis-citation — it holds no secrets coverage |
| server-api.run-shell | /api/ws/runs/{id}/shell post-mortem PTY shell in a preserved worktree | server-api | covered-deterministic | TestRunShell_Gates (pkg/server/runs_shell_test.go), TestShellEligible (pkg/server/runs_shell_test.go) | the gate (which runs may be shelled into) is the part that matters; the PTY itself is Unix-only with a Windows stub |
| cloud.oauth-connections | /api/me/oauth/* connect / authorize / refresh a subscription credential for cloud runs | cloud | covered-deterministic | TestOAuthConnections_SealsAndNeverEchoesTheCredential (pkg/server/oauth_routes_test.go) | the store is the oracle: the persisted payload must not contain the plaintext token, no response may echo it, and a second account is isolated from the first one's credential |
| server-api.static-spa | embedded studio SPA served from the binary | server-api | covered-deterministic | pkg/server/spa_test.go | |
| dispatcher.poll-dispatch | poll a tracker and dispatch one run per eligible issue | dispatcher | covered-deterministic | TestDispatcherE2E_DispatchAndRelease (e2e/dispatcher_test.go) | |
| dispatcher.retry | retry with backoff after a failed run, give up at max attempts | dispatcher | covered-deterministic | TestDispatcherE2E_RetryAfterFailure (e2e/dispatcher_test.go), TestDispatcherGivesUpAfterMaxAttempts (pkg/dispatcher/dispatcher_test.go) | |
| dispatcher.cancel | cancel an in-flight dispatched run | dispatcher | covered-deterministic | TestDispatcherE2E_CancelInFlight (e2e/dispatcher_test.go) | |
| dispatcher.state-transitions | issue state transitions around dispatch + revert on failure | dispatcher | covered-deterministic | TestDispatch_TransitionsToInProgress (pkg/dispatcher/loop_state_test.go), TestDispatcherE2E_RespectsTerminalStateChange (e2e/dispatcher_test.go) | |
| dispatcher.hooks | hook mechanics (script, env, timeout, failure, validation) + the before_remove hook FIRES | dispatcher | covered-deterministic | pkg/dispatcher/hooks_test.go, TestCleanupWorkspace_RunsBeforeRemoveBeforeDeletingDir (pkg/dispatcher/cleanup_workspace_test.go) | |
| dispatcher.hooks-lifecycle-firing | the after_create / before_run / after_run hooks actually FIRE at their lifecycle point | dispatcher | covered-deterministic | TestRunWorker_FiresLifecycleHooksInOrder (pkg/dispatcher/hooks_lifecycle_test.go), TestRunWorker_AfterRunFiresEvenWhenRunnerFails (pkg/dispatcher/hooks_lifecycle_test.go) | every phase appends to one shared order log, so the assertion is the ORDER of what actually fired — deleting any of the three Run() calls turns it red |
| dispatcher.per-ticket-bot | per-ticket bot override + assignee routing | dispatcher | covered-deterministic | TestBuildSpec_PerTicketBotSetsRouteKey (pkg/dispatcher/loop_bot_override_test.go) | |
| dispatcher.concurrency | per-state concurrency caps + claim conflict handling | dispatcher | covered-deterministic | TestDispatcherRespectsClaimConflict (pkg/dispatcher/dispatcher_test.go), TestDispatch_SlotCountedFromClaimTime (pkg/dispatcher/loop_setup_offload_test.go) | |
| dispatcher.cost-cap | daily cost cap gate blocks further dispatch | dispatcher | covered-deterministic | TestRefreshCostCapGatesWhenExceeded (pkg/dispatcher/cost_cap_test.go) | |
| dispatcher.workspace-cleanup | per-run workspace/worktree teardown policies | dispatcher | covered-deterministic | TestCleanupWorkspace_RemovesLinkedWorktreeRegistration (pkg/dispatcher/cleanup_workspace_test.go) | |
| dispatcher.pr-lookup-retry | board PR lookup outage, recovery, drain and terminal refusals | dispatcher | covered-deterministic | TestBoardDispatcher_PRLookupFailureRetriesAfterRecovery, TestBoardDispatcher_PRLookupDrainReturnsUnlaunchedCard, TestBoardDispatcher_PRLookupFailureHonoursAttemptCap, TestBoardDispatcher_PRLookupForkRemainsTerminal, TestBoardDispatcher_PRLookupForeignGrantRemainsTerminal (pkg/server/boarddispatch_pr_lookup_test.go) | real PR context resolution and dispatcher; durable retry ledger and no launch on refusal |
| dispatcher.native-tracker | native filesystem kanban tracker (board.json/issues/events) | dispatcher | covered-deterministic | pkg/dispatcher/native/store_test.go, TestNativeStore_Conformance (pkg/dispatcher/boardmongo/conformance_test.go) | |
| dispatcher.github-tracker | GitHub Issues tracker adapter | dispatcher | covered-deterministic | pkg/dispatcher/tracker/github_test.go | real GitHub API is stubbed at the HTTP boundary |
| dispatcher.forgejo-tracker | Forgejo/Gitea tracker adapter | dispatcher | covered-deterministic | pkg/dispatcher/tracker/forgejo_test.go | real Forgejo API is stubbed at the HTTP boundary |
| triggers.subscription-registry | subscription CRUD through the REST surface | triggers | covered-deterministic | pkg/server/triggers_routes_test.go | matching/mode semantics: pkg/trigger/subscription_test.go |
| triggers.subscription-query | subscription query by repo / by bot (tenant-scoped; by-repo also returns the workspace-wide ones) | triggers | covered-deterministic | TestSubscriptionStoreQueries (pkg/trigger/store_query_test.go) | |
| triggers.evaluator | evaluator matches an event to subscriptions and launches | triggers | covered-deterministic | pkg/trigger/evaluator_test.go | |
| triggers.board-source | board transition promotes a card / direct-launches a bot | triggers | covered-deterministic | pkg/trigger/board_integration_test.go, TestIssueTriageTrigger_E2E_ConsumeAndLaunch (e2e/issue_triage_trigger_test.go) | |
| triggers.consume-labels | consume_labels strips the matcher labels atomically pre-launch | triggers | covered-deterministic | pkg/trigger/consume_labels_test.go, TestIssueTriageTrigger_E2E_ConsumeAndLaunch (e2e/issue_triage_trigger_test.go) | |
| triggers.run-outcome | run completion events chain the next bot | triggers | covered-deterministic | TestEvaluatorRunCompletionChaining (pkg/trigger/evaluator_test.go), pkg/runview/trigger_emit_test.go, pkg/trigger/runoutcome_test.go | the chaining itself (a finished run's event matches a direct-mode subscription and launches the next bot), the emitter that publishes it from runview, and the envelope/episode-id builder shared with the runner |
| triggers.assistant-run-watch | durable target lifecycle → safe assistant diagnostic turns until Done or explicit Stop | triggers | covered-deterministic | pkg/runwatch/fs_test.go, pkg/server/assistant_run_watch_test.go, pkg/server/assistant_run_watch_autoarm_test.go | pins episode dedup/CAS lease recovery, persistence across recoverable assistant failures, repeated target outcomes, cancellations and old pending episodes, chat-only wake eligibility (ask_user and paused_operator excluded), host-selected assistant binding and host-event transcript semantics. A live LLM outcome→diagnostic smoke remains manual; Mongo/NATS use the same store/claim and outcome-bus contracts. The studio half (runChat rendering, the action-request offer) lives in studio/src/lib/runChat/messagesFromEvents.test.ts and studio/src/components/ChatDock/AssistantActionOffer.test.tsx — named here rather than cited, because neither has been read against THIS row promise (#2799) |
| triggers.scheduler | schedule-kind subscriptions tick on their cron | triggers | covered-deterministic | pkg/trigger/scheduler_test.go, pkg/trigger/scheduler_gate_test.go | |
| triggers.eventbus-inproc | in-process event bus delivery | triggers | covered-deterministic | pkg/eventbus/inproc_test.go | |
| triggers.eventbus-nats | NATS event bus on the separate ITERION_EVENTS stream | triggers | covered-deterministic | pkg/eventbus/nats_test.go | needs a NATS endpoint; skips cleanly without one |
| triggers.cloudsched | cloud recurring-bot scheduler with a multi-replica CAS ticker | triggers | covered-deterministic | pkg/cloudsched/cloudsched_test.go | includes assertMarkRunOutcomeContract running the LastRun* back-write property against both twins (#1426) |
| triggers.schedule-outcome | run terminal outcome back-written onto the schedule record via the eventbus | triggers | covered-deterministic | pkg/server/outcome_spine_schedule_test.go | success clears a previous failure, failure stamps error + code (a SANDBOX_DRIVER_UNAVAILABLE from #1425's runtime path lands here), a non-scheduled run does NOT mutate any schedule, a missing run is a silent no-op |
| triggers.retry-policy | usage_window retry policy resolution across all layers | triggers | covered-deterministic | pkg/retrypolicy/policy_test.go | |
| webhooks.gitlab | GitLab MR open/reopen + /revi note re-review launches | webhooks | covered-deterministic | pkg/server/webhooks_gitlab_test.go, pkg/webhooks/webhooks_test.go | |
| webhooks.github | GitHub PR events launch the configured bot | webhooks | covered-deterministic | pkg/server/webhooks_github_test.go | |
| webhooks.forgejo | Forgejo/Gitea PR events launch the configured bot | webhooks | covered-deterministic | pkg/server/webhooks_forgejo_test.go | |
| webhooks.generic | generic JSON inbound trigger | webhooks | covered-deterministic | TestGenericWebhook_HappyPath_VarsPrecedence (pkg/server/webhooks_generic_test.go), TestGenericWebhook_MissingBotIs400 (pkg/server/webhooks_generic_test.go), TestGenericWebhook_BodyHashIdempotency (pkg/server/webhooks_generic_test.go), pkg/webhooks/generic/generic_test.go, pkg/webhooks/router_test.go | the endpoint itself: var precedence on launch, 400 without a bot, body-hash idempotency, oversized-var refusal; the parser in pkg/webhooks/generic and the command routing in router_test.go |
| webhooks.rerequest-review | forge-native "Re-request review" on the bot reviewer relaunches the review (repeatable per click, bot-actor echo filtered, publish self-assigns the GitLab reviewer) | webhooks | covered-deterministic | TestGitLabWebhook_ReRequestReviewLaunches (pkg/server/webhooks_gitlab_test.go), TestGitHubWebhook_ReviewRequestedLaunches (pkg/server/webhooks_github_test.go), TestGitLabAddSelfAsPullReviewer (pkg/forge/gitlab/reviews_test.go), TestForgePublishReview_SelfAssignsReviewer (pkg/server/forge_publish_gate_test.go) | gate opt-out releasing forced review_on_sync: TestProvision_GateDisabledPin* (pkg/forge/orchestrator_botrules_test.go) |
| webhooks.auth | iwh_ token / HMAC admission, rate limits, idempotent delivery | webhooks | covered-deterministic | pkg/webhooks/match_test.go, pkg/server/webhooks_routes_test.go | |
| webhooks.authz-outage | a forge authorization outage is acknowledged, audited as launch_error, and launches no bot | webhooks | covered-deterministic | TestWebhookAuthzErrorsAreAcknowledgedWithoutLaunching (pkg/server/webhooks_authz_error_test.go) | Six real handler lanes; even an authorized=true result accompanied by an error refuses execution. |
| webhooks.handoff | produces/consumes hand-off stamps the next run's launch vars | webhooks | covered-deterministic | pkg/server/webhooks_handoff_test.go, bots/handoff_declarations_test.go | |
| webhooks.merge-gate | Revi posts a deterministic revi/review commit status | webhooks | covered-deterministic | TestForgePublishReview_GatePostedEvenWhenTheReviewFails (pkg/server/forge_publish_test.go), pkg/server/forge_gate_relaunch_test.go, pkg/server/forge_gate_autofix_test.go | pkg/forge/reviews_test.go is a PR-URL parser + markdown folder — orthogonal to the gate |
| forge.connections | forge connection / repo-integration / OAuth-app stores | forge | covered-deterministic | pkg/forge/oauth_app_store_test.go | |
| forge.admin-clients | per-provider admin clients (repos, hooks, permissions) | forge | covered-deterministic | pkg/forge/forgelive_test.go, pkg/forge/command_map_test.go | provider APIs stubbed at the HTTP boundary |
| forge.github-app | GitHub App manifest flow + installation-token minting | forge | covered-deterministic | pkg/server/forge_app_resolution_test.go | the interactive App-creation consent screen is excluded (needs a real GitHub org) |
| forge.bot-avatar | the iterion-bot avatar on a connection's account — auto on a GitLab bot identity at connect, explicit apply (force on an unflagged account), refusals on OAuth / GitHub, /brand/ public route | forge | covered-deterministic | pkg/server/forge_avatar_routes_test.go, pkg/server/brand_routes_test.go, TestSetAvatar_MultipartShape (pkg/forge/gitlab/client_test.go), TestSetAvatar_JSONBase64 (pkg/forge/forgejo/client_test.go), pkg/brand/brand_test.go | TestLiveGitLabAvatar (pkg/forge/forgelive_test.go, tag forgelive) proved on 2026-09-05 that a project-token bot user takes the avatar on GitLab 19.2; Forgejo's endpoint is swagger-verified, not live-tested |
| forge.orchestrator | provision/deprovision: webhook + secret + bindings + schedules | forge | covered-deterministic | pkg/forge/orchestrator_test.go | |
| forge.token-refresh | background token refresh worker | forge | covered-deterministic | pkg/forge/refresh_test.go | |
| forge.config-share | shared-config read/write under a synthetic share grant | forge | covered-deterministic | pkg/configshare/configshare_test.go | |
| cloud.auth-session | login / logout / refresh / session cookies + JWT claims | cloud | covered-deterministic | pkg/auth/service_test.go, pkg/server/auth_routes_test.go | |
| cloud.run-login-return | signed-out run links offer sign-in, then restore the full run URL | studio-ui | covered-deterministic | studio/src/tests/runSignIn.test.tsx, studio/src/auth/returnTo.test.ts, studio/src/views/auth/tests/ForcedPasswordChange.test.tsx | real App routes + AuthProvider + Login with HTTP and destination chrome stubbed; password retry, existing session, restricted account, query/hash, forced password rotation and unsafe return paths covered |
| cloud.sso-oidc | OIDC SSO providers + domain verification | cloud | covered-deterministic | pkg/auth/oidc/generic_test.go, pkg/server/org_sso_routes_test.go | a real IdP consent round-trip is excluded (see integrations.third-party-oauth) |
| cloud.password-reset | password reset request/confirm + mail delivery fallback | cloud | covered-deterministic | pkg/auth/password_reset_test.go, pkg/mail/mail_test.go | |
| cloud.orgs-teams | two-level tenancy: org membership, team scoping, context switch | cloud | covered-deterministic | pkg/identity/team_test.go, pkg/server/auth_teams_test.go | |
| cloud.invitations | team/org invitations: create, lookup, accept | cloud | covered-deterministic | TestInvitations_AdminIssuesAnonymousLookupAcceptGrantsMembership (pkg/server/invitations_e2e_test.go), TestInvitations_DeletingAPendingInviteKillsItsToken (pkg/server/invitations_e2e_test.go), TestInvitationFlow (pkg/auth/service_test.go) | the whole grant path over real HTTP: a plain member is refused (403, nothing created), an admin's token is looked up ANONYMOUSLY (the invitee has no session yet), accepting creates the membership — proven by the outsider's 403 on the team turning into a 200 that lists them at the invited role — and the token is single-use, login-gated, and dead once the admin deletes the invitation. The previous citation (auth_views_test.go) holds only an org-tree test |
| cloud.pat | personal access tokens (iap_ bearers) | cloud | covered-deterministic | TestPATBearer_AuthenticatesProgrammaticClientsThroughTheRealServer (pkg/server/pat_bearer_e2e_test.go), TestPATBearer_DiesWithTheMembershipItResolvesThrough (pkg/server/pat_bearer_e2e_test.go), pkg/server/pat_routes_test.go, pkg/pat/pat_test.go | the promise is programmatic access, and it lives in the auth MIDDLEWARE: the token is minted over real HTTP and then presented as a bearer to a server built by the real New() wiring — identity + pinned team served back, a protected endpoint reachable, cross-team read 403, unknown iap_ failing closed (never falling through to JWT parsing), revoke and membership removal each turning the next call into a 401 while the JWT session survives. Deleting the iap_ branch from requireAuth leaves the store and handler tests green and turns these red |
| cloud.audit-log | tenant + platform audit log of control-plane mutations | cloud | covered-deterministic | TestAuditEndToEnd (pkg/server/audit_routes_test.go), pkg/audit/audit_test.go | a control-plane mutation through the routes leaves the row an operator reads back on /api/teams/{id}/audit; the store half (TTL, filters) in pkg/audit |
| cloud.quotas | per-org monthly run/cost counters gate the launch | cloud | covered-deterministic | pkg/orgusage/orgusage_test.go, pkg/server/launch_gate_test.go | |
| cloud.usage-cap-runtime | usage-cap percentages retunable at runtime via the super-admin API: env stays the default, a DB record overrides it on BOTH enforcement points within the resolver TTL, /healthz echoes the effective value, non-admin and invalid input rejected, update audited | cloud | covered-deterministic | TestAdminUsageCaps_EnvFallback (pkg/server/admin_settings_routes_test.go), TestAdminUsageCaps_UpdateOverridesEnvAndAudits (pkg/server/admin_settings_routes_test.go), TestAdminUsageCaps_NonAdminRejected (pkg/server/admin_settings_routes_test.go), TestAdminUsageCaps_InvalidValuesRejected (pkg/server/admin_settings_routes_test.go), TestUsageCapPreflight_RuntimeSettingsChangeIsLive (pkg/runner/usage_cap_test.go), TestResolver_UpdatePropagatesWithinTTL (pkg/usagecap/settings_test.go) | the server half runs over real HTTP through the production auth middleware; the runner half proves the tightened cap refuses the next claim on the SAME Runner value (no restart); the resolver half pins the TTL as the propagation bound with a fake clock (ADR-090) |
| cloud.adaptive-routing-policy | the adaptive-routing policy (ADR-121, pkg/llmroute) resolves retrypolicy-style per field (first setter wins), a lock at a level vetoes the setters ABOVE it and the field is decided at-or-below the lock (provenance <level>_lock when it pins the default), strict is monotone without a lock and the launcher's explicit lock reopens it, the platform ceiling prunes triggers only (an emptied set is an ANSWER — "never switch" — that survives re-Normalize and the snapshot's non-omitempty round-trip); the platform level rides the platform_credentials record (routing block: pair_order wire-spelled harness+credential against a closed vocabulary — forfaits and <provider>_key, no pool until the pool door exists — triggers subset of usage_window/unavailable/transient_exhausted/auth with budget/schema/any/unclassified refused BY RULE, refused_pinned_key forfait / park default forfait, strict), validated at write (API PUT replaces wholesale, null clears; env dials ITERION_PLATFORM_PAIR_ORDER/_TRIGGERS/_REFUSED_PINNED_KEY/_ROUTING_STRICT refused at boot when unreadable) and read back with the effective answer and provenance; every launch snapshots the resolved policy on the run doc (llm_route_policy + sources); refused_pinned_key=forfait (the arbitrated default) keeps a shared key refused by its cap OUT of a wire family another credential holds — the routes naming its provider spend the holder (audit line names it) — park restores it pinned-only, facades (zai/moonshot) are out of the knob's scope and always come back, and the preview predicts the same answer in both directions | cloud | covered-deterministic | TestResolve_FirstSetWinsPerField, TestResolve_LocksStopTheDescent, TestResolve_LockWithoutValuePinsTheDefault, TestResolve_LockPinningDefaultSaysSo, TestResolve_LowestLockGoverns, TestResolve_StrictIsMonotoneWithoutLock, TestResolve_StrictLockReopensForTheLauncher, TestClamp_PrunesOnlyTriggers, TestClamp_EmptyAnswerSurvivesReNormalize, TestValidate_PairOrder, TestValidate_TriggersClosedVocabulary, TestValidate_ExplicitEmptyTriggersRefused (pkg/llmroute/policy_test.go), TestPlatformCredentials_RoutingValidate, TestValidateEnv_RoutingDials (pkg/platformcfg/platformcfg_routing_test.go), TestBackendNamesAreNameableInTheRoutingPolicy (pkg/dsl/ir/validate_fallbacks_llmroute_test.go), TestKeyProviderSlotsMirrorSecretsProviders (pkg/llmroute/policy_test.go), TestAdminPlatformCredentials_RoutingReplacesAndClears, TestAdminPlatformCredentials_RoutingEffectiveDefaults, TestResolveRunLLMRoutePolicy_PlatformRecordWinsAndNamesItself, TestResolveRunLLMRoutePolicy_EnvDialAnswersWithItsOwnProvenance, TestResolveRunLLMRoutePolicy_HigherLayerOutranksThePlatform, TestRunLLMRoutePolicy_EmptyTriggersSurviveJSON (pkg/server/route_resolve_test.go), TestRefusedPinnedKey_DefaultForfaitKeepsTheKeyOut, TestRefusedPinnedKey_ParkRestoresPinnedOnly, TestRefusedPinnedKey_EnvDialPark, TestRefusedPinnedKey_FacadeCarveOutAlwaysRestores, TestCredentialPreviewMatchesSealedBundleUnderRefusedPinnedKey (pkg/server/cloudpublisher/refused_pinned_key_test.go) | bot (routing: manifest block), binding and run levels join the fold in the next slices; the bot/binding/run pair selection, the mid-run switch and the pool door are slices 3–5 |
| cloud.usage-cap-trust-window | a stored usage reading is trusted for a bounded time (ITERION_USAGE_CAP_TRUST_WINDOW, default 3h) after it was observed, not until its reset instant: a pre-reset reading cannot lock a credential out through a reset the provider made early, a recent one still parks, and a malformed window refuses to start | cloud | covered-deterministic | TestReadingFresh_TrustWindowBoundsADatedReading (pkg/usagecap/usagecap_test.go), TestPreflight_StaleDatedReadingDoesNotBlock (pkg/usagecap/usagecap_test.go), TestTrustFromEnv (pkg/usagecap/usagecap_test.go), TestUsageCapPreflight_StaleReadingDoesNotRefuseAdmission (pkg/runner/usage_cap_trust_test.go) | the runner half replays the production shape of #690 (0.99 on seven_day, reset four days out, observed 17h37m earlier) at the admission gate that refused every claude_code run; a run refused there never fires the hook that would refresh the reading, which is why the bound is on AGE |
| cloud.usage-readings-clear | a super-admin forgets ONE credential's stored usage readings by fingerprint (DELETE /api/admin/usage-readings/{fingerprint}, iterion remote admin usage-readings clear): every key the credential was metered under, nothing else, audited with the count, idempotent, non-admin refused | cloud | covered-deterministic | TestAdminUsageReadings_ClearForgetsOneCredential (pkg/server/admin_usage_readings_routes_test.go), TestAdminUsageReadings_NonAdminRejected (pkg/server/admin_usage_readings_routes_test.go), TestMemStore_Conformance (pkg/usagecap/store_conformance_test.go), TestMongoStore_Conformance (pkg/usagecap/store_conformance_test.go) | over real HTTP through the production auth middleware; the store half is one conformance suite run against both twins (the Mongo one under the mongo-conformance job) |
| cloud.usage-retry-pull-forward-probe | the pull-forward probe observes a window an operator reopened by hand: the pass moves a not-yet-due usage_window retry to now under the deployment's own bounds (ITERION_RETRY_PROBE_FLOOR, ITERION_RETRY_PROBE_BUDGET, 0 disarms it, batch 2 a tick), the pull is a CAS that spends no attempt and holds the announced instant aside, the probe's re-failure re-arms free of charge while it follows its pull inside store.ProbeMarkerTTL (past that hour the resume is metered and the park is charged, max_attempts of each per run), a resume that never reached the runner stands the probe down on the instant the provider named, and iterion_runs_retry_probes_total{result} says which of the three happened so a wasted pod is tellable from a useful one; the BUDGET that bounds the probes belongs to the WALL (so one provider window costs budget probes beside at most max_attempts charged parks) while the PACE belongs to the ROW and no arming clears it (one pod per floor per run, whatever the provider does to its dates), and a row armed before any of this existed is probed on whatever budget its own ledger has left and carries its wall from its first arming — without being handed a second one | cloud | covered-deterministic | TestSweepRetryProbes_PullsANotYetDueRetryForwardToNow (pkg/server/retry_probe_test.go), TestSweepRetryProbes_OperatorBoundsReachTheScan (pkg/server/retry_probe_test.go), TestSweepRetryProbes_ALostPullIsNotCountedAsAProbe (pkg/server/retry_probe_test.go), TestRetryProbes_TheCounterSaysWhatTheProbeFound (pkg/server/retry_probe_result_test.go), TestArmUsageWindowRetry_AReFailureAfterAProbeSpendsNoAttempt (pkg/runner/usage_retry_probe_test.go), TestArmUsageWindowRetry_AStaleMarkerParkIsChargedAndKeepsTheWallsProbeLedger (pkg/runner/usage_retry_probe_test.go), TestArmUsageWindowRetry_AWindowDurationWindowIsOneWall (pkg/runner/usage_retry_wall_identity_test.go), TestArmUsageWindowRetry_TheCapturedFacadeWindowIsOneWall (pkg/runner/usage_retry_wall_identity_test.go), TestArmUsageWindowRetry_AProviderThatMovesItsWallIsStillPacedByTheFloor (pkg/runner/usage_retry_wall_identity_test.go), TestArmUsageWindowRetry_ALegacyRowGainsItsWallWithoutASecondBudget (pkg/runner/usage_retry_wall_identity_test.go), TestArmUsageWindowRetry_EveryParkThatFollowedAPullReportsWhatItFound (pkg/runner/usage_retry_wall_identity_test.go), TestArmUsageWindowRetry_ARefusedParkFeedsTheSharedCircuitOnceItLands (pkg/runner/usage_retry_wall_identity_test.go), TestStandDownProbedRetry_AMarkerFromAFinishedWaitLeavesTheSettlement (pkg/server/retry_probe_test.go), TestScheduleRunRetry_TheProbeLedgerBelongsToTheWall (internal/retryprobetest/wall_test.go), TestProbeScan_ALegacyRowIsProbedAndThenCarriesItsWall (internal/retryprobetest/wall_test.go), TestRunRetryProbe_TheProbeLedgerBelongsToTheWall (pkg/store/mongo/runs_retry_probe_test.go), TestProbeGuarantee_TheRunnersArmingKeepsItOverRandomSequences (pkg/runner/usage_retry_property_test.go), TestProbeGuarantee_TheSweepersPassesKeepItOverRandomSequences (pkg/server/retry_probe_property_test.go), TestRunRetryProbeModel_AnswersWhatTheStoreAnswers (pkg/store/mongo/runs_retry_model_conformance_test.go) | the two property drivers assert the seven invariants of internal/retryprobetest after EVERY step of seeded random sequences over the real arming and the real passes, with a coverage assertion that the sequences reached each state — including a park whose refusal names a window DURATION, the shape no per-wall invariant can see, and the two invariants that are not per wall (the pace, and only the provider moving a wall); the Mongo row of the pairing runs under the mongo-conformance job (ITERION_TEST_MONGO_URI) and the in-memory twin everywhere, which is what makes a model-based verdict worth anything |
| cloud.usage-park-wall | the unit of a usage park is the provider WALL, persisted on the retry row (RunRetryState.Wall) by the park that decided it and read by every reader instead of being re-derived: the run owner's push notification gets its own kind (usage_window_parked, "paused on a usage window, resumes at T") with the wall folded into the episode id, the operator's incident keys on the same wall, the pull request's pause notice speaks for the park that OPENED the wall and for a wall the provider MOVED but not for a park that re-stated it (its evidence is the probe ledger, so a wall no probe ever pulled is announced by each charged park on it, at most max_attempts), and a run whose retry budget is spent has no wall and notifies as the failure it has become; only the provider's own DATED word moves a wall — a refusal naming a window duration resolves against the moment it is read and says nothing new, and so does an absolute instant a producer rendered into a sentence that also states the duration (the zai facade's capture), which the parser answers from the pair of facts rather than from a list of shapes — the message names the instant the row is ARMED for and not the wall (they differ by days when a skipped credential reopens sooner), and a row from before the field keeps the per-transition episode until its first arming names its wall, which clears no budget; the run-completion callback keys on the wall too (#2707): a park is a wait and delivers nothing, the settled run delivers once — the sweeper's abandonment fires what the executing lanes never will | cloud | covered-deterministic | TestUsagePark_OneNotificationPerProviderWall (pkg/usernotify/usage_park_test.go), TestUsagePark_TheSweepPreCheckDerivesTheSameEpisode (pkg/usernotify/usage_park_test.go), TestUsagePark_TheBodyNamesTheInstantTheRunComesBack (pkg/usernotify/usage_park_test.go), TestNoticeGate_TheWallThatWasProbedIsNotAnnouncedTwice (pkg/server/retry_abandon_cas_test.go), TestNoticeGate_ALegacyRowKeepsTheLedgerItShippedWith (pkg/server/retry_abandon_cas_test.go), TestOpsSweep_AUsageParkCostsNoRunLoadOnceItsEpisodeIsSettled (pkg/alert/opsdispatch_usage_park_test.go), TestUsageParkWall_OnlyTheProvidersOwnDateIsNewsAboutTheWall (pkg/runner/usage_retry_wall_identity_test.go), TestArmUsageWindowRetry_TheCapturedFacadeWindowIsOneWall (pkg/runner/usage_retry_wall_identity_test.go), TestArmUsageWindowRetry_OneDatedWindowIsOneWallWhateverCarriedIt (pkg/runner/usage_retry_wall_identity_test.go), TestParseResetHintDated (pkg/backend/delegate/usage_limit_test.go), TestProbeGuarantee_TheRunnersArmingKeepsItOverRandomSequences (pkg/runner/usage_retry_property_test.go), TestProbeGuarantee_TheSweepersPassesKeepItOverRandomSequences (pkg/server/retry_probe_property_test.go), TestFireForRun_UsageParkWaitsForSettlement (pkg/notify/completion_test.go), TestFireForRun_LegacyParkedRowKeepsThePerTransitionBehaviour (pkg/notify/completion_test.go), TestAbandonRetry_SettlementDeliversTheCompletionCallback (pkg/server/retry_abandon_cas_test.go), TestAbandonRetry_ALostRaceDeliversNothing (pkg/server/retry_abandon_cas_test.go) | the property driver runs the REAL usernotify.Dispatcher with its real episode claim over the modelled row and asserts at most one owner notification per wall — and not none — over seeded sequences of parks, probe pulls, free re-arms, settlements and wall changes; measured before it: about twenty-five "Run failed" pushes a day per run parked on a weekly wall |
| cloud.usage-retry-settlement-cas | the settlement of a usage-window retry is a conditional write on the armed instant the caller read, so a sweep tick that pulled the row forward to probe it (or re-armed it) is not overwritten: a lost race reports it, counts nothing as abandoned and does NOT republish the run's outcome, so no consumer that stood down on the armed retry treats a run still waiting on its wall as dead; the escape hatch is the zero instant, for a cause about the run's own future (its pull request closed or re-queued) | cloud | covered-deterministic | TestAbandonRunRetry_RefusesARowThatMovedUnderIt (internal/retryprobetest/wall_test.go), TestAbandonRetry_CarriesTheInstantTheScanListed (pkg/server/retry_abandon_cas_test.go), TestAbandonRetry_ALostRaceLeavesTheRowAlone (pkg/server/retry_abandon_cas_test.go), TestRunRetryProbeModel_AnswersWhatTheStoreAnswers (pkg/store/mongo/runs_retry_model_conformance_test.go) | the store half is the twin, paired operation by operation against Mongo under the mongo-conformance job; the server half pins the wiring (the sweeper carries the instant its own scan listed) and the irreversible half (no outcome replay on a lost race) |
| cloud.usage-retry-skipped-credential | the usage-window retry arms on the EARLIER of the failed credential's reset and the reopening of the earliest credential the launch's walk passed over (run.skipped_cred_reopens_at, reset_source: skipped_credential), re-stamped on every resume — EXCEPT on the last attempt the budget allows, which is reserved for the failed credential's own reset when that reset is still ahead and inside max_wait (reset_source: <evidence>+last_attempt_pinned), and EXCEPT while the row holds the instant a pull-forward probe set aside, where no other credential's reopening is weighed on either arming (the write that took the earlier wake would consume the only durable copy of the provider's own word) | cloud | covered-deterministic | TestUsageWindowRetryAt_ArmsOnTheEarliestReopeningCredential (pkg/runner/usage_retry_skipped_test.go), TestArmUsageWindowRetry_ReadsTheSkippedCredentialFromTheRun (pkg/runner/usage_retry_skipped_test.go), TestArmUsageWindowRetry_AFreeReArmKeepsTheAnnouncedWallOverASkippedCredential (pkg/runner/usage_retry_probe_test.go), TestArmUsageWindowRetry_ASpentProbeBudgetCostsTheSkippedCredentialTheRestOfTheWait (pkg/runner/usage_retry_probe_test.go), TestUsageWindowRetryAt_BudgetOutlastsTheAuthoritativeWall (pkg/runner/usage_retry_budget_test.go), TestUsageWindowRetryAt_SpeculativeWakeSurvivesWhileTheBudgetCanSpareIt (pkg/runner/usage_retry_budget_test.go), TestArmUsageWindowRetry_ReadsTheSpentAttemptsFromTheRun (pkg/runner/usage_retry_budget_test.go), TestRunRetry_SpentAttemptsAreReadableFromTheRunDocument (pkg/store/mongo/runs_retry_test.go), TestResolve_ReportsWhenTheSkippedForfaitReopens (pkg/server/cloudpublisher/skipped_reopens_test.go), TestApiKeyUsable_ReportsTheEarliestReopening (pkg/server/cloudpublisher/skipped_reopens_test.go) | the production shape of #684: a team key refused on its five-hour window (reopens 16:40Z) and a platform forfait walled until Monday — armed at 16:41Z, not Monday; #922 is the budget half — both clocks spend one max_attempts purse, so five wakes on a five-hour cycle used to retire a run six days before a seven-day wall fell; the store field rides the CredFingerprintMeter conformance row |
| cloud.credential-ceiling-counting | a key's max_concurrent_runs counts alive runs whose RESOLVED routes can spend it (the stamp is narrowed through model.EffectiveProviders), model-idle or not: run.llm_idle_since (toggled by the runner at model-node boundaries, cleared by every re-stamp) is observability, because a slot lent out during tool-only work has no path back when the next model node starts | cloud | covered-deterministic | TestResolve_StampsOnlyTheSpendableFingerprints (pkg/server/cloudpublisher/stamp_narrowing_test.go), TestCredSlotTracker_IdleBetweenModelNodes (pkg/runner/cred_slot_test.go), TestCredSlotObserver_WiresWorkflowAndStore (pkg/runner/cred_slot_test.go), TestConformance_FilesystemShared (pkg/store/storetest_run_test.go) | the two production shapes of #661: a rite pinned to the forfait holding a facade-key slot (next launch died AUTH_FAILED), and a finished agent node sitting in a sixty-minute tool-only gate — which, under the hard cap of #2643, keeps its slot and parks the next launch on that key until it ends; the count filter is the CredFingerprintMeter row of the shared store conformance, run on both twins |
| cloud.credential-observability | the run API exposes the credential stamp (cred_fingerprints, llm_idle_since, skipped_cred_reopens_at), the key views report alive_runs from the SAME count the ceiling asks, and last_used_at is bumped under the run's tenant for a tenant's own key and across tenants for a platform/pool key | cloud | covered-deterministic | TestHeaderFromRun_ExposesTheCredentialStamp (pkg/runview/snapshot_cred_test.go), TestListApiKeys_ReportsAliveRuns (pkg/server/byok_alive_runs_test.go), TestMarkCredFingerprintsUsed_ScopesByTier (pkg/runner/loop_spend_scope_test.go), TestMemoryApiKey_MarkFingerprintUsed_ScopesToTheContextTenant (pkg/secrets/byok_fp_scope_test.go), TestMongoApiKey_MarkFingerprintUsed_ScopesToTheContextTenant (pkg/secrets/byok_mongo_fp_scope_test.go) | #659 pt 3; the Mongo twin of the scoping runs under the mongo-conformance job |
| cloud.shared-tier-forfait-first | on a shared tier (org or platform) holding a forfait AND an API key on one wire family, the forfait takes the family and the key funds only the routes that name its provider — on both wires (Claude forfait vs a z.ai key, ChatGPT forfait vs an openai key); a facade key (z.ai, Moonshot) is the anthropic wire's default only as facade_default allows (auto: in a tier holding no Claude forfait or anthropic key; never; always), so under auto a closed Claude forfait is restored and the run parks on it, while a same-vendor key takes the wire it leaves free; a failed forfait read still lets the keys fill; keys_first puts keys back in front on both tiers; keys_first and facade_default are validated, stored, cleared back to their env defaults ("", or null for keys_first) (ITERION_PLATFORM_KEYS_FIRST, ITERION_PLATFORM_FACADE_DEFAULT, refused at boot when unreadable) and read back with their effective values; a key the forfait shadows stamps no reopening; the pool is not asked for a provider a key sealed for the run's routes already funds; the preview lists the same selection, route-only keys included, and never predicts a shared key for a provider an earlier stage holds | cloud | covered-deterministic | TestPlatformTier_forfaitHoldsTheWireOverAMeteredKey (pkg/server/cloudpublisher/forfait_first_test.go), TestOrgTier_forfaitHoldsTheWireOverAMeteredKey (pkg/server/cloudpublisher/forfait_first_test.go), TestSharedTier_codexForfaitHoldsTheOpenAIWire (pkg/server/cloudpublisher/forfait_first_test.go), TestSharedTier_facadePolicyTruthTable (pkg/server/cloudpublisher/forfait_first_test.go), TestPlatformTier_forfaitStoreFailureStillFillsTheKeys (pkg/server/cloudpublisher/forfait_first_test.go), TestOrgTier_forfaitStoreFailureStillFillsTheKeys (pkg/server/cloudpublisher/forfait_first_test.go), TestPlatformTier_bothClosedRestoresInTheFillOrder (pkg/server/cloudpublisher/forfait_first_test.go), TestPlatformTier_keysFirstPutsTheSameVendorKeyOnTheWire (pkg/server/cloudpublisher/forfait_first_test.go), TestSharedTier_aShadowedKeyStampsNoReopening (pkg/server/cloudpublisher/forfait_first_test.go), TestAdminPlatformCredentials_KeysFirstMergesAndReadsBack (pkg/server/platform_settings_test.go), TestAdminPlatformCredentials_FacadeDefaultIsValidatedAndClearsToTheEnv (pkg/server/platform_settings_test.go), TestValidateEnv_RefusesWhatTheKnobsCannotRead (pkg/platformcfg/platformcfg_test.go), TestCredentialPreviewMatchesSealedBundleAcrossTiers (pkg/server/cloudpublisher/credential_preview_test.go), TestCredentialPreviewMatchesSealedBundleUnderTheFacadePolicy (pkg/server/cloudpublisher/credential_preview_test.go), TestCredentialPreviewAgreesWhenAProviderIsAlreadyHeld (pkg/server/cloudpublisher/credential_preview_test.go), TestPoolTier_isNotAskedForAProviderARouteKeyFunds (pkg/server/cloudpublisher/credpool_apikey_grant_test.go) | the order is per tier: an org credential still holds the wire over the platform's (the org tier fills first); a run in flight when its forfait closes waits for the forfait's reset (no per-family reopening instant); a tier whose store cannot say whether it holds a Claude credential reads as holding none (said on the server log) |
| cloud.shared-tier-restore-order | when every tier refused its credentials, the restore hands back, per wire family, the EARLIEST tier's credential — the tenant's own before the org's, the org's before the platform's — in that tier's fill order (the tenant's keys before its forfaits, a shared tier's forfaits before its keys unless keys_first); a refused shared key comes back in the channel its fill would have sealed it in (the facade policy holds there; beside the credential holding its family, only for the routes that name its provider), never over a key another tier already sealed for those routes; the tenant's own refused key never displaces a key a shared tier sealed for its provider's routes — it waits for every tier and comes back only into a family none refilled and only when some route may spend that key's slot as the run's default (model.ProviderResolution.AnthropicWireDefaultReads), then as the default over that key, marks included; the preview agrees | cloud | covered-deterministic | TestRestore_TheTenantsOwnCredentialComesBackFirst (pkg/server/cloudpublisher/forfait_first_test.go), TestRestore_theOrgsCredentialComesBackBeforeThePlatforms (pkg/server/cloudpublisher/forfait_first_test.go), TestRestore_theFacadePolicyHoldsThere (pkg/server/cloudpublisher/forfait_first_test.go), TestRestore_aRouteKeyAnotherTierFundedIsNotReplaced (pkg/server/cloudpublisher/forfait_first_test.go), TestRestore_aSharedRouteKeyKeepsItsRoutesOverTheTenantsRefusedKey (pkg/server/cloudpublisher/forfait_first_test.go), TestRestore_theTenantsOwnKeyIsTheLastParkPointOfAnEmptyWire (pkg/server/cloudpublisher/forfait_first_test.go), TestRestore_aRunWhoseRoutesAllNameTheProviderKeepsTheSharedKey (pkg/server/cloudpublisher/forfait_first_test.go), TestRestore_aReaderOfAnotherSlotDoesNotBringTheTenantsKeyBack (pkg/server/cloudpublisher/forfait_first_test.go), TestEffectiveProvidersAnthropicWireDefaultReads (pkg/backend/model/forfait_first_walk_test.go), TestPlatformTier_bothClosedRestoresInTheFillOrder (pkg/server/cloudpublisher/forfait_first_test.go), TestCredentialPreviewMatchesSealedBundleAcrossTiers (pkg/server/cloudpublisher/credential_preview_test.go), TestCredentialPreviewMatchesSealedBundleUnderTheFacadePolicy (pkg/server/cloudpublisher/credential_preview_test.go) | the refused-credential memory keeps the first tier's credential PER SLOT, so tier order shows across two slots of one family (an org forfait against a platform key) |
| cloud.shared-tier-pin-beside-forfait | a route pinning the provider a sealed forfait answers (anthropic beside a Claude forfait, openai beside a ChatGPT one) gets the shared key beside it, pinned-only — claw and pi cannot spend a Claude forfait on its plan — while the forfait keeps the family, and claude_code under an anthropic hint spends the run's own default key, then the forfait, then the pinned key; a pin on another provider of the wire (zai beside a Claude forfait) is funded the same way; the preview agrees | cloud | covered-deterministic | TestSharedTier_aPinOnTheForfaitsOwnProviderIsFundedPinnedOnly (pkg/server/cloudpublisher/forfait_first_test.go), TestPlatformTier_pinnedFacadeRouteKeepsItsKeyBesideTheForfait (pkg/server/cloudpublisher/forfait_first_test.go), TestCredentialPreviewMatchesSealedBundleOnAPinBesideAForfait (pkg/server/cloudpublisher/credential_preview_test.go), TestAnthropicHint_PrefersTheForfaitOverAPinnedSharedKey (pkg/backend/delegate/claude_code_cred_pinned_test.go) | |
| cloud.forfait-first-consumers | a key sealed for routes only is stamped (takes a slot of its concurrency ceiling) only when some route may spend it: a route whose backend declares it spends the run's forfait of that provider first (delegate.RegisterForfaitFirst — claude_code for anthropic, codex for openai), beside a sealed forfait of that provider, never reaches it; the walk's answer is a POSITIVE list — a route it cannot vouch for counts as spending the key: a backend read from the environment, a supervisor, claw (whose openai order depends on runner settings the server cannot see); claw spends, for openai/…, the run's default key, then the ChatGPT forfait while the runner lets it serve, then the pinned key — the same credential in process and in the sandbox for every combination of keys, forfait, ITERION_OPENAI_USE_OAUTH and OPENAI_BASE_URL — and the spend ledger books it where it went | cloud | covered-deterministic | TestStamp_aPinnedKeyNoRouteSpendsIsNotCounted (pkg/server/cloudpublisher/forfait_first_test.go), TestEffectiveProvidersForfaitFirstIsPositive (pkg/backend/model/forfait_first_walk_test.go), TestClawDeclaresNoForfaitFirst (pkg/backend/model/claw_openai_forfait_first_test.go), TestClawOpenAICredentialParity (pkg/backend/model/claw_openai_forfait_first_test.go), TestCredentialSlotForRoute_AClawOpenAIRouteIsBookedWhereClawSpent (pkg/runner/usage_cap_moonshot_test.go), TestResolveWithContext_OpenAIForfaitBeforeAPinnedKey (pkg/backend/model/claw_openai_forfait_first_test.go), TestForwardableProviderEnv_OpenAIForfaitBeforeAPinnedKey (pkg/backend/model/claw_openai_forfait_first_test.go), TestAnthropicHint_PrefersTheForfaitOverAPinnedSharedKey (pkg/backend/delegate/claude_code_cred_pinned_test.go) | the claw witnesses resolve through the runner's own lookups (model.RunCredentialsLookup, RunOAuthDirLookup); a real ChatGPT-forfait call is not re-proven here |
| cloud.usage-cap-route-preflight | the cloud runner's usage-cap pre-flight judges each primary anthropic-wire route on the credential IT spends (claude_code through the delegate's own composition under the executor's hint, claw/pi as the spend ledger books them, kimi and grok not at all, opencode on the pod's ambient meter; an unreadable route on the run's default; a route the delegate refuses before spawning not at all), parks on a HARD cap only when every execution from the entry to a terminal crosses a capped route (a fan-out's branches all run) and on a SOFT cap — or a hard one on any backend but claude_code, whose sessions alone report the readings the mid-run guard reads — as soon as the run may reach it (nothing stops it in flight), arms the retry for the earliest reopening after which the run could start, fails open per credential, and decides the same whatever order the routes were read in | cloud | covered-deterministic | TestUsageCapPreflight_AClosedForfaitDoesNotParkRoutesOnAPinnedKey (pkg/runner/usage_cap_routes_test.go), TestUsageCapPreflight_AWalledPinnedKeyParksTheRoutesThatSpendIt (pkg/runner/usage_cap_routes_test.go), TestUsageCapPreflight_ParksOnlyWhenEveryPathCrossesACappedRoute (pkg/runner/usage_cap_routes_test.go), TestUsageCapPreflight_ParksUntilAPathReopens (pkg/runner/usage_cap_routes_test.go), TestUsageCapPreflight_UnreadableAndRefusedRoutes (pkg/runner/usage_cap_routes_test.go), TestUsageCapPreflight_FailsOpenPerCredential (pkg/runner/usage_cap_routes_test.go), TestParkDecision_IsDeterministic (pkg/runner/usage_cap_routes_test.go), TestAnthropicWireRoutesReadsThePrimaryElement (pkg/backend/model/wire_reach_test.go), TestAlwaysReachesWalls (pkg/dsl/ir/uses_llm_test.go), TestUsageCapPreflight_FanOutAndSoftCapsPark (pkg/runner/usage_cap_routes_test.go), TestAlwaysReachesTakesEveryFanOutBranch (pkg/dsl/ir/uses_llm_test.go), TestCanReach (pkg/dsl/ir/uses_llm_test.go), TestUsageCapPreflight_AHardCapNoGuardObservesParksWhenReachable (pkg/runner/usage_cap_routes_test.go), TestUsageCapPreflight_APiRouteIsJudgedOnTheKeyItSpends (pkg/runner/usage_cap_routes_test.go), TestUsageCapPreflight_OwnConfigBackendsAreNotJudgedOnTheRunsCredentials (pkg/runner/usage_cap_routes_test.go), TestUsageCapPreflight_ACappedSupervisorParksTheRun (pkg/runner/usage_cap_routes_test.go) | a route is judged on its FIRST chain element (what the executor spends before a rescue); a supervisor is a route on every execution that nothing stops in flight — its model's credential capped, soft or hard, parks the run |
| cloud.usage-cap-launch-vars | the usage-cap pre-flight walks the run's OWN topology under its launch vars: a capped route behind a branch those vars made false is not a route the run may reach, and neither is the unconditional fallback a holding guard pre-empts; everything the vars cannot decide keeps the conservative whole-graph answer (a predicate on a node's output, the Condition field form, every branch of a fan-out, the exit beside a loop or foreach back-edge, a round-robin target no guarded sibling pre-empts, a bounded iteration still unspent); a hard cap the vars make unavoidable parks in advance, a resume parks until the SAME instant as its launch, and the park reason counts only the routes on the run's own paths; a redelivered launch prunes on the vars it carries and a resume folds the run's stored inputs; and a prune leaving no terminal, or emptying a node's exits, is discarded instead of read as a wall | cloud | covered-deterministic | TestUsageCapPreflight_LaunchVarsDecideWhichBranchesTheRunCanReach, TestUsageCapPreflight_AnUndecidablePredicateKeepsTheBranch, TestUsageCapPreflight_AFanOutBranchStaysReachableWhateverItsPredicate, TestUsageCapPreflight_AHardCapTheLaunchVarsMakeUnavoidableParks, TestUsageCapPreflight_ALoopIterationDoesNotExcludeItsExit, TestUsageCapPreflight_AForeachIterationDoesNotExcludeItsExit, TestUsageCapPreflight_ARoundRobinTargetIsNotPreEmptedByAGuard, TestUsageCapPreflight_ARedeliveredLaunchPrunesOnTheVarsItCarries, TestUsageCapPreflight_AResumeFoldsTheRunsStoredInputs, TestUsageCapPreflight_AResumeParksForTheSameInstantAsItsLaunch, TestUsageCapPreflight_AVacuousPruneDoesNotPark, TestUsageCapPreflight_ANodeWithNoTransitionLeftDoesNotPark, TestUsageCapPreflight_ASoftCapBehindAStoppedNodeDoesNotPark, TestUsageCapPreflight_TheParkReasonCountsOnlyTheRoutesOnTheRunsPaths (pkg/runner/usage_cap_launch_vars_test.go), TestUsageCapPreflight_SpentIterationKeepsThePrune (pkg/runner/usage_cap_node_death_test.go), TestLaunchVarEdgeFilterDropsOnlyWhatTheLaunchDecided, TestLaunchVarEdgeFilterNodeEndsAndIterationExits, TestReachWalksReadTheOpenEdgesOnly, TestEdgeSelectionClassifiesEveryRouterMode (pkg/dsl/ir/launch_var_edges_test.go) | the whole prune is ONE seam, ir.Workflow.LaunchVarEdgeFilter, which both reach walks read (CanReachVia, AlwaysReachesVia); AlwaysReachesLLM is deliberately NOT narrowed, since narrowing it would refuse runs rather than admit them. The vars the walk reads are the launch's own: TestARunsLaunchInputsAreWrittenOnlyFromTheLaunchMap (pkg/runner) and TestRunVarsAreBuiltOnceAndNeverWritten (pkg/dsl/ir) read the tree with go/ast, so the next writer of either map reddens there |
| cloud.usage-cap-supervisor-watch | a supervisor is a metered route only where the run's own spawn would start it: one whose every WATCHED node the launch vars excluded arms nothing (a coordinator evaluates only while a watched node is active) and leaves the pre-flight's walk, as do the specs a sovereign runner pool or a team_gateway content boundary refuses — and the drop may take nothing the spawn keeps, so an unpooled run keeps its whole-run supervisor judged and a pooled run keeps a gateway-pinned one; each drop a pre-flight makes (a pool-refused supervisor, a monitor the spec walk cannot parse) is logged where an operator reads admission, with the run id as a field | cloud | covered-deterministic | TestUsageCapPreflight_ASupervisorWatchingOnlyExcludedNodesIsNotARoute, TestUsageCapPreflight_APooledRunIsNotParkedForASupervisorTheSpawnRefuses, TestUsageCapPreflight_ThePoolFilterNamesTheSupervisorItDrops, TestUsageCapPreflight_EveryAdmissionLineNamesTheRun (pkg/runner/usage_cap_supervisor_watch_test.go) | every fixture here pins BOTH reviewers off the Anthropic wire, so the declared supervisor is the run's only metered route and each verdict is its own — measured: with the claude reviewer left on the wire, a mutation dropping every watch-scoped supervisor on an unreadable resume left the whole package green. The attribution half is read from a JSON logger because pkg/log's human format drops the fields map |
| cloud.credential-meter-attribution | a usage reading and a route's spend are booked on the credential that served them: an anthropic-env session (the pod's inherited env) charges no bundle credential, a GLM route on the anthropic wire is booked on the z.ai key (default or pinned beside a forfait), a spec naming its provider on the key pinned for it, a claw anthropic/… spec on the Anthropic key held for its route, a claw openai/… spec in claw's order (default key, ChatGPT forfait, pinned key) and never on a Moonshot key for anthropic/…, a pi anthropic/… or openai/… spec on that provider's key (default, then pinned — never a forfait nor a facade), a kimi, grok or opencode route on nobody, and a claude_code route on the credential its session named (fingerprint on delegate_finished) whatever its provider hint | cloud | covered-deterministic | TestUsageCapCredKeys_AmbientEnvChargesNoBundleCredential (pkg/runner/usage_cap_moonshot_test.go), TestCredentialSlotForRoute_GLMIsBookedOnTheZAIKey (pkg/runner/usage_cap_moonshot_test.go), TestCredentialSlotForRoute_APrefixPinIsBookedOnItsPinnedKey (pkg/runner/usage_cap_moonshot_test.go), TestCredentialSlotForRoute_AClawAnthropicSpecIsBookedOnItsKey (pkg/runner/usage_cap_moonshot_test.go), TestCredentialSlotForRoute_AClawOpenAISpecFollowsClawsOrder (pkg/runner/usage_cap_moonshot_test.go), TestRecordCredentialSpend_BooksAClaudeCodeRouteOnTheCredentialItsSessionNamed (pkg/runner/cred_spend_test.go), TestDelegateModelReachesStore (pkg/backend/model/delegate_observability_test.go), TestUsageCapCredKeys_ReadingFollowsTheSessionSource (pkg/runner/usage_cap_test.go), TestUsageCapCredKeys_APinnedKeyDoesNotScopeTheDefaultCredential (pkg/runner/usage_cap_moonshot_test.go), TestCredentialSlotForRoute_EachBackendIsBookedInItsOwnOrder (pkg/runner/usage_cap_moonshot_test.go) | the run is metered on its DEFAULT credential's ledger; a pinned key keeps its owner's |
| cloud.queue-dispatch | NATS work queue: enqueue, claim, schema-version handling | cloud | covered-deterministic | pkg/queue/types_test.go, TestSchemaVersionMismatchIsATypedTransientError (pkg/queue/schema_version_transient_test.go) | |
| cloud.runner-pod | runner claims a queued run, executes, reports status back | cloud | covered-deterministic | pkg/runner/loop_test.go | |
| cloud.runner-credentials | credential injection + sealing into a runner pod | cloud | covered-deterministic | pkg/runner/git_credentials_test.go, pkg/secrets/run_secrets_test.go | |
| cloud.probes-readiness | /readyz gates on CRITICAL dependencies only (a non-critical blip stays 200 degraded), /healthz never follows a dependency | cloud | covered-deterministic | TestReadyzCriticalVsDegraded (pkg/server/health_test.go), TestHealthzAlwaysOK (pkg/server/health_test.go) | every replica pings the same backends, so a critical check on a shared one converts a blip into a fleet-wide outage — the split is the promise, and the table drives all three outcomes through the real mux |
| cloud.probes-resilience | a dependency ping that panics, hangs, or is already in flight cannot kill the pod, hang the probe, or evict a healthy one | cloud | covered-deterministic | TestReadyzSurvivesAPanickingCheck (pkg/server/health_test.go), TestReadyzDoesNotHangOnAContextIgnoringCheck (pkg/server/health_test.go), TestReadyzDoesNotRelaunchAWedgedCheck (pkg/server/health_test.go), TestReadyzConcurrentProbesShareOneAnswer (pkg/server/health_test.go) | the probe runs pings concurrently in their own goroutines, i.e. outside net/http's recover and outside the request's lifetime — each test pins one way that turned into an un-drained exit (panic → process death, hang → wedged handler + blocked Shutdown, relaunch → goroutine leak → OOMKill) or into evicting a healthy pod (overlapping probes). All four were found by adversarial review, not by the original tests |
| cloud.probes-lame-duck | SIGTERM flips /readyz to 503 draining while the listener still accepts, for ITERION_SHUTDOWN_DELAY, with /healthz still 200 | cloud | covered-deterministic | TestServerCommandBootsLocalModeAndShutsDownOnSignal (e2e/cli_server_boot_test.go), TestReadyzDrainingIs503AndSkipsChecks (pkg/server/health_test.go), TestBeginDrainWaitsButRespectsContext (pkg/server/health_test.go) | the e2e half polls the REAL binary between SIGTERM and exit — the only place the window is observable; drop the draining branch and it fails within the 5s window (verified by mutation) |
| cloud.probes-runner | runner /healthz tells a busy consume loop from a wedged one; /readyz reports the whole lame-duck drain | cloud | covered-deterministic | TestHealthAliveDistinguishesBusyFromWedged (pkg/runner/health_test.go), TestShutdown_CompleteMode_LetsRunFinish (pkg/runner/loop_test.go), TestShutdown_NoInFlight_NoOp (pkg/runner/loop_test.go) | replaces a tcpSocket probe that could not distinguish them; the Shutdown tests are the WIRING half (flag actually flipped at drain start), the health tests the policy half |
| cloud.credential-pool | pledge/lease broker as the fourth credential tier | cloud | covered-deterministic | pkg/credpool/broker_test.go, pkg/server/cloudpublisher/credpool_tier_test.go, pkg/server/credpool_routes_test.go | |
| cloud.secrets-sealing | AES-256-GCM sealing + BYOK/generic/bot-binding domains | cloud | covered-deterministic | pkg/secrets/sealer_test.go, pkg/secrets/run_secrets_test.go | |
| cloud.bot-sources | team-authored bot bundles (fork a catalog bot, author a new one) | cloud | covered-deterministic | pkg/botsource/botsource_test.go, pkg/server/bot_sources_routes_test.go | |
| cloud.platform-bot-overrides | platform bot overrides: super-admin push/delete under the platform: sentinel, resolution team → platform → baked at every launch surface, digest-audited, size-capped | cloud | covered-deterministic | TestAdminBots_PlatformOverrideLifecycle (pkg/server/admin_bots_routes_test.go), TestAdminBots_SuperAdminOnly (pkg/server/admin_bots_routes_test.go), TestAdminBots_SizeLimits (pkg/server/admin_bots_routes_test.go), TestEffectiveEntries_PlatformOverlay (pkg/server/platform_settings_test.go), TestBotResolutionSweep_NoRawRegistryReads (pkg/server/bot_resolver_sweep_test.go) | the sweep test is what keeps NEW pkg/server code from bypassing the resolver — the class stays closed |
| cloud.platform-bot-runner-rebuild | a stored-bot run rebuilds its bundle from the DB ref on the runner, verifying the version (drift/missing row fail loudly, never a stale-baked fallback) | cloud | covered-deterministic | TestMaterializeBotBundle (pkg/runner/botbundle_test.go), TestValidate_DualAcceptWindow (pkg/queue/types_test.go) | the dual-accept test pins the v8/v9 rolling-deploy guarantee |
| cloud.platform-bot-roles | webhook role→bot bindings runtime-mutable (bot_roles settings family; consts stay defaults) | cloud | covered-deterministic | TestRoleBots_DefaultsAndOverride (pkg/server/platform_settings_test.go), TestAdminBotRoles_PutMergeAndOrigin (pkg/server/platform_settings_test.go), TestBotRoleSweep_ConstsOnlyInDefaults (pkg/server/bot_resolver_sweep_test.go) | |
| cloud.platform-sandbox-image | sandbox: auto default image runtime-mutable, resolved at publish and pinned on the RunMessage | cloud | covered-deterministic | TestAdminSandboxSettings (pkg/server/platform_settings_test.go), pkg/platformcfg/platformcfg_test.go | the runner consumes msg.SandboxImage via runtime.WithSandboxDefaultImage; publish-time pinning is what makes a redelivery rerun identically |
| cloud.marketplace | hosted registry entries (bot + plugin), moderation, visibility | cloud | covered-deterministic | TestMarketplace_SubmitListGetInstall (pkg/server/marketplace_routes_test.go), TestMarketplace_InstallThenUninstall (pkg/server/marketplace_routes_test.go), pkg/marketplace/moderation_test.go, pkg/marketplace/visibility_test.go, pkg/marketplace/jsonstore_test.go, pkg/cli/marketplace_test.go | submit → list → get → install/uninstall over the REST surface (plus the disabled/404/malformed-bundle refusals), with moderation status and visibility scope asserted at store level and the CLI half in pkg/cli |
| cloud.memory-store | cloud MemoryStore adapter + quota | cloud | covered-deterministic | pkg/knowledge/scope_test.go, pkg/memory/space_test.go, pkg/server/memory_routes_test.go | |
| cloud.dlq | dead-letter queue show/replay/delete (admin REST surface) | cloud | covered-deterministic | TestDLQAdmin_ListPeekReplayLeavesAuditTrail (pkg/server/dlq_admin_routes_test.go), TestDLQAdmin_NonSuperAdminCannotReplayOrDiscard (pkg/server/dlq_admin_routes_test.go) | the endpoints behind iterion remote admin dlq: super-admin gate (an unprivileged caller never reaches the queue), list/peek/replay/discard state transitions, the platform audit row naming the replayed run, and 400/404/502 mapping. Driven through the real New()/routes()/auth middleware with an in-process QueueBackend double (the seam added for it); the broker-side half is cloud.dlq-jetstream |
| cloud.dlq-jetstream | DLQ transport semantics: park with headers, inspect verbatim payload, replay/discard, report depth | cloud | covered-deterministic | TestSchemaRolloutMixedFleet (pkg/runner/schema_rollout_integration_test.go) | CI's nats-conformance job starts a real JetStream broker (nats -js) and verifies delayed redelivery, final park headers/reason, ListDLQ, PeekDLQ byte identity, RepublishDLQ/DiscardDLQ and DLQDepth. |
| cloud.lock-delivery-exhaustion | a delivery that cannot take the run lock reaches the DLQ at MaxDeliver without changing the run | cloud | covered-deterministic | TestHeldLockFinalDeliveryArchivesOnNATS (pkg/runner/lock_delivery_test.go), TestHeldLockDoesNotSpendFinalDeliverySilently (pkg/runner/lock_delivery_test.go), TestHeldLockArchiveFailureLeavesDurableEvidence (pkg/runner/lock_delivery_test.go), TestLockFailureReasonSeparatesHeldFromUnconfirmed (pkg/runner/lock_delivery_test.go), TestExhaustedPublishDeadlineStillRecordsTheAuditRow (pkg/runner/lock_delivery_test.go) | Real JetStream payload round trip in the NATS one (the broker CI's nats-conformance job starts). The pure-Go ones cover the failure halves it cannot reach: an archive failure stays visible on the run timeline, the archived reason separates CONFIRMED contention from a lock error that leaves ownership unconfirmed, and a publish deadline exhausted by a broker outage still records run_delivery_exhausted — the only confirmed trail when the DLQ copy is not. Every one asserts the run doc is untouched. |
| cloud.migrate-blobs | migrate to-cloud artifact/blob upload to S3 | cloud | covered-deterministic | TestMigrateToCloud_TransfersEveryVersionAndIsIdempotent (cmd/iterion/migrate_to_cloud_test.go), TestMigrateToCloud_DryRunWritesNothing (cmd/iterion/migrate_to_cloud_test.go), TestS3Client_ArtifactUploadRoundTripAndLayout (pkg/store/blob/s3_roundtrip_test.go), TestS3Client_DeleteRunSweepsOnlyThatRunsPrefix (pkg/store/blob/s3_roundtrip_test.go) | both halves of the one-way door: the walker (migrateRun between two real stores — every artifact VERSION, events, interactions, --dry-run writes nothing, re-runs don't duplicate) and the S3 upload driven by the REAL AWS SDK against an in-process path-style gateway (canonical key layout on the wire, byte round-trip, ErrArtifactNotFound mapping, prefix-scoped DeleteRun). Not covered: the Mongo store's own WriteArtifact→blob glue, which needs a live Mongo (mongo-conformance CI job) |
| cloud.desktop-exchange | desktop auth token exchange endpoint | cloud | covered-deterministic | pkg/server/desktop_sso_test.go | |
| cloud.valkey-state | ephemeral cross-replica state (OAuth/CSRF, board tokens, rate buckets) | cloud | covered-deterministic | TestValkeyCrossReplica_ForgeOAuthStartedOnACompletesOnB (pkg/server/valkey_cross_replica_test.go), TestValkeyCrossReplica_BoardTokenMintedOnAAuthorizesWriteOnB (pkg/server/valkey_cross_replica_test.go), TestValkeyCrossReplica_LoginRateBudgetIsSharedNotPerPod (pkg/server/valkey_cross_replica_test.go), pkg/server/valkey_stores_test.go | two servers built through the real New() wiring over ONE miniredis: each flow starts on replica A and finishes on replica B (OAuth connect → callback creates the connection, board token → board write, login burst → 429). Give each replica its own Valkey and all three fail. store-level TTL/GETDEL/write-failure surfacing in valkey_stores_test.go. Not covered: valkey.New's Sentinel-HA topology (needs a real Sentinel quorum; the single-node client is what miniredis speaks) |
| bots.catalog-universality | catalog bots stay repo- and stack-agnostic | bots | covered-deterministic | bots/catalog_universality_test.go | |
| cloud.bot-bundle-snapshot | cloud launch and resume freeze resources plus transitive subbots; a divergent runner catalog cannot replace the snapshot | cloud | covered-deterministic | TestSnapshotCatalogLaunchAndResume (pkg/server/bot_snapshot_test.go), TestSnapshotChildUsesServerAuthorityAndRefusesMissingChild (pkg/server/bot_snapshot_test.go), TestSnapshotRunnerIgnoresDivergentCatalog (pkg/runner/bot_snapshot_test.go), TestSnapshotRunnerFetchesAndChecksStoredBytes (pkg/runner/bot_snapshot_test.go), TestSnapshotOffloadIsImmutableAcrossResume (pkg/server/cloudpublisher/bundle_snapshot_test.go), TestSnapshotPromptsTravelInCompiledPayload (pkg/server/cloudpublisher/bundle_snapshot_test.go) | real child execution against conflicting runner catalog; object transport tested with fake blob backend; live coordinated rollout not covered |
| bots.catalog-compiles | every catalog bot parses + compiles clean | bots | covered-deterministic | bots/catalog_parse_compile_test.go, bots/catalog_typing_test.go | |
| bots.catalog-freshness | the generated bot-catalog skill matches the manifests | bots | covered-deterministic | bots/catalog_freshness_test.go | |
| bots.golden-replay | bot golden replay revalidates frozen LLM outputs | bots | covered-deterministic | TestGoldens (pkg/botreplay/goldens_test.go) | |
| bots.feature-dev | feature-dev (Featurly): campaign + gate convergence | bots | covered-deterministic | TestVibeFeatureDev_ConvergesFirstPass (e2e/feature_dev_test.go), TestVibeFeatureDev_RedVerifyRoutesBackToCampaign (e2e/feature_dev_test.go) | |
| bots.whole-improve-loop | whole-improve-loop (Willy) | bots | covered-deterministic | TestWholeImproveLoop_ContinuesUntilComplete (e2e/whole_improve_loop_test.go) | |
| bots.branch-improve-loop | branch-improve-loop (Billy) | bots | covered-deterministic | TestBranchImproveLoop_ContinuesUntilClean (e2e/branch_improve_loop_test.go) | |
| bots.docs-refresh | docs-refresh (Doki) | bots | covered-deterministic | TestDocsRefresh_ConvergesFirstPass (e2e/docs_refresh_test.go) | |
| bots.whats-next | whats-next (Nexie) chat loop + dispatch | bots | covered-deterministic | TestWhatsNextV2_ChatLoop_PauseResumeClose (e2e/whats_next_loop_test.go) | |
| bots.secured-renovacy | secured-renovacy (Renovacy) patch/minor/fix-loop paths | bots | covered-deterministic | TestSecuredRenovacy_PatchFastTrack (e2e/secured_renovacy_test.go) | |
| bots.sec-audit-source | sec-audit-source (Seki): cap_findings + scan_health gates | bots | covered-deterministic | TestSecAuditSource_ScanHealth_GuardsAgainstFacade (e2e/sec_audit_scan_health_test.go), TestSecAuditSource_CapFindings_BoundsScannerOutput (e2e/sec_audit_cap_findings_test.go) | |
| bots.sec-audit-deps | sec-audit-deps (Depsy): per-ecosystem + generic CVE heuristics | bots | covered-deterministic | TestSecAuditDeps_GenericHeuristic_DetectsMalwareSignals (e2e/sec_audit_deps_heuristics_test.go) | |
| bots.e2e-coverage | e2e-coverage (Endy): matrix gate + continuation loop | bots | covered-deterministic | TestE2ECoverage_ContinuesUntilComplete (e2e/e2e_coverage_bot_test.go), bots/e2e_coverage_matrix_gate_test.go | |
| bots.review-pr | review-pr (Revi): finding ids, stale anchors, gate status, concise publication and collapsed AI run details | bots | covered-deterministic | bots/review_pr_finding_id_test.go, bots/review_pr_stale_anchor_test.go, TestReviewPRRunDetails (bots/review_pr_run_details_test.go), TestReviewPRConcisePublication, TestReviewPRConciseScope (bots/review_pr_concise_test.go) | brief visible verdict; authenticated instance run link strips callback query/fragment; full findings/replacements survive missing anchors; gaps and decisions stay visible; verification limits are collapsed; served model/harness, run tokens and audited scope are collapsed; missing telemetry remains unavailable |
| bots.review-pr-mono-gpt | mono_family is the family of the WHOLE review on the bot the catalog ships: a mono-GPT launch is admitted while the Anthropic window is spent, soft cap or hard, with NO dial set at all — both claude reviewers, the claude merge converge and the claude pacer are off its topology, converge_gpt merges on claw and pacer_gpt paces off the wire, and the bound is twelve evaluations per armed pacer (twenty-four where both watch sets arm); the claude family still merges on the wire and still parks there, the twin's three dials pointed back at the wire park the run again (an explicit choice is honoured, never replaced), and the merge guard reads vars.mono_family ALONE — so review_tier: audit fans both reviewers onto the gpt twin wherever a detected family serves that pin (a set that cannot serve it refuses the launch, a detected family whose slot has no servable route refuses it pin or no pin — the resolver's own choice is judged, #1221 — and a host detecting no participating family at all resolves dual and deletes the pin like any dual), while an explicitly dual launch loses its pin to reviewtopology.InjectAll and merges on converge wherever that injection runs — which is the three in-process lanes always, and a cloud launch only when its sealed bundle resolved a family (an empty bundle rides the runner's env fallback, keeps the pin and merges on the twin) | bots | covered-deterministic | TestUsageCapPreflight_MonoGPTOnTheReviewPrBot, TestReviewPrTheTwoPacersEvaluationBound (pkg/runner/usage_cap_review_pr_test.go), TestReviewPrTheMergeGuardFollowsTheLaunchPinNotTheTopology, TestReviewPrAnExplicitlyDualLaunchDropsTheGptMergePin, TestReviewPrGptMergeTwinLeavesTheAnthropicWire (bots/review_pr_mono_gpt_merge_test.go), TestSubmitLaunchKeepsAnExplicitPinWhenTheBundleResolvedNothing (pkg/server/cloudpublisher/publisher_topology_test.go) | compiled from bots/review-pr/main.bot and its imports, so the walk reads the bot a launch publishes and not a shape standing for it; every ITERION_VIBE_* dial is blanked in the fixture. What stays outside this harness is the OPS half of #1828 — one webhook-launched mono-GPT review with no override at all, on a deployment whose Anthropic window is spent |
| bots.dep-update-guard | dep-update-guard (Vetty): prepare/gate/automerge routing | bots | covered-deterministic | bots/dep_update_guard_gate_test.go, bots/dep_update_guard_automerge_test.go | |
| bots.feed-watch | feed-watch: state machine, routing, SSRF-safe fetch | bots | covered-deterministic | TestFeedWatch_ScriptsStateMachine (e2e/feed_watch_test.go), TestFeedWatch_FetchRejectsSSRF (e2e/feed_watch_test.go) | |
| bots.feed-watch-split | feed-watch (Vigie): a digest over the per-message budget is delivered whole as consecutive numbered messages — per-sink max_chars, max_messages ceiling, no orphaned heading, intra-sink partial failure does not fail the run, nothing delivered does | bots | covered-deterministic | TestFeedWatch_NotifySplitsLongDigest (e2e/feed_watch_test.go) | drives the real python notify script against an httptest webhook that records every payload; the reassembly assertion is what proves no content is lost |
| bots.vuln-watch | vuln-watch (Senti): exploitation-driven alert policy, KEV re-fire, alias dedup, word-boundary inventory matching, explicit token failure, SSRF posture, at-least-once delivery, zero-LLM invariant | bots | covered-deterministic | TestVulnWatch_PolicyAndRefire (e2e/vuln_watch_test.go), TestVulnWatch_WordBoundaryMatching, TestVulnWatch_MissingTokenFailsHard, TestVulnWatch_FetchRejectsSSRF, TestVulnWatch_DryRunAndNoSinksDoNotConsume, TestVulnWatch_ZeroLLMInvariant | the real python tool scripts run against httptest GitHub/advisory/webhook fakes + a file:// KEV; the engine-side dependabot_tokens flow is covered by pkg/forge/security_read_test.go |
| bots.issue-triage | issue-triage (Triagy): author trust gate + label consumption | bots | covered-deterministic | TestIssueTriageTrigger_E2E_ConsumeAndLaunch (e2e/issue_triage_trigger_test.go) | |
| bots.golden-master-sync | the golden-master bot and its oracle harness stay in sync (same functions, same report fields) | bots | covered-deterministic | bots/golden_master_harness_sync_test.go | |
| bots.review-topology | mono/dual review topology injected only into opting-in bots; a mono_family no credential serves REFUSES instead of being substituted, at launch AND at resume on all three lanes (cloud against the re-sealed bundle, studio/API/detached and iterion resume against the host's set), through one read-only predicate that leaves the run's frozen inputs byte-identical; the refusal is typed, so the auto-retry sweeper abandons the run with the reason on its document rather than re-arming it | bots | covered-deterministic | TestReviewTopology_MonoClaudeSingleFamily (e2e/review_topology_test.go), TestFrozenPinNoCredentialServesRefusesTheResume, TestFrozenPinJudgementRewritesNothing, TestFrozenPinIsJudgedWhateverItsProvenanceOrMode, TestThePinRemedyBelongsToItsMoment (pkg/reviewtopology/frozen_pin_test.go), TestSubmitResume_RefusesAPinNoCredentialServesAnyMore, TestSubmitResume_AcceptsAPinACredentialStillServes (pkg/server/cloudpublisher/resume_frozen_pin_test.go), TestResume_LocalLaneRefusesAPinThisHostCannotServe, TestResume_TheCloudLaneIsNotJudgedAgainstThisHost (pkg/runview/resume_frozen_pin_test.go), TestResume_CLIRefusesAPinThisHostCannotServe (pkg/cli/resume_frozen_pin_test.go), TestSweepDueRetries_AnUnservableFamilyAbandonsInsteadOfReArming (pkg/server/retry_sweeper_test.go) | A credential set that resolves NOTHING is not judged on the cloud lane, by parity with its launch: the runner's env fallback is unknowable from the publisher |
| bots.verify-gate | the deterministic verify_build/verify_run gate resists drift | bots | covered-deterministic | bots/verify_run_drift_test.go, bots/verify_probe_wiring_test.go | |
| bots.live-covered-catalog | the catalog bots that HAVE a live test (evolve, adr-cartograph, adr-rechallenge, rgaa-audit, bmady, devbox-setup, revi-converse, feature-gap-fill, test-coverage) | bots | covered-live | e2e/live_bot_adr_cartograph_test.go, e2e/live_bot_adr_rechallenge_test.go, e2e/live_bot_evolve_test.go, e2e/live_bot_rgaa_audit_test.go, e2e/live_bot_bmady_test.go, e2e/live_bot_devbox_setup_test.go, e2e/live_bot_revi_converse_test.go, e2e/live_bot_feature_gap_fill_test.go, e2e/live_bot_test_coverage_test.go | gate: paid — the ledger rows of these bots read never except test:live:bot:rgaa-audit (pass 2026-10-13) — an honest gap per #1424 clause 2. these bots' value IS the LLM work product (a review, an ADR map, an accessibility audit); they have no deterministic graph gate to assert against, so the live layer + quality panel is the honest coverage. Their graphs are compile-checked by bots.catalog-compiles |
| bots.wiki-gen | wiki-gen (Wikky): the OKF wiki's structural pipeline — scan_repo's noop state machine (a wiki current for HEAD on a clean tree finishes at zero LLM cost, a committed code change names the changed files, an uncommitted tree never skips), gen_indexes regenerating every index from the concept pages' frontmatter (sorted, reserved files never listed, OKF stamp on the root only, committed when changed), validate_wiki's truth oracle (type-only frontmatter, dead intra-wiki links and anchors, docs-only scope; orphan concepts and a missing quickstart warn without blocking) and the graph gate that converges on the validator's verdict, never the author's claim | bots | covered-deterministic | TestWikiGen_ScanRepo_NoopStateMachine (e2e/wiki_gen_tools_test.go), TestWikiGen_GenIndexes_RegeneratesFromFrontmatter (e2e/wiki_gen_tools_test.go), TestWikiGen_ValidateWiki_GateRefusesAndWarns (e2e/wiki_gen_tools_test.go), TestWikiGen_UpdateCache_PersistsHeadAndPageShas (e2e/wiki_gen_tools_test.go), TestWikiGen_Run_CurrentWikiSkipsAtZeroCost (e2e/wiki_gen_tools_test.go), TestWikiGen_Run_ScopeViolationLoopsUntilTheGatePasses (e2e/wiki_gen_tools_test.go) | the prose the author writes is the one live-layer subject and has no live target yet (the ~$20/target spend is the operator's, #1424 D7/D32); everything that JUDGES that prose structurally is deterministic and pinned here. Mutation seen red: the gate's scope_ok term dropped, the loop test reddened on its author-count assertion |
| bots.supply-shield | supply-shield (Shieldy): the malware shield's deterministic pipeline — diff_scope scoping a run to the lockfiles a change touches (full mode walks the tree and says so), normalize_deps coercing every enumerator shape into the canonical cache key (dedup, install-path prefixes stripped), coverage_gate refusing the 0-finding façade (floor absent = exit 1; deeper analyzers absent = degraded banner, never a fail), the verdict cache (kind=malware isolation, TTL staleness, scanner-version skew, the republish guard re-inspecting a name@version whose checksum changed), sarif_gen's SARIF 2.1.0 for code-scanning | bots | covered-deterministic | TestSupplyShield_DiffScopeScopesToChangedLockfiles (e2e/supply_shield_pipeline_test.go), TestSupplyShield_NormalizeDepsCoercesEnumeratorShapes (e2e/supply_shield_pipeline_test.go), TestSupplyShield_CoverageGateRefusesTheFacade (e2e/supply_shield_pipeline_test.go), TestSupplyShield_CacheRoundTripDedupTTLAndRepublish (e2e/supply_shield_pipeline_test.go), TestSupplyShield_SarifGenMapsVerdicts (e2e/supply_shield_pipeline_test.go) | what a signal MEANS (malware or not — the deep-read and the verdicts) is the reviewer's LLM call and has no live target yet (the ~$20/target spend is the operator's, #1424 D7/D32); the scanner parsers are copy-shared with sec-audit-deps, whose heuristics carry their own row — fix one, check the other two. Mutation seen red: the gate's floor sys.exit(1) neutralized, the façade test reddened on its exit-code assertion |
| bots.supply-shield-cve | supply-shield-cve (Vulny): the CVE shield's deterministic pipeline — supply-shield's shape on the CVE axis: the universal floor's osv-scanner parser (advisory id, max_severity folding a CVSS float to a label, aliases, per-package fold with summed weights), coverage_gate keyed on the trivy + osv floor, and the cache consulting ONLY kind=cve lines on a SHORT TTL (7 days + advisory_db_date — a clean-today version can gain a CVE tomorrow; the malware sibling's verdicts never satisfy a CVE shield) | bots | covered-deterministic | TestSupplyShieldCve_GenericHeuristicParsesOsvScanner (e2e/supply_shield_cve_pipeline_test.go), TestSupplyShieldCve_CoverageGateRequiresTheCveFloor (e2e/supply_shield_cve_pipeline_test.go), TestSupplyShieldCve_CacheConsultsOnlyCveLinesOnShortTtl (e2e/supply_shield_cve_pipeline_test.go) | advisory-vs-version applicability (range / fixed version / reachability) is the reviewer's LLM call and has no live target yet (the ~$20/target spend is the operator's, #1424 D7/D32); the tool bodies are copy-shared with sec-audit-deps and supply-shield. Mutation seen red: the kind=cve filter dropped, the axis-isolation test reddened on its own assertion |
| bots.modernize | modernize (Mady): the lot programme's gate semantics — a converged lot is marked by the gate exactly once, a refused or worker-written done verdict goes back round repair_loop and a refusal that outlives it fails the run (never a finished that would land an unproven done on a merge), a gate timeout fails at once, a declared-blocked lot stops without marking, provenance refusals are verdicts | bots | covered-deterministic | TestModernize_ConvergedLotIsMarkedByTheGate (e2e/bot_modernize_test.go), TestModernize_RefusedVerdictNeverEndsGreen (e2e/bot_modernize_test.go), TestModernize_GateTimeoutFailsAtOnce (e2e/bot_modernize_test.go), TestModernize_DeclaredBlockedStopsWithoutMarking (e2e/bot_modernize_test.go), TestModernize_RefusalsAreVerdicts (e2e/bot_modernize_test.go), TestModernizeLotVerifyRunsTheDeclaredSuites (bots/modernize_lot_verify_suites_test.go), TestModernizeMarkDone (bots/modernize_mark_done_test.go), TestModernizePlanReadContractShape (bots/modernize_plan_read_test.go) | the aggregate row's claim was stale — these suites predate it. The worker's code changes are LLM work the gates cannot judge; what they judge is that nothing unproven lands, and that is the surface a regression bites. Mutation seen red: the lot_gate refused term (done_self_written) dropped, the refused-verdict test reddened on its terminal-status assertion |
| bots.app-dev | app-dev (Devly): the interview loop's exhaustion exits — a spent operator-answer cap refuses typed (INTERVIEW_NOT_CONVERGED, terminal: an exhausted loop stays exhausted), a budget-starved turn parks the run resumable, a live-steering grant extends the cap in force, and nothing (no plan, no campaign) runs behind an interview that did not converge; the verify_build/verify_run drift gates hold its deliverables to the same tail as every bot's | bots | covered-deterministic | TestAppDevInterviewGrantExtendsTheCap (e2e/app_dev_interview_exhaustion_test.go), TestAppDevInterviewExhaustionRefusesTyped (e2e/app_dev_interview_exhaustion_test.go), TestAppDevInterviewBudgetStarvedParksResumable (e2e/app_dev_interview_exhaustion_test.go), TestAppDevVerifyRunEnforcesTheSameTail (bots/verify_run_drift_test.go), TestCleanTreeClauseNamesASafeDiscardVerb (bots/clean_tree_clause_test.go) | the scaffolded app's quality is the campaign's LLM work and has no live target yet (the ~$20/target spend is the operator's, #1424 D7/D32); what is deterministic is the loop's decision surface, and the row claims exactly that. Mutation seen red: the cap edge >= loosened to >, the exhaustion test reddened on its failure-code assertion |
| bots.smoke-board-capabilities | smoke (board_smoke.bot): a three-call exercise of the __mcp-board MCP server (two create_issue + one transition) by a real model | bots | excluded | a capability smoke fixture, not a shipped bot: no manifest.yaml, so it cannot launch as a bot nor be installed from the catalog; its parse+compile floor stays with bots.catalog-compiles. The capability it exercises manually is covered deterministically where it lives: the board tool surface and its capability filtering (TestRegisterClawBoardTools_ExecuteHitsStore, TestRegisterClawBoardTools_FiltersByCaps — pkg/backend/tool/claw_board_tools_test.go) and the internal-server registration (TestWireUserMCPPreservesInternalServers — pkg/backend/delegate/mcp_reserved_names_test.go) | |
| studio-ui.run-console | run console view: timeline, node detail, diffs, chat | studio-ui | covered-deterministic | studio/e2e/specs/run-console.spec.ts | Playwright against the real server: the seeded run's IR graph (node ids, kinds, per-node status incl. the never-reached fail), its declared budget, its replayed edge-selection log lines and a published artifact's real payload all render. Chat needs a live agent — out of this row's deterministic reach |
| studio-ui.launch-modal | Launch modal: bot picker, vars, overrides, target repo | studio-ui | covered-deterministic | studio/e2e/specs/launch.spec.ts | Playwright: ?bot= resolves through the catalog to the workflow, the vars: block splits into required primaries vs defaulted Bot options, missing-required validation blocks the launch, and Engine options offers the workflow's own agent node for model retargeting. The "target repo" attach needs a forge connection (covered at REST level by the forge rows). The same spec found a REAL BUG — preview-cost answers nodes: null for a workflow with NO agent/judge node and CostPreviewChip dereferenced it, dropping the whole view into its error boundary — fixed in 457374dd; the test now asserts the positive contract (such a workflow still renders a launchable view) |
| studio-ui.board | /board kanban with drag-and-drop | studio-ui | covered-deterministic | studio/e2e/specs/board.spec.ts | Playwright: the native board's real columns + the CLI-seeded card render, and moving the card through the UI is asserted against the native store's REST read AND a reload (an optimistic-only client update fails). The move uses the selection toolbar rather than an HTML5 drag gesture — same mutation, same endpoint, no synthetic-DnD flake |
| studio-ui.pipelines | /pipelines control-center board + concurrency cap | studio-ui | covered-deterministic | studio/e2e/specs/pipelines.spec.ts | Playwright: the banner renders the cap the server actually booted with (--max-concurrent-pipelines, read back from /api/server/info — changing one side alone turns it red), and the seeded finished runs land as Closed cards linking to their own run console |
| studio-ui.dispatcher | /dispatcher live dashboard | studio-ui | covered-deterministic | studio/e2e/specs/dispatcher.spec.ts | Playwright drives the whole lifecycle: Start refused without a config, settings saved, dispatcher started, then the dashboard asserted against THAT instance's real settings (name, tracker, poll interval, slots) and lane counters, then stopped through its confirm dialog. A second test asserts the server's own config-validation error reaches the dialog. Poll pinned to 1h so the actor never claims the seeded card mid-test |
| studio-ui.bots-gallery | /bots gallery, per-bot home, guided builder /bots/new | studio-ui | covered-deterministic | studio/e2e/specs/bots-gallery.spec.ts | Playwright: the gallery lists what botregistry discovered on disk (and its search filters that set), the per-bot home renders the manifest fields + "declares no vars", and the guided builder is driven end to end — the assertion is the bundle it WRITES (main.bot + manifest.yaml under bots/scaffold-probe/) and the registry picking it up, not a toast |
| studio-ui.editor | workflow editor + /api/parse round-trip | studio-ui | covered-deterministic | studio/e2e/specs/editor.spec.ts | Playwright: the canvas + inspector are asserted against what the Go parser produced (node kinds, the tool node's command, entry node, budget) plus the real compiler diagnostic (C128), and an inspector edit is followed through save into the REWRITTEN .bot source on disk and back out through a fresh parse |
| studio-ui.source-view | the per-file Source view of a bot in SEVERAL files: the picker over the unit's files, per-file render by provenance, the writer's refusal, and a per-file save that touches that file alone | studio-ui | covered-deterministic | studio/e2e/specs/source-view.spec.ts, studio/src/components/SourceView/SourceView.unit.test.tsx, studio/src/components/SourceView/SourceView.salvage.test.tsx | Playwright against the real server over a multi-file fixture whose lib/refused.bot the writer genuinely cannot reproduce (#1612), so the refusal banner is the server's own. The headline no unit test can prove is the per-file save: after an edit + Save the bot's three OTHER files are byte-identical on disk. The merged entry is asserted by its CONTENT — a merged render that fails leaves the previous file on screen under a picker that says "Merged program", and the read-only note alone still passes. One test is a layout hit test: the picker's wrapper is position: relative, so any overflow paints above the static Apply/Cancel buttons and swallows their clicks — found by this spec on its first run |
| studio-ui.editor-llm-meta | agent/judge canvas cards show the resolved model and the authored fallbacks: chain | studio-ui | covered-deterministic | studio/src/components/Canvas/WorkflowNode.test.tsx, studio/src/lib/modelLabel.test.ts | promoted from unit-only (#2799): the card needs no live model — the resolution answer is data in the query cache (what fetchResolvedModel would fill), and a vitest render of the real WorkflowNode pins the wiring (dropping resolvedModel from the displayModel call, or the displayFallbackChain call, reddens it on its own assertion); Playwright would add only the browser chrome |
| studio-ui.secrets-view | Secrets view gated on server_info.secrets_enabled | studio-ui | covered-deterministic | studio/e2e/specs/secrets.spec.ts | Playwright: the gate flag is asserted, then add → list → delete is driven through the UI with the SEALED store file on disk as the oracle (name + last4 present, plaintext absent). ITERION_HOME/HOME/ITERION_SECRETS_KEY are redirected into the throwaway workspace so the operator's own store and keychain are never touched |
| studio-ui.browser-pane | Browser pane / preview attach | studio-ui | covered-deterministic | studio/e2e/specs/browser-pane.spec.ts | Playwright: a fixture tool node prints [iterion] preview_url=…, the runtime turns it into a preview_url_available event, and the pane auto-reveals carrying exactly that URL (bar + open link). The negative case — a run that published none has no Browser tab — pins the trigger. The URL is the suite's own loopback origin, so the iframe never leaves the test server. Live CDP attach needs a real Chromium session and stays out of the deterministic layer |
| desktop.wails-app | Wails desktop wrapper (window, runtime bridge, packaging) | desktop | unit-only | studio/src/tests/desktopBridge.test.ts | the Go half is cgo + build-tagged (desktop,webkit2_41) and excluded from lint/CI; an e2e needs a GUI session — the bridge contract is the testable seam (read 2026-10-30, #2799: confirmed — the cited test pins browser-mode rejection and desktop-mode dispatch against mocked bindings; a GUI e2e would re-test the harness) |
| integrations.third-party-oauth | real third-party OAuth consent flows (GitHub App, IdP, MCP OAuth) | integrations | excluded | needs a real provider tenant and a human consent screen; iterion's side (state/PKCE, callback handling, token storage) is covered by cloud.sso-oidc, forge.github-app and tools-mcp.mcp-oauth. Re-read 2026-10-31 (#2798): premise holds — the three carriers' citations resolve and no third-party consent flow gained an iterion-side harness | |
| integrations.claude-md-autoload | claw TUI CLAUDE.md auto-load | integrations | excluded | a claw-code-go TUI-boot behaviour with no iterion workflow surface; iterion uses its own command registry (carried over from docs/e2e_coverage.md). Re-read 2026-10-31 (#2798): premise holds — the autoload lives in claw's internal ConversationLoop (its CtxAssembler), which iterion never constructs; iterion drives its own generation loop and renders the memory walk itself (claw_ambient.go) | |
| integrations.claw-lifecycle-hooks | claw lifecycle hooks / ctx propagation into hook handlers | integrations | covered-deterministic | TestTheProductionHostHandsTheDefaultLifecycleHooksToTheClawBackend (pkg/runview/claw_lifecycle_wiring_test.go), TestTheSandboxedRunnerHandsTheDefaultLifecycleHooksToItsBackend (cmd/iterion/claw_runner_lifecycle_wiring_test.go), TestClawBackend_DefaultLifecycleHookBlocksInsideTheLoop (pkg/backend/model/claw_backend_lifecycle_hooks_test.go), TestNewDefaultLifecycleHooks_BlocksDangerousCommand (pkg/backend/model/lifecycle_hooks_test.go), TestGenerateTextDirect_LifecycleHooks_BlockShortCircuits (pkg/backend/model/generation_test.go), TestGenerateTextDirect_LifecycleHooks (pkg/backend/model/generation_test.go) | re-measured 2026-10-31 (#2798): the old WHY went stale — iterion DOES install a lifecycle runner (runview builds NewDefaultLifecycleHooks, a PreToolUse SafetyHook, through WithBackendLifecycleHooks and WithLifecycleHooks) and claw_backend lands it in the loop's opts.Hooks, where a Block short-circuits Execute with a refusal tool_result the model reads; the wiring test drives the real backend over the default hook and was seen red with opts.Hooks = b.lifecycleHooks deleted, the loop-level pair pins event ordering and the short-circuit, the unit suite pins the patterns; the runview handoff itself is guarded at its own site (the backend registry BuildExecutor fills stays private, so the witness reads the source); the sandboxed __claw-runner rides the same runner, so the refusal holds in-container even on a workspace whose settings carry no hooks, and its handoff is guarded the same way |
| integrations.fixer-workflow-permission | refuse a workflow-changing delivery unless the runtime token PROVES the permission — Billy before analysis, Vetty at commit time on the alignment it is about to push | integrations | covered-deterministic | TestFixerWorkflowDiffRefusesBeforeAnalysis (pkg/runtime/fixer_delivery_preflight_test.go), TestForgeDeliveryPreflightUsesActualTokenProof (pkg/server/forge_delivery_preflight_test.go), TestDepUpdateGuardDeliveryProbeJudgesTheAlignmentNotTheBump (bots/dep_update_guard_delivery_probe_test.go), TestDeliveryProbeReadsTheForgeAPIsScopeListOnlyWhenItAnswers (bots/delivery_probe_fault_matrix_test.go) | real catalog workflow and shell tools; forge mint/callback HTTP fixtures; the bot-side rows drive the REAL compiled probe of both carriers against stub servers; live App permission rollout not covered |
| integrations.delivery-preflight-unavailable | a delivery preflight that answered NOTHING never becomes a blocking verdict on a pull request: the run ends FORGE_PREFLIGHT_UNAVAILABLE and the remedy the gate check carries names the cure | integrations | covered-deterministic | TestDeliveryProbeOnlyAWellFormedAnswerCanHold (bots/delivery_probe_fault_matrix_test.go), TestDeliveryProbeCarriersRefuseAnUnprovableDelivery (bots/delivery_probe_drift_test.go), TestForgeDeliveryPreflightNeverAnswersItsOwnOutageAsAVerdict (pkg/server/forge_delivery_preflight_test.go), TestForgeDeliveryPreflightRefusesOnlyFromTheProvidersAnswer (pkg/server/forge_delivery_preflight_test.go), TestGateReconcile_APreflightThatDidNotAnswerSendsTheReaderToTheRunsCure (pkg/server/forge_gate_reconcile_test.go) | fault-injection matrix: a raw-socket stub dies at every stage (refused, reset before/mid headers, mid body, short or over-long body, garbage, missing field, chunked cut, timeout, redirect, duplicate Content-Length, unreadable token) against the real compiled program of both carriers, the 409 and 503 the endpoint answers included; the server half is the cross-product of the connection record, the credential's owner, the mounted token's identity and the mint proof, beside the store/sealer outage rows of TestForgeDeliveryPreflightNeverAnswersItsOwnOutageAsAVerdict and a sweep of every field of those records, taken from their Go types and moved to every value its type declares, to every string value the handler's own source spells (both read from source, not listed in the test) and to the absent and the unexpected, under a granting provider map — asserting only the provider's recorded permission map can refuse, in any spelling and at any value either source names |
| integrations.permission-modes-auto | claw permission modes auto / dontAsk | integrations | excluded | iterion runs headless (workflow mode) and never surfaces these modes; the two it does use are covered by tools-mcp.permission-gate. Re-read 2026-10-31 (#2798): premise holds — claw's auto / dontAsk modes are resolved in its internal TUI runtime config, which iterion never drives; its claw nodes run iterion's own gate instead | |
| reliability.admission-before-execution | a run whose declared execution context does not match the run record is refused at launch AND at resume, with a durable decision either way, and the refused resumable run stays resumable | reliability | covered-deterministic | TestAdmissionEnforceDeniesBeforeExecution (pkg/runtime/admission_test.go), TestAdmissionResumeDenialDoesNotDestroyResumableRun (pkg/runtime/admission_test.go), TestAdmissionLegacyRunIsRecordedAndAllowed (pkg/runtime/admission_test.go) | the gate's position — before attachment promotion, workspace setup, sandbox start and the first executor call — is the CALL SITE's order (pkg/runtime/engine_run.go, resume.go), not this row's: the cited tests drive a workflow whose only node is done, so none of them would notice a node that ran. A witness driving a real executor node and asserting zero dispatch is owed (ADR-152) |
| reliability.one-structured-cause | the ONE versioned diagnostic projection every consumer reads — cause, responsible node, recoverability, next action — is built once and the snapshot a reader polls carries that same projection, not a second one | reliability | covered-deterministic | TestBuildDiagnostic_ProjectsNestedNodeOutputDetail (pkg/runview/diagnostic_test.go), TestBuildDiagnostic_ProjectsResumableFailureAndRetryEvidence (pkg/runview/diagnostic_test.go), TestSnapshotBuilder_IncludesTheSameDiagnosticProjection (pkg/runview/diagnostic_test.go) | what reddens is the AGREEMENT of the projection's two producers — BuildDiagnostic and SnapshotBuilder — on the version, the responsible node, the outcome and the next action, plus the recoverability and the allow-listed evidence fields. No test enters through the CLI, Studio or Copi, so this row claims one projection rather than three surfaces reading it; a per-surface witness is an owed qualifier (ADR-152's ownership table) |
| reliability.declared-store-divergence | an UNDECLARED run-store or business-store divergence is blocked before fan-out | reliability | uncovered | the admission gate compares the workflow revision, the lineage and the runner-expected triple, never the declared stores — and the identity it would compare is not comparable yet: Root() is empty for the Mongo store by design, so the derived id is ONE constant for every database, and the kind comes from %T, which varies under a store decorator that changes nothing operationally. Owed: a store-identity capability on the RunStore seam, then the comparison at the admission chokepoint (ADR-152 decision 6). BusinessStores has no producer at all | |
| reliability.producer-bounded-repair | a tool node whose recipe is a command or a script is repaired at its producer when its postcondition stays unmet — rung 3 entered only for a recipe a corrected command can replace, rung 4 spending exactly the declared max_agent_attempts, the postcondition the only success oracle, with a durable node_verified_action outcome | reliability | covered-deterministic | TestVerifiedAction_SelfRepair (pkg/backend/model/executor_verified_action_test.go), TestVerifiedAction_OnlyARepairableRecipeEntersTheSelfRepairRung (pkg/backend/model/executor_funding_test.go), TestOnlyPolicyRecoverHandsTheGoalToTheRungAgent (pkg/backend/model/permission_recovery_rung_test.go), TestVerifiedAction_RecoverExhaustedFails (pkg/backend/model/executor_verified_action_test.go), TestVerifiedActionEngineEmitsAndStrips (e2e/verified_action_test.go) | ADR-044's ladder — the shipped bound of #1006 criterion 4, which reads partial on this very gap; dsl.verified-action covers the quad's compilation. The Scenario claims only what reddens: rung 3's genre gate (ir.ToolRecipeSelfRepairable; a registry recipe that budgets repairs ALONE reaches no rung and is warned C179 — with max_agent_attempts budgeted the same recipe reaches rung 4 and C179 stays silent) and rung 4's declared attempt, spent exactly once. The author-declared limit as a WHOLE is not claimed: the self-repair rung's attempt count is covered by nothing — inflate that loop and the whole producer package stays green (191s, zero failures) — and its witness is an owed slice (ADR-152 decision 3) |
| reliability.schema-reask-bound | a schema-invalid model output that IS eligible for a re-ask is re-asked EXACTLY once — the validation error as the model's next input — and then fails on the field still missing, and a provider error returned as the answer is named rather than validated | reliability | covered-deterministic | TestValidateAndRetry_InjectsSchemaFeedback (pkg/backend/model/executor_retry_feedback_test.go), TestSchemaReask_AsksOnceThenFails (pkg/backend/model/schema_reask_test.go), TestValidateAndRetry_AProviderErrorAsResultIsNamedNotValidated (pkg/backend/model/executor_retry_feedback_test.go), TestValidateAndRetry_BooksTheClawRecoveryCall (pkg/backend/model/executor_retry_feedback_test.go) | the EXACTLY-once half reddens on both exits: TestValidateAndRetry_InjectsSchemaFeedback asserts two backend calls when the re-ask corrects the output, TestSchemaReask_AsksOnceThenFails asserts two delegations when it does not (fire a second re-ask and it fails on three), and TestValidateAndRetry_BooksTheClawRecoveryCall pins the bill of the re-ask and of the claw recovery behind it. The ELIGIBILITY gate is NOT claimed here: nothing refuses a re-ask bought for an ineligible shape — force retryEligible true and the whole producer package stays green (387s, zero failures) — because the only test that feeds a type mismatch, TestExecuteBackendKeepsSpendOnInvalidStructuredOutput, asserts the bill and never the call count. Owed: an ineligible shape buys exactly ONE backend call (ADR-152 decision 3), a witness that lands with the wiring of decision 5, where the eligibility becomes the corrector's answer to the engine |
| reliability.engine-correction-ledger | the engine's optional correction loop keeps a durable episode, a budget that survives a process restart and an unchanged-fingerprint stop, and publishes only a valid output | reliability | unit-only | TestSchemaValidation_BoundedCorrectionPersistsSuccess (pkg/runtime/validation_test.go), TestSchemaValidation_PreseededTerminalEpisodeIsNotReinvoked (pkg/runtime/validation_test.go), TestSchemaValidation_UnchangedCorrectionStopsImmediately (pkg/runtime/validation_test.go) | the loop needs four conditions at once and two of them have no production caller: WithOutputValidation is called from tests only, and the sole OutputCorrector implementors are test doubles. An e2e would drive the same double the unit suite already drives. The operator decided on 2026-11-03 to wire the seam — ClawExecutor becomes its production implementor (ADR-152 decision 5) — and this row is restated with that unit, whose witness drives the real producer |
| reliability.unchanged-repair-stop | an identical correction cannot spend a budget without progress at the producer | reliability | uncovered | every production correction IS bounded in the code — one re-ask by construction, the ladder by its declared attempt budget (witnessed on rung 4 by TestOnlyPolicyRecoverHandsTheGoalToTheRungAgent, unwitnessed on rung 3) — so no unbounded loop exists; what is missing is the STOP: the self-repair rung carries the previous stdout, stderr and resolved command forward and compares no fingerprint, so an identical corrected command spends every attempt. Owed: the stop and a witness that reddens on an identical proposal (ADR-152 decision 4). Model nodes reach the engine episode's stop once ADR-152 decision 5's wiring lands; the ladder's stop is owed either way | |
| reliability.resume-artifact-compatibility | a resume refuses artifacts incompatible with the declared contracts, and a forced compatibility survives a workflow restamp without destroying an effect | reliability | covered-deterministic | TestValidateArtifactContractsRefusesIncompatibleRevisionInEnforce (pkg/runtime/artifact_contract_test.go), TestForcedArtifactCompatibilitySurvivesWorkflowRestamp (pkg/runtime/artifact_contract_test.go) | |
| reliability.shared-outage-circuit | an open shared-outage circuit delays a due retry without spending one of its attempts, and the circuit key is composed from the workflow revision rather than from the run | reliability | covered-deterministic | TestSweepDueRetries_OpenCircuitDelaysWithoutSpendingAttempt (pkg/server/retry_sweeper_test.go), TestKeyPrefersWorkflowRevision (pkg/retrycoord/circuit_test.go) | the sweeper is the witness that reddens on the DELAY: under an open circuit the due retry is delayed and no attempt is spent (gut RecordFailure/Open and it fails), and the key test refuses a key that ignores the workflow revision. Cross-scope ISOLATION is NOT witnessed and this row does not claim it: every circuit-store double in the tree discards the key and answers one circuit, so a mutant collapsing the sweeper's key into one global circuit keeps the cited tests and the whole package green — the witness is owed (two runs, two revisions, one open circuit; ADR-152). The durable open/close streak is pinned by TestRetryCircuitFailureOpensAndSuccessCloses (pkg/store/mongo/retry_circuit_test.go), which needs ITERION_TEST_MONGO_URI and runs in CI's Mongo conformance job |
| reliability.watcher-new-evidence | a watcher does not repeat an action without new evidence: a durable cursor per run, supervisor and watched set suppresses a duplicate wake, and a cursor read that failed stops the actions instead of replaying them | reliability | covered-deterministic | TestCoordinatorPersistsCursorAndSuppressesDuplicateWake (pkg/supervise/coordinator_cursor_test.go), TestCoordinatorCursorLoadFailureStopsActionsAndWrites (pkg/supervise/coordinator_cursor_test.go) | |
| reliability.scenario-matrix | ONE suite drives launch, fan-out, crash, resume, rewind, quota park and bounded repair over simulated executors and asserts zero duplicated external effect, zero unauthorised destructive resume and zero unbounded identical loop | reliability | uncovered | the coverage exists lot by lot (pkg/runtime, pkg/runview, pkg/runner, and the rewind halves of e2e/) and no single suite asserts the three invariants together, so a regression report cannot name the scenario it broke; botreplay is deliberately not a graph simulator. Owed: the suite on the scenarioExecutor seam already in e2e/ (#1006 criterion 10) | |
| reliability.rollout-mode-floor | the reliability mode resolves in tiers with the process environment as a FLOOR, an explicit legacy as a cap no level overrules, and an unparseable value warning once without lowering the gate | reliability | covered-deterministic | TestResolveKeepsTheRollbackLeverAgainstALevel (pkg/reliability/rollout_test.go), TestResolveLetsALevelRaiseButNeverLowerTheFloor (pkg/reliability/rollout_test.go), TestTheAliasIsNotTheRollbackLever (pkg/reliability/rollout_test.go), TestInvalidModeWarnsOnceAndKeepsValidAlias (pkg/reliability/rollout_test.go) | the database level above the floor is ADR-132 and #2699 territory; the variable is operator-grade refused to bots and to .env files, so no repo-provided environment can set it |
| reliability.baseline-blocked-admissions | the pilot baseline counts the admissions it BLOCKED, reads them against the admissions it recorded, and never reads an unknown admission verdict as allowed | reliability | covered-deterministic | TestSummarizeCountsBlockedAdmissions (pkg/reliability/rollout_test.go), TestSummarizeDoesNotReadAnUnknownVerdictAsAllowed (pkg/reliability/rollout_test.go), TestReliabilityReportBaselineReadsTheSourceItIsGiven (pkg/cli/reliability_test.go) | |
| reliability.pilot-comparison | the pilot report compares cost, delay and avoidable interventions with the baseline over a declared observation window | reliability | uncovered | the artifact and its counters ship (see reliability.baseline-blocked-admissions); the comparison needs a pilot that RAN — ITERION_RELIABILITY_MODE=report on the deployment, one values line through the chart's existing env passthrough, which is the operator's gesture (#1006 criterion 11) |
