iterion

ADR-044 — Adaptive recovery for deterministic ACTION nodes (gates stay deterministic)

Status: accepted (partial) (2026-06-23) — the Verified Action quad is implemented on tool nodes: layers A (postcondition), E (policy spine), and C (self-repair) ship enabled; B (agent recovery) is wired but opt-in and OFF by default (recovery.max_agent_attempts: 0). Layer D (robust primitive library, e.g. iterion __commit) and extending the quad to compute nodes are deferred follow-ups. Originally proposed / exploratory the same day (kept below).

Implementation (2026-06-23)

The quad is goal + command/script (recipe) + postcondition + policy on a tool node — all optional and fully backward-compatible (a node with no postcondition behaves exactly as before: recipe only, exit code = success).

• DSL/IR: new optional tool properties goal:, postcondition:, policy: (required | recover | best_effort), and a recovery: block (max_repair_attempts, max_agent_attempts, model, agent_tools). Parser + AST + IR + unparse round-trip them.
• Runtime ladder lives in pkg/backend/model/executor_verified_action.go: idempotent-skip → recipe → self-repair → agent-recovery → policy, with the postcondition re-checked at every rung as the single source of truth (success is keyed on it, never the recipe exit code). Self-repair’s corrected command and the postcondition check are emitted as tool_called events; the per-node outcome is a node_verified_action event.
• Anti-Goodhart firewall is enforced at compile time, not left to convention: C103 (invalid policy), C104 (recovery without a postcondition — no truth oracle), C105 (recovery attached to a gate where recipe == postcondition), C106 (recovery bounds under a non-recover policy — dead config). A gate stays the degenerate quad (recipe == postcondition, no rungs 3–4, policy: required).
• Demonstration: bots/secured-renovacy’s commit_changes node now carries a postcondition (“working tree has no relevant uncommitted changes left, caches excluded”) + policy: recover, so it self-heals past the two brittle failures cited below instead of hard-blocking.

---

Status (original): proposed / exploratory (2026-06-23) — reflection, no implementation yet.

Context

Deterministic tool/compute nodes are iterion’s anti-Goodhart / anti-façade backbone: scan_health, coverage gates, streak_check, cap_findings (see docs/workflow_authoring_pitfalls.md). They are fast, cheap, auditable, and never drift — that is why review loops are trustworthy.

But on the action side they are brittle, and brittleness hard-blocks a run. Evidence from the 2026-06-23 re-test, on a single node, twice in a row:

• secured-renovacy commit step failed on a stray Go module cache overflowing Node’s 1 MB execSync maxBuffer (…1050864 more characters, exit 1);
• after that was patched, it failed again on :!__pycache__ — git short-form pathspec magic mis-parsing a _-leading pattern (Unimplemented pathspec magic '_', exit 128).

Each needed a code patch. An LLM asked to “commit the relevant changes, excluding caches” would have adapted past both without a human in the loop. sec-audit-source’s voters similarly hard-failed on one missing structured-output field. The deterministic path’s strength (no drift) is its weakness (no adaptation → hard block on the unanticipated).

The crux: not all non-LLM nodes are equal

• GATE / verification nodes (judge coverage, enforce the approval streak, check scan health, cap findings) — MUST stay deterministic. Their whole purpose is to be un-gameable by the LLM (the Goodhart firewall). An LLM fallback here reintroduces the façade risk the gate exists to prevent. → never LLM-fallback a gate.
• ACTION / side-effect nodes (commit, git ops, file writes, invoking a scanner, mechanical transforms) — have no gaming surface; their only failure mode is brittleness. → this is precisely where adaptivity / fallback helps.

So the answer to “should non-LLM nodes have an LLM fallback?” is: yes for action nodes, no for gates. The reflex to “just let a model handle it” is right for the commit and wrong for the coverage gate.

Approaches (for ACTION nodes), most-promising first

A. Adaptive action + deterministic postcondition (recommended where a clean postcondition exists). An agent achieves the goal (commit), then a cheap deterministic node verifies the postcondition (e.g. “exactly one commit on iterion/run/* whose tree excludes go/, target/, __pycache__”). You get adaptability AND safety — and the verify is mechanical (a property check, not a quality judgment), so it stays Goodhart-safe. This is the cleanest split: adaptive doing, deterministic checking.

B. Deterministic-first, LLM-recovery-on-failure — a DSL affordance like on_error: agent("commit the changed manifests/lockfiles/vendor, exclude caches"). The fast deterministic command stays the default; only on a hard exit does the runtime hand the agent {command, stderr, goal} + real tools to finish, then continue. 95% cheap/predictable path, 5% adaptive recovery instead of a block. Minimal, general, opt-in. (Generalizes the engine’s existing resumable-failure idea into an in-run recovery step.)

C. LLM-repairs-command, determinism-executes (self-healing, auditable middle ground). On failure an LLM proposes a corrected command (it sees the error); the runtime re-runs it deterministically, bounded attempts. Execution stays deterministic + visible in events (you see the fixed command); the model only fixes the recipe, never does the side effect blind. Good for “the command was almost right” cases (both Renovacy failures were exactly this).

D. Robust-primitive library (complements A–C). Most brittleness is ad-hoc shell: git pathspecs, JSON shape handling, buffer limits, glob portability (cf. the dash/bash pitfalls doc). Ship tested iterion primitives — e.g. an iterion __commit --exclude-caches subcommand, a json-shape normalizer, a __git-status-clean — that nodes call instead of fragile inline shell. Robust by construction; shrinks the surface A–C have to cover.

E. Per-node failure semantics — a node declares required | best_effort | recover. Today most hard-fail; the commit should recover (B/C), an optional enrichment should best_effort, a security gate stays required. Also subsumes the campaign’s “inconsistent rate-limit handling” finding (some nodes graceful-pause, others hard-fail — make it a declared policy).

Recommendation

1. Keep gates deterministic (the Goodhart firewall) — do not LLM-ify them.
2. For action nodes: prefer (A) where a postcondition is expressible (commit, file-producing steps); add (B)/(C) as the general opt-in recovery for the rest.
3. Build (D) primitives for the recurring brittle operations — commit with cache-exclusion is the obvious first one (it failed twice this session).
4. Prototype on the commit node (the proven pain point), measure, then generalize.

Consequences

Determinism is kept exactly where it guarantees trust (gates) and softened exactly where it only adds brittleness (actions). Bots get materially more reliable — fewer hard blocks on unanticipated inputs — without weakening the anti-façade guarantees. Cost: a recovery agent invocation on the rare failure path, and a new DSL surface (on_error: / failure-semantics) to design carefully so it can’t be abused to paper over a real gate failure (recovery is for actions, never gates).

Synthesis — the “Verified Action” pattern (mix, don’t choose)

The five approaches are not alternatives; they are layers that compose, and they sit at different levels:

• D (robust primitives) = the foundation (fewer edge cases become failures).
• A (deterministic postcondition) = the load-bearing spine (the truth oracle).
• C (self-repair) then B (agent recovery) = graduated recovery rungs.
• E (failure semantics) = the policy that bounds it.

The load-bearing insight: A is what makes B and C safe. On their own, an LLM recovery / self-repair reintroduces the façade risk (the model can claim it committed). With a cheap deterministic postcondition checked after every rung, adaptive recovery becomes trustworthy — the agent cannot fake success past a property check. So A is not “one option among five”; it is the enabler that lets you mix in the adaptive ones without losing the anti-Goodhart guarantee.

The unifying model

Every node becomes a quad: goal + recipe + postcondition + policy.

• goal — the outcome, in words (fuel for the agent rungs).
• recipe — the deterministic command (the fast path; what self-repair fixes).
• postcondition — a cheap deterministic property check (the single source of truth).
• policy — required | recover | best_effort.

Runtime escalation (cheapest → most adaptive, postcondition is truth at each rung):

1. Idempotent skip — postcondition already met? do nothing (resume-safe).
2. Recipe (robust primitive). Postcondition met → done. (the ~95% path)
3. Self-repair (C) — LLM proposes a corrected recipe from the error; re-run deterministically, bounded. Postcondition met → done. (auditable: the fixed command is in the events; no blind side effect)
4. Agent recovery (B) — agent achieves the goal with real tools, bounded. Postcondition met → done.
5. Policy (E) — still unmet: required→fail, best_effort→warn+continue.

Why this is the most robust + flexible + clean + universal

• Universal — the quad subsumes every node type as a degenerate case:
  • a gate = recipe == postcondition, no rungs 3–4, policy: required (pure determinism — the Goodhart firewall, unchanged);
  • a transform/compute = recipe + schema-as-postcondition + self-repair;
  • a side-effecting action (commit, file write, scanner) = full escalation;
  • even an LLM node = the agent is the recipe, schema is the postcondition — which is exactly what the review judges and sec-audit’s missing-field retry already do. One model for the whole graph.
• Robust — postcondition-as-truth, not exit-code-as-truth. Exit codes lie (nothing to commit exits 1 though the goal may be met; a command can exit 0 yet not achieve the outcome). Keying success on the postcondition + the idempotent skip makes retries/resumes naturally correct. Both Renovacy failures this session were “recipe almost right” → caught at rung 3 with zero human/code.
• Flexible — authors opt into exactly the rungs they want; a bot can ship recipe-only (today’s behaviour) and add a postcondition later, then recovery.
• Clean — one mental model (generate → verify) for the entire engine.

The deepest framing

iterion is already “adaptive generate + deterministic verify” — that is what a review loop is (agent generates, judges + gates verify). Action nodes are the lone exception: deterministic generate, no verify → brittle and unguarded. This pattern simply brings action nodes onto the same spine the rest of the engine already runs on. The synergy isn’t five features bolted together; it’s making one existing principle universal.

Recommended first cut: implement A+E as the spine (postcondition + policy on action nodes), add C (self-repair) as the cheap first recovery rung, B (agent) as the opt-in second rung, and grow D (primitives) for the operations that keep breaking (commit-with-cache-exclusion first). Prototype on the commit node.

This site is open source. Improve this page.