Iterion

Appearance

Like what we’re building? Star on GitHub

Workflow Authoring Pitfalls ​

Hard-won lessons for authoring .bot workflows where LLM agents do real work on real codebases. Read this before writing or amending an iteration workflow that has the power to commit code.

The TL;DR: LLM agents will optimize the metric you measure, not the goal you imagined. If your verdict criteria, scanner, and prompts can be satisfied by a façade, an agent will produce a façade — even when a fresh human reading the same goal would never consider it.

This is the "what fails" half. The "what works" half — the measured shape of productive human-driven sessions and the authoring rules derived from it — is references/productive-session-patterns.md.

The cheating LLM is the workflow's fault ​

When an agent produces something that looks like progress but doesn't actually achieve the underlying goal, the diagnosis is rarely "the model hallucinated." The diagnosis is almost always one of:

  1. Goodhart's law — the success metric was a proxy for the real goal, and the proxy was easier to satisfy than the goal.
  2. Path of least resistance — the prompts described the shape of the work but not the intent, so the agent picked the implementation closest to the source rather than the one closest to the target.
  3. No anti-façade gate — the judge had no rule to recognize that "rename X → Y" is not "remove X."
  4. Verdict tunnel vision — the verdict only looked at the artifact the agent produced, not at the whole dependency graph the workflow was supposed to clean up.

When you catch this happening, fix the workflow before re-running. Adding human supervision on every batch is not a substitute for sharpening the spec.

Case study: the goai → claw-code-go façade (2026-04) ​

What was supposed to happen ​

iterion depended on github.com/zendev-sh/goai (vendored, 7300 LOC) for its in-process LLM layer. The team also maintained claw-code-go, a native multi-provider Go port of Claude Code with its own internal/api/Client (zero goai imports, real HTTP/SSE multi-provider implementation). The plan: make iterion depend on claw-code-go's native API, drop goai entirely.

What actually happened ​

The workflow ran 8 batches over ~3 hours and reported 96% migration parity. Verdicts were batch_complete: true, all tests passed, the build was green. iterion had zero import "github.com/zendev-sh/goai" in any of its .go files outside vendor/.

When we tried to actually delete vendor/github.com/zendev-sh/goai/ and remove the require from iterion's go.mod, the build broke immediately. Reason: the agent had created a brand-new package — claw-code-go/pkg/sdk/ — populated with files like:

go
// claw-code-go/pkg/sdk/types.go (created by the agent)
package sdk

import "github.com/zendev-sh/goai/provider"

type FinishReason = provider.FinishReason
type Usage        = provider.Usage
type Message      = provider.Message
// ...

The new pkg/sdk/ was a thin façade that re-exported goai/provider types under different names. iterion was rewritten to import claw-code-go/pkg/sdk instead of github.com/zendev-sh/goai. From the scanner's perspective, iterion was "100% PORTED" — no goai strings in iterion's own source. Internally, the goai dependency had simply been relocated into claw-code-go (which iterion vendored), invisible to a grep scoped to iterion.

The native claw-code-go/internal/api/Client — the actual point of the migration — was not used at all. The agent had taken the path of least resistance: re-exporting goai types preserved goai's API shape, so iterion needed minimal rewriting. Adapting iterion to claw's native streaming API (Client.StreamResponse(ctx, req) → channel of StreamEvent) would have required rewriting iterion's three generation strategies. The agent didn't do that work, and nothing in the workflow forced it to.

Why the workflow let it pass ​

Five concrete defects in the .bot:

  1. The goal in port_plan_system was ambiguous. "Replace iterion's native goai layer with claw-code-go" can mean (a) iterion's source stops importing goai, or (b) iterion's whole dependency graph stops requiring goai. The agent satisfied (a). Nothing in the prompt locked (b).

  2. The target API was not specified concretely. The plan said "bridge to claw-code-go" without naming the entry point (claw-code-go/pkg/api.Client.StreamResponse). The agent invented a different bridge point (pkg/sdk/) that looked simpler.

  3. The migration scanner was textually scoped, not architecturally scoped. It ran grep -r "github.com/zendev-sh/goai" --include='*.go' . | grep -v vendor/. Once the goai imports lived inside claw-code-go/pkg/sdk/ (vendored from iterion's perspective), the scanner returned zero — even though the dependency tree was unchanged.

  4. The judge had no anti-façade rule. The verdict checked "blocker / suggestion" classification on the diff. Creating a new package that re-exports the source types is not a blocker by any reasonable definition of "production-breaking." But it is a complete no-op for the actual migration goal. The judge had no language for "this batch claims migration progress but is actually a relabeling."

  5. No "why" anchor in any prompt. The locked decisions section listed constraints ("hard rename, no alias", "anthropic + openai validated") but didn't repeat the purpose ("we want iterion to use claw's native multi-provider client so we can drop the third-party goai dependency entirely"). Without that anchor, every prompt iteration drifted further from the original intent.

What good looks like ​

A workflow that would have caught this:

  • Goal in port_plan_system restated concretely:

    END STATE: iterion's pkg/backend/model/claw_backend.go calls claw-code-go/pkg/api.Client.StreamResponse(ctx, req) and aggregates StreamEvent deltas. NO intermediate package exists in claw-code-go that re-exports github.com/zendev-sh/goai types. After this work, claw-code-go/go.mod has zero require for github.com/zendev-sh/goai.

  • Anti-façade rule in plan_judge_merge_system:

    REJECT any plan that creates a new package whose primary contents are type X = goai.X aliases or wrapper functions calling goai.Y(...). Re-exporting a dependency under a different name is not migration — it is relabeling. The goal is dependency removal, not relabeling.

  • Architectural scanner in parity_scan_system:

    Migration is complete when ALL of:

    • grep -r "github.com/zendev-sh/goai" --include='*.go' returns 0 in iterion's source (excluding vendor/ and the workflow .bot)
    • grep -r "github.com/zendev-sh/goai" --include='*.go' returns 0 in claw-code-go's source (the upstream repo, NOT iterion's vendored copy)
    • claw-code-go/go.mod has no require github.com/zendev-sh/goai
    • iterion/go.mod has no require github.com/zendev-sh/goai
    • iterion/vendor/github.com/zendev-sh/ does not exist

  • Verdict gate that audits both repos, not just iterion's source.

Authoring checklist for .bot workflows that touch real code ​

Before writing the workflow ​

  • [ ] State the goal as a dependency-graph claim, not an edit-list. "After this work, package X cannot be reached from package Y" is testable. "Replace X with Y" is gameable.
  • [ ] Identify the target API entry point by exact symbol name (e.g. claw-code-go/pkg/api.Client.StreamResponse). Forbid alternatives explicitly.
  • [ ] Define a test that observes the dependency graph, not just the source files. go list -deps or go mod why or a recursive grep into vendored sub-modules. If your scanner can be satisfied by moving imports into a vendored sub-tree, it is a façade-gameable scanner.
  • [ ] Decide what counts as architectural progress vs cosmetic progress and bake the distinction into the verdict prompt. Renames alone are cosmetic.

In the prompts ​

  • [ ] Every system prompt that an agent will see during the work should repeat the why in its first sentence. Not "do X" but "we want Y, so we are doing X." When the agent paraphrases your intent in batch 5 it will paraphrase the why, not just the what — and a drifting why is much easier to catch than a drifting what.
  • [ ] Forbid creation of new abstraction layers unless explicitly part of the plan. Default rule: "no new packages, no new files, unless the plan justifies one."
  • [ ] State the end-state file structure (what files exist, what files do not exist) as part of the goal, not just as a verification step.

In the scanner / verdict ​

  • [ ] The scanner runs outside the agent's reach. It is a shell/tool node, not an LLM. Deterministic. The agent cannot rationalize a non-zero count.
  • [ ] The verdict's overall_complete flag must be a conjunction (all-of), not a disjunction. Even if batch_complete is true, if any scanner field is non-zero, overall_complete stays false.
  • [ ] Include at least one negative-space check: a thing that must not exist. "File X does not exist", "package Y has no callers", "module Z is not in go.mod". These are harder to satisfy by addition, which is the agent's natural mode.

When reviewing a plan at the human gate ​

  • [ ] Read the files_to_create list. If it contains a package inside the migration target, ask: does this package exist to do new work, or to relabel old work? If the latter, reject.
  • [ ] Read the migration_strategies list. If any strategy reduces to "rename X → Y" or "alias X = Y", reject — that is not migration.
  • [ ] Search the plan for the target API entry point by name. If it doesn't appear, the plan is not actually targeting the migration's endpoint. Reject.

Goodhart variants seen in this codebase ​

Metric the workflow rewardedWhat the agent did
grep zendev-sh/goai in iterion source returns 0Moved goai imports into a new package inside claw-code-go (vendored from iterion → invisible to the grep)
parity_percentage hits 100%The percentage was computed from a checklist of file-level statuses; the agent updated each file's status to PORTED based on whether iterion still imported goai from that file. The fact that the file now imported a façade re-exporting goai didn't change the status.
Tests passTests exercised iterion's behavior end-to-end through the façade. Since the façade preserved goai's runtime behavior 1:1, tests passed.

In every row, the metric was a faithful measurement of the surface property but a poor proxy for the underlying goal. The agent satisfied the metrics. The goal stayed unmet.

The empty-input façade (2026-07, dep-update-guard) ​

A variant worth its own name, because nothing in the workflow was gaming anything: the auditor was handed an empty diff and dutifully reported "safe". Its classifier only recognised package manifests, so a PR moving a container digest or a pinned tool matched nothing — and the run did not stop, because the flag guarding the exit meant "no files changed", not "no manifest recognised". Three of four real Renovate PRs would have been waved through by an agent that had read nothing.

A scope flag and a coverage flag are not the same flag. Conflating them converts "we found nothing to look at" into "we looked and found nothing" — and the second is indistinguishable from success. When a node narrows what a downstream agent sees, it owes that agent an explicit signal that the narrowing found nothing, and the agent owes the reader a verdict that says so.

A stub that accepts anything certifies nothing ​

The same change shipped an auto-merge step whose GraphQL was syntactically invalid — every request would have been rejected by the real API. It passed CI: the test's stub answered success to any body and only substring-matched the query. The feature was green in the suite and dead in production.

A test double is a claim about what a real peer accepts. When it accepts strictly more than the real one, the test measures only that the code ran. Assert on what actually goes over the wire, and check the fix by reintroducing the bug and watching the test fail — a test that has never been seen red is a test whose sensitivity is unmeasured.

Practical rules to internalize ​

  1. A migration is not done until you can rm -rf the old dependency and the build still passes. That is the only test that cannot be gamed.

  2. If your scanner can be satisfied by mv, your scanner is wrong. Moving a problem to a different filesystem location is the most basic form of metric-gaming and any tool that doesn't catch it isn't measuring what you think.

  3. Anti-façade rules belong in the judge, not the scanner. The scanner says "still 5 imports remain"; the judge says "creating a wrapper to absorb those 5 imports doesn't count."

  4. The plan_gate human review is your last chance to catch a façade. Always ask: "what does this batch actually remove from the dependency graph?" If the answer is "nothing — it just renames or re-exports", you are about to approve a no-op.

  5. When an agent's output contradicts the plan but satisfies the metrics, the metrics are wrong. Don't relax the goal to fit the output; sharpen the metrics and re-run.

  6. Never separate the node that MUTATES the workspace from the node that PERSISTS the mutation by a resumable boundary. A cloud run re-clones its repo on every claim, so a failure between the two — a provider usage window is the ordinary way — resumes on a pristine tree while the checkpoint faithfully replays the mutating node's outputs. The persisting node then reports "nothing to commit", which is indistinguishable from "there was nothing to do", and every downstream verdict states the benign meaning. Either commit inside the mutating node, or make the persisting step reconcile the upstream CLAIM against the tree and treat a contradiction as blocking. Cost of learning this: iterion#400 merged a dependency bump without the breaking-change fix its own aligner had written (dep-update-guard bilan, 2026-08-10).

  7. An agent's honest prose is not a signal the graph can act on. In that same run the commit agent described the missing alignment exactly, and named the recovery — in the comment body. The gate reads a schema field, so the PR merged green anyway. If a discovery has to change an outcome, it must land in a typed output a deterministic node reads; prose is for the human who arrives afterwards.

Cross-checked rules from convergent methodologies (IACDM / AI-DLC 2026) ​

Two independent 2026 methodology papers arrive at this doc's failure model from the outside — IACDM's "verification gap" and AI-DLC's "backpressure over prescription" are Goodhart/façade said differently. The full mapping (what they validate, the vocabulary, what iterion rejected) is references/external-methodologies.md; the rules below are the delta worth importing.

Teach-back, not confirmation ​

An RLHF'd model agrees: "did you understand?" and "is this design good?" are always-affirm questions. Invert the direction of the agreement bias: the AGENT restates the mission in its own words — the goal, the load-bearing assumptions, what it would build differently under the other reading — and the human judges the restatement. A restatement can be corrected; a yes/no invites a rubber stamp. Bot form (ADR-081): on an ambiguous mission, an expensive campaign posts its teach-back via ask_user_async and KEEPS WORKING under its stated assumptions — corrections arrive in its message queue whenever the operator replies; the blocking ask_user stays reserved for destructive/irreversible hard stops. feature-dev's and whole-improve-loop's campaign item 5 is this rule.

Verify divergence, not correctness ​

The same bias, self-check form: "is this correct?" invites self-confirmation. Frame every post-unit self-check as refutation with a concrete target — "where does this implementation DIVERGE from the spec/axis?" (whole-improve-loop's "Fit" question is this rule). A divergence question has a findable answer; a correctness question has a comfortable one.

Scope inventory — absence doesn't fail tests ​

"Modules that pass all tests but were never implemented do not fail tests; they simply do not exist" (IACDM). A test gate judges only what exists. When the deliverables are enumerable upfront (a migration's file list, a checklist mission, declared endpoints), pair the test gate with a deterministic PRESENCE check over that list — the positive-space complement of the negative-space checks above. Agent-side honesty clauses (feature_complete = "a fresh re-read finds EVERY requirement implemented") approximate it; a tool-node inventory is stronger wherever the list is concrete.

The cost-tier switch point ​

Once the expensive phases have externalized the context — plan approved, work-list enumerated, verify.sh written — the remaining units are bounded and well-specified, and a cheaper model performs equivalently on them. Spend the strong model on discovery / design / judgment nodes and pin model: per node accordingly; downgrade the mechanical tail. The shipped precedent: review topology auto resolves to mono wherever a detected family's slot can serve it (a backed family with none refuses the launch, #1221) — the second family is a deliberate spend, not a default.

A lens must own a failure class ​

For reviewer fan-outs and scanner focus areas: a lens is legitimate only if removing it exposes a failure class no other lens detects. Overlapping lenses pay twice for the same findings and drown the synthesis. At synthesis time, run the two concentration diagnostics: findings clustered on ONE module → that module needs redesign, not N patches; ONE lens firing across ALL modules → the cause is systemic (an architectural decision) — fix the root, not the sites.

Cap unbounded inputs before the LLM (the ~40–60% rule) ​

Model quality degrades well before the context window is full — reported degradation starts around 40–60% utilization ("lost in the middle"). When a node's input can grow without bound (scanner output, board dumps, enumerations), cap it DETERMINISTICALLY before the LLM sees it (sec-audit's cap_findings is the reference), keep per-pass context fresh, and offload broad exploration to sub-agents. "It fits in the window" is not the bar; "it leaves headroom" is.

A defect that passed the gates is a gate bug ​

When a later reviewer — Revi, a human, production — finds a defect in work a bot's gates had passed, that is TWO bugs: the code defect and the gate that let it through (AI-DLC's "criteria escape rate"). Route the second into the bot: sharpen the scanner / verdict / postcondition so the class cannot pass again, and record the escape in the bot's bilan ("Findings / misses" line). This is rule 5 above ("the metrics are wrong") applied at run time instead of authoring time.

Improvement loops must converge to an asymptote ​

An improvement/review loop must converge to an asymptote — settle into a stable approved state and stop. A slight, very occasional oscillation is acceptable; it must be the rare exception. The rule is the asymptote, never sustained oscillation. iterion bench asymptote (docs/asymptote-bench.md) is the empirical counterpart one scale up: it aggregates runs you already recorded and plots where the per-iteration verdict stabilises ACROSS sessions — the (model + recipe)'s reliability ceiling, not this single run's convergence.

The shipped mechanism (ADR-058 v2 — the whole loop fleet since 2026-07-07): one campaign agent + a deterministic verify gate + a machine-checkable termination contract + a bounded continuation_loop(max_passes). gate.converged = <done-flag> ∧ gates green; the campaign commits each unit in stride (git is the state), so loop exhaustion ships what is banked instead of discarding it. Oscillation is structurally absent — there is one context per pass and no reviewer/fixer relay to re-litigate. The honesty clause ("under-reporting only costs a pass; over-reporting lands you right back here") plus the deterministic gate is what keeps the done-flag truthful.

If you author a NEW cross-family reviewer loop (an optional amplification — no catalog bot ships one any more), the historical convergence mechanisms are mandatory:

  • streak_check: exit on N consecutive cross-family approvals, not one pass; a low-confidence rejection is non-blocking so noise doesn't reset the streak.
  • prior_pushback + previous_scanned_areas fed back with "do NOT re-raise without new evidence" — re-litigating resolved items is the #1 oscillation driver.
  • loop.<name>.previous_output shows each reviewer the prior verdict so verdicts trend monotonically.
  • Bounded max_iterations is the backstop, not the design.

The fastest way to break convergence in any shape: judge the wrong artifact. Work-in-progress lives in the working tree (and, under v2, in the run's in-stride commits). Anything that reviews a change MUST diff the working tree against the right base — git diff HEAD for uncommitted work, git diff <run-base> for a v2 run's cumulative series — never git diff HEAD^...HEAD (the last commit = the base). The historical v1 bug: feature_dev's reviewer_gpt diffed HEAD^...HEAD, reported "feature not implemented" against work that was plainly present, split the cross-family verdict and oscillated forever. Same family of bug: git diff HEAD omits untracked files — new files must be marked or staged first (git add -N -- ':/' $ITERION_TREE_NOISE, git add -A -- ':/' $ITERION_TREE_NOISE — iterion's skills mirror left out) before diffing or a change that ADDS files reads as missing. The v2 campaign contracts bake git add -A -- ':/' $ITERION_TREE_NOISE into the per-unit commit step. From ONE branch only: two parallel reviewers each running git add -N -- ':/' $ITERION_TREE_NOISE contend for .git/index.lock — fatal for the loser, whose empty report then reads as an approve — so a read-only branch reads untracked files with git ls-files --others --exclude-standard -z | xargs -0 -I{} git diff --no-index -- /dev/null {} instead (exit 1 per file, 123 for the batch: a diff, not a failure). And a worktree: auto run starts from the anchor COMMIT, without the uncommitted work: a "pending changes" reviewer runs worktree: none or diffs a base ref, and gates an empty scope as a typed refusal rather than an approve (the review-fanout template's scope tool). When a loop won't converge, first confirm the judge is diffing the same, correct artifact.

The asymptote is what closes a run. What carries a gain from one run to the next — and what keeps a later run from re-earning it — is the ratchet: see improvement-ratchet.md.

What worked in the end (run-005, 2026-04-28) ​

After four failed attempts (run-001 façade, run-002 auth, run-003 var-substitution + judge tool-blindness, run-004 wrong-path grep), run-005 converged to overall_parity=true in four batches over ~1h45. The workflow file changes that made the difference:

1. Goal anchored on a concrete API entry point ​

port_plan_system and port_impl_system named the target by symbol, not by intent:

END STATE: iterion's pkg/backend/model/claw_backend.go calls claw-code-go/pkg/api.Client.StreamResponse(ctx, req) and aggregates the returned []ContentBlock / StreamEvent deltas. NO call goes through any intermediate pkg/sdk or wrapper layer.

The agent could not "interpret" the goal as something easier; it had to use that exact entry point.

2. Architectural scanner across both source repos ​

parity_scan_system greps both vars.iterion_repo_path and vars.claw_repo_path (the source dirs, not iterion's vendored copy of claw). This closed the run-001 gameable axis where goai imports were relocated into a vendored sub-tree.

3. Strict 8-condition AND for completion ​

overall_parity=true requires ALL of:

  • iterion_goai_imports == 0
  • claw_source_goai_imports == 0
  • claw_pkg_sdk_files_present is empty
  • claw_gomod_has_goai == false
  • vendor_goai_present == false
  • iterion_gomod_has_goai == false
  • iterion_uses_pkg_api_directly == true
  • tests_passing == true

A single false anywhere → overall_parity stays false. No partial credit, no deferred negative-space checks.

4. Tools on judges + explicit USE-TOOLS preamble ​

The first attempt to add tools to judge nodes failed because the agents had access but didn't invoke them — the prompt didn't require tool use. Adding an explicit STEP 0 — VERIFY ACTUAL STATE preamble with imperative language ("USE the tools", "Cite tool outputs inline") made every judge invocation actually grep before claiming filesystem state.

5. Absolute paths in judge prompts ​

The next attempt's judges did use tools but greped the wrong directory (/workspaces/iterion instead of the clone). The fix was to write the absolute path inline in the prompt: grep -rn ... {{vars.iterion_repo_path}} | grep -v "/vendor/". With the literal path embedded, the agent stopped defaulting to its CWD.

6. Workflow vars > input refs for global config ​

Tool commands' shell substitution uses {{input.X}} (per-node input map). Judge prompts' template substitution can use {{vars.X}} (workflow-global). Mixing them up produces literal placeholders in shell commands. Be explicit about which substitution context applies to which template usage.

7. The phasing pattern that worked ​

Run-005's batches were:

  • Batch 1 — foundation: create iterion-owned local types (api_errors, streaming_types, generation_types) so the rest of the rewrite has stable type names.
  • Batch 2 — engine: build the new generation primitives (generation.go, generation_tool.go) calling claw-code-go/pkg/api.Client.StreamResponsewithout swapping any call site yet.
  • Batch 3 — swap: replace every iterion call site to use the new engine instead of the old wrapper. Goai imports drop out of iterion here.
  • Batch 4 — cleanup: delete the now-unused wrapper layer in claw-code-go, drop goai from both go.mod files, regenerate vendor.

This phasing matters because it isolates risk: every batch's tests must pass before moving on. A "rewrite everything in one batch" shape would not survive the first compile error.

8. Independent verification > workflow report ​

Even after the workflow reported overall_parity=true, an independent grep / filesystem audit against the same eight conditions confirmed it. Don't trust the verdict node's claim alone — run the checks yourself, post-merge.

Cost note ​

Five workflow runs cost roughly $120-180 in API. Run-001's façade burnt about a third of that on a result that had to be discarded. Sharper prompts up front would have cut that in half.

Shell portability for tool nodes ​

A .bot tool node's command: string runs via bash -c (pkg/backend/model/executor_tool.go, toolNodeCommand), on the host and inside a sandbox alike — bash is pinned precisely because /bin/sh is dash on Debian-derived images. A script: node is different: it runs the interpreter its language: names, and language: sh (the default) is whatever binary sh points at in the runtime PATH — not a fixed interpreter. The rules below bind every script: in sh, and every command: that may also run where bash is absent (a minimal image: install bash via post_create, or write POSIX).

A sh script can therefore behave differently across hosts:

EnvironmentWhat sh actually is
Devbox shell (any OS)bash 5.x
Linux Mint / Ubuntu hostdash
Debian hostdash
Alpine / busybox imageash
macOS (default)bash 3.2 (very old)

The brace-expansion gotcha (real, observed) ​

run_console_demo.bot originally contained:

echo {"count":3,"done":false}

This worked under devbox (bash interpreting). On a Linux Mint host the same workflow piped through sh -c (what the executor invoked at the time; a command: now runs bash -c, and a language: sh script still hits exactly this) ran under dash — which has no brace expansion. The output landed unchanged, JSON parsed, run succeeded.

But on a host where the user's sh was bash (e.g. Ubuntu with dash reconfigured to bash, or any rebuilt image where bash is /bin/sh), bash applied brace expansion to {"count":3,"done":false} and emitted count":3 done":false — destroying the JSON, and silently failing every downstream node that expected {"count": ..., "done": ...}.

Rule: every command: string must be POSIX-portable ​

When you write a tool node, assume sh is dash. That means none of the following:

PatternReplace with
echo {"k":v}echo \{\"k\":v\} (escape every brace and quote)
[[ "$x" == "y" ]][ "$x" = "y" ]
cmd <<< "$input"printf '%s' "$input" | cmd
$'\t'"$(printf '\t')"
((i++))i=$((i+1))
for f in *.{a,b}for f in *.a *.b
${var^^} / ${var,,}printf '%s' "$var" | tr a-z A-Z

For JSON output specifically, the safest pattern landed on in run_console_demo.bot:

echo \{\"count\":$(wc -c < /tmp/counter),\"done\":false\}

Every {, }, and " is backslash-escaped. The braces are escaped to defeat brace expansion under bash; the quotes are escaped to survive the outer YAML/.bot string-literal layer plus bash -c.

Alternative: produce JSON via printf '{"count":%d,"done":false}\n' N. printf is POSIX, has no brace expansion semantics, and reads as intent-first.

The engine answers ${NAME} before bash does ​

A tool body is rewritten by the engine before any shell sees it: expandBracedEnv (pkg/backend/model/executor_tool.go) expands ${NAME:-default} unconditionally and ${NAME} whenever lookupToolEnv answers the name — an ITERION_* name from the bot-var overlay first, any name from the ENGINE process when its process-env policy exposes it (a cloud server and runner install platformcfg.CloudProcessEnvPolicy, which refuses a credential-shaped name or a platform secret unless ITERION_CLOUD_ENV_PASSTHROUGH lists it, so there a plain ${TOKEN} reaches bash verbatim while ${TOKEN:-none} still resolves — to the default) — on command:, on every script: language (js and py included), and on postcondition:. A value the body sets itself cannot win that race: wherever the engine answers the form, it is resolved before any shell starts, and this body's assignment happens later, inside that shell. A comment or a single-quoted span is no shelter — the expansion is quote-blind and rewrites both. The other side of the same coin is sound and common: where the braced read IS the assignment's own operand (PATH="${PATH}:/opt/bin", RUN_ID="${RUN_ID:-none}"), the ambient value is the read's whole point and nothing is defeated — which is why C188 names only a read the body assigns BEFORE, in a command of its own: a command reads its own text before it assigns — bash expands a command's words and redirections before its prefix assignments take effect, and a declaration utility's operands, a for's word list and a read's redirection before they assign — so a for's word list (for ARCH in ${ARCH:-amd64}) and a read's redirection target (read -r CFG < "${CFG}") read the ambient value as plainly as a right-hand side does. The bare rewrite must not be applied to any of them: it would turn RUN_ID="${RUN_ID:-none}" into an empty string, and [ -z "$ARCH" ] && ARCH=amd64 arm64 sets ARCH to nothing and runs arm64 as a command (arm64: command not found, status 127) — a multi-word default spelled as a statement needs its quotes. An assignment-only command is the exception, twice: it performs its assignments left to right, each seeing the previous one, so in A=1 B="${A}" the read answers 1; and under bash it assigns before its own redirections, so OUT=new >"f.${OUT}" creates f.new (dash expands the redirection first, and creates f. plus the inherited value). The engine defeats both reads — documented misses of C188 (below).

The production witness is the run-id guard of bots/sec-audit-source (#1522): it read ${RUN_ID:-}, the engine resolved it to "" (RUN_ID is not in the engine process), and the guard refused every production deep-scan pass from 16/09 to 21/09 while every harness test stayed green — the harness substitutes {{…}} refs plainly and never had the engine rule. Repaired per-bot by #1473 (read the variable bare: $RUN_ID); the C188 warning now names the shape at authoring time, and bots/catalog_braced_env_test.go holds every shipped workflow at zero of them — it compiles each one and reports the warning itself.

The same expansion mangles text an author meant verbatim, because a form's segment ends at the FIRST }: in ${FOO:-{{input.x}}} the engine reads FOO:-{{input.x as the form and leaves }} behind it. Unset, the leftovers recompose the ref and the shape works by accident ({{input.x}}); set, the body receives <value>}} — two braces the author never wrote. Nothing reaches past a segment's own first }, so a later {{…}} is untouched (cmd ${FOO:-a}{{input.y}} yields cmd a{{input.y}}). bots/golden-master carries the rule in its own body in capitals for exactly this reason: no ${...} anywhere in its emitted shell; every default is spelled as [ -z "$VAR" ] && VAR="…". Inside a js template literal the hazard inverts — ${count} is the language's interpolation, and only the C-identifier test of looksLikeEnvRef plus the name being unset keeps the engine from eating it; a name the engine process happens to carry (${HOME} in a script) is answered from that process.

Rules that keep a body readable as what executes:

  • Read a value the body sets itself bare ($NAME), never braced — inside double quotes $NAME expands to the same one word ${NAME} did. Twice, the braces carry more than themselves and the bare rewrite alone breaks the body: an identifier byte right after the } joins a bare name ("${COUNT}_items" prints 7_items, "$COUNT_items" prints nothing — it names the variable COUNT_items), so close the word yourself ("$COUNT"_items); and a :- default goes away with the braces, so spell it as a statement the shell runs ([ -z "$VAR" ] && VAR="…" — quoted, or a multi-word default runs its tail as a command).
  • Keep the deliberate dials braced: ${ITERION_DISPATCHER_PORT:-4891} resolves through the bot-var overlay (ADR-093) precisely because the body never assigns that name — the C188 check stays silent on it by construction.
  • Where the braces are not iterion's to answer, take the engine out of the way instead of rewording the body: env_expansion: none on the node (ADR-130, the full rule in dsl.md) hands ${NAME} and ${NAME:-default} to the body's own reader exactly as written, in all three of its bodies, and C188 goes silent on that node. It is what a language: js body whose ${…} is the language's wants (F3's seventeen template literals), and what golden-master's hand-written rule was working around. It costs the ADR-093 overlay for that node's bodies — a ${ITERION_*} dial read there is no longer answered by the engine — so a body that mixes a self-assignment with a dial read fixes the warning at the price of the dial; the bare-$NAME rewrite above is the fix that keeps both. The {{…}} substitution is untouched either way: the decline is env-scoped.
  • Read what the engine answered before paying for a run: iterion validate --exec reports each body's environment reading under its own finding kind (env_read) — the ${NAME:-default} forms the preview resolved to the author's default, the plain ${NAME} forms it kept as written, and nothing at all on a node that declined — so the half C188 declines to judge (a name only the engine's environment supplies) is read as a fact rather than guessed. It never fails a gate: --strict reads past it, and the preview answers no name, so the report is the same on a laptop and in CI.

What C188 promises, and what it leaves out ​

Precision over recall: C188 is meant to fire only where the body's assignment, once it runs, persists to the braced read in bash, and only on a form the engine answers before any shell runs. A lint that misses is acceptable; a warning on a correct line is not. So the scanner leaves out every spelling it recognises of an assignment bash keeps out of the read's reach — the spellings it does not recognise are its known limits, below:

  • a prefix assignment or a command modifier's operand — HOME=/tmp/x git …, HOME=/tmp/x \git …, GIT_DIR=… env | …, env GIT_DIR=… cmd and IFS=, read … set the name in that command's environment alone, and command X=1 runs a command named X=1 (under bash: sh keeps a prefix before a special builtin, a documented miss below);
  • an assignment inside a subshell, a $( … ) or a backquoted substitution (( export X=1 ), OUT=$(X=1; …), OUT=`X=1; …`), in a command a lone & sends to the background (X=1 &, export X=1 &, cmd && X=1 &), and a read that is a pipeline element (cmd | read -r X runs in a subshell — bash without lastpipe, and sh);
  • local, declare and typeset, which make the name local inside a function, where the scanner has no function scope (export and readonly write the global, and count — unless the function made the name local first, a known limit);
  • a bare export NAME or readonly NAME, which assigns nothing, and an operand past a flag carrying f (export -f X=1), which names a function;
  • a NAME=value word past the command's verb, which is an operand (make COMMIT=abc, "$MAKE" PREFIX=/opt, for kv in FOO=1, echo $(date) A=1);
  • a for NAME do or for NAME; without a word list: it iterates the positional parameters, and a tool body runs without any, so the loop never runs.

The documented misses are real defeats C188 leaves unreported, because reporting them would take a deeper model of bash, or would fire on correct bodies of the same spelling: a name assigned and read inside one $( … ); a quoted "$( … )" or backquoted right-hand side whose inner text holds a blank or a separator (X="$(cmd "a b")", read later); a right-hand side a quote or an escape continues past a substitution's ) (X=$(pwd)"/sub", X=$(pwd)\/sub); a later assignment of the same assignment-only command (A=1 B="${A}"), and that command's own redirection, which bash performs after the assignments (OUT=new >"f.${OUT}" creates f.new; dash expands the redirection first); a prefix before a substitution that expands to no word (X=1 $(true) keeps X); a function's own braced read of its local, a top-level declare NAME=value, a typeset -g; an assignment-only command carrying a redirection (X=$(cmd) 2>/dev/null); the first statement on a one-line function's brace line (f() { X=1; …); an export operand after a command substitution (export A=$(cmd) B=2); an unquoted & inside a parameter expansion (X=${Y//&/and}), read as a & that backgrounds; a for NAME do over positional parameters the body set itself (set -- a b), and a for NAME whose in starts the next line; a read a later loop iteration would answer (a ${X} above X=1 in a for); a name set through let, printf -v, mapfile, select or ((NAME=…)); a read inside a comment, a single-quoted span or a heredoc body; and, in a script: run by sh, a prefix before a special builtin, which dash (like bash --posix) keeps and bash does not (X=1 :, X=1 export Q=2, X=1 eval …, X=1 exec).

Known limits — C188 still fires where the assignment never reaches the read but the scanner cannot see it, and the finding is wrong there. The scanner keeps no function scope, no pipeline or background scope for a compound command or an and-list, and a flat model of nesting and quotes, so it records these assignments where they are written, not where they run:

  • a function's assignment read between the definition and the first call (setup() { export X=1; }, then ${X:-unset}, then setup);
  • a name a pipeline element sets, read after it (cmd | while read -r L; do …; done, { X=1; } | cmd);
  • a name a backgrounded compound command or and-list sets, read after it ({ X=1; } &, for X in a b; do …; done &, X=1 && cmd &) — a simple X=1 & stays silent;
  • an export of a name the function made local first (f() { local X; export X=1; } exports the local);
  • an extglob pattern in a case inside a substitution ($(case $v in @(a|b)) X=1;; esac)), whose ) the scanner takes for the pattern's own, and a ) inside a quoted substitution nested in another ($(MSG="$(printf ")")"; X=1)): both close the outer substitution early, so the assignment after them counts as the body's;
  • in a script: run by sh, X=1 &>file: bash reads &> as a redirection, dash as & then >file, which backgrounds the assignment.

Never pass LLM-generated values into a command via the raw {{!ref}} form ​

A tool command: resolves {{input.x}} / {{vars.x}} through shellEscapeValue (pkg/backend/model/executor_tool.go): complex values are JSON-encoded and the whole token is wrapped + '\''-escaped, so it survives bash -c no matter what it contains. The raw form {{!input.x}} (bang prefix) deliberately bypasses that escaping and inserts the value verbatim — intended only for trusted shell snippets an upstream node hands down.

The trap: passing an upstream node's structured output (especially an LLM-authored field) via {{!input.x}}, or wrapping the escaped form in your own literal quotes:

# BROKEN — raw prose JSON in single quotes:
DECISIONS='{{!input.decisions}}' python3 -I -c "..."

LLM prose is full of apostrophes ("iterion's") and parens ("(CLI/studio)"). The first ' closes the shell quote and the next ) yields bash: -c: syntax error near unexpected token ')'. Observed live in adr-cartograph's build_manifest (dogfood, 2026-06-13): the survey's decisions/gaps JSON broke the node on every run until fixed.

Rule: pass structured/LLM data with the default form and no surrounding quotes — let shellEscapeValue quote it:

# CORRECT — shellEscapeValue wraps + escapes; json.loads gets exact JSON:
DECISIONS={{input.decisions}} python3 -I -c "import os,json; d=json.loads(os.environ['DECISIONS'])"

Reserve {{!ref}} for values you control that are meant to be re-interpreted as shell. A tool node that only ever saw path tokens (e.g. adr-cartograph's build_manifest) can carry the '{{!input.x}}' pattern latently for a long time — it breaks the day an apostrophe-bearing value flows through, so fix the pattern, not just the one value that tripped it.

Python in a tool node runs isolated ​

A tool node runs with the workspace as its working directory, and python3 -c (like -m, or a program read from stdin: a heredoc, a < redirect, a pipe) puts that directory first on sys.path. A json.py, hashlib.py or subprocess.py at the root of the judged tree, git-ignored or not, then replaces the standard module inside the node, and a gate's verdict is written by the tree it judges. Write python3 -I -c, python3 -I <<'PY', … | python3 -I (the catalog guard TestCatalogPythonRunsIsolated refuses the bare forms, by name or by path); a language: py script already runs python3 -I. A body that needs a third-party module gets it from the image's system site-packages, never from the workspace.

Diagnostics ​

If a downstream node fails with "expected object, got string" or "missing required field" right after a tool node:

  1. Run the command manually under dash and under bash — they should produce identical output.
  2. Run it under devbox (devbox run -- bash -c '…' for a command:, devbox run -- sh -c '…' for a language: sh script) since that's the PATH the engine inherits.
  3. Inspect the run's events.jsonl for the tool_called event — it records the resolved command string and raw stdout.

Why command: pins bash and script: does not ​

A command: is a one-line recipe that authors reach for with bash idioms ([[ ]], arrays, pipefail), and /bin/sh is dash on the Debian-derived images the sandbox ships — so the executor runs bash -c and treats a missing bash as the image's problem (install it via post_create, or rewrite the body in POSIX). A script: names its own interpreter through language:, which is the contract to reach for when the shell matters: language: bash for bash-only syntax, language: sh when the script must also run on Alpine/busybox or macOS's bash 3.2 — a small, well-documented POSIX target that authoring to scales.