iterion

Willy — whole-improve-loop run bilans

Whole-repository alternating Claude/GPT review-fix loop. Reviews the workspace in per-package chunks, fixes blockers in place, converges on two consecutive cross-family approvals. See bots/whole-improve-loop/.

Convergence machinery (alt round-robin → reviewer_*streak_checkfix_*) is shared with Billy (branch-improve-loop), whose full cross-family convergence to an asymptote is validated in branch-improve-loop.md. This page covers Willy’s whole-repo specifics.

2026-07-07 — v2 on a RELIABILITY axis: converges (not capped) on 9 hang/leak fixes at the I/O boundary (run 019f3afc)

2026-07-03 — v2 on an AMBITIOUS axis: agent self-plans a 7-commit dedup refactor, build+test green, converges (run 019f28ae)

2026-07-03 — v2 MINIMAL-FRAMING (ADR-058) live-proven: one agent, its natural flow, clean in-stride commits (run 019f288b)

2026-07-03 — LIVE PROOF of the ADR-057 axis-sweep: enumerates + transforms + commits a real cross-cutting refactor across 4 packages, converges (run 019f27d4)

2026-07-03 — LIVE PROOF of the ADR-055 redesign: converges + lands a real fix + terminates (runs 019f2750, 019f275e)

2026-07-03 — whole-repo production-readiness, mono sonnet-5: real hardening but did NOT converge → harvested + triggered ADR-055 redesign (run 019f2247)

2026-07-02 — validated the launch-time per-node model/backend override (mono fable/sonnet) (run 019f2236)

2026-07-01 — dogfood surfaced (and fixed) a claw+gpt explore-mode façade; grounded reviews restored (runs 019f1cf7, 019f1d24)

The bug — claw+gpt-5.5 “explore-mode façade” (the headline finding)

In explore mode the reviewer must READ the chunk files (source is not inlined) and report files_reviewed. reviewer_gpt (claw / gpt-5.5) made zero tool calls, fabricated a plausible verdict from priors, and self-reported files_reviewed paths it never opened — sailing through the engagement gate (which trusts the self-reported list). reviewer_claude (claude_code) read the files and behaved correctly. So Willy’s GPT half of the cross-family review was a façade, and the “no logs for reviewer_gpt in the studio” symptom was just its consequence (no tool calls → nothing to stream).

Root cause (verified with a 1-node probe + code trace): iterion always sent tool_choice: nil (→ provider default “auto”), and gpt-5.5 under “auto” declines to use provided tools and answers directly, even when the prompt says “you MUST call read_file”. Claude is agentic under “auto”; gpt-5.5 is not. The sub-agent’s first hypothesis (a missing ToolChoice passthrough in claw’s OpenAI chat/completions path) is a real latent bug but not the operative cause here: ChatGPT-OAuth always routes to /v1/responses, which does forward ToolChoice — so nil→auto was the actual lever.

The fix — ForceInitialToolUse (agentic parity for claw)

pkg/backend/model: new GenerationOptions.ForceInitialToolUse. The tool loop pins tool_choice="any" on the first turn (and until the first tool call lands), then reverts to auto so the model can finish. Enabled for every tool-equipped claw agent/judge (claw_backend.go). The /v1/responses path maps “any”→”required”, so the force reaches the ChatGPT-forfait endpoint.

Value the run produced (in the worktree, pre-convergence)

Grounded cross-family review of pkg/runtime → real edits: convergence.go, engine_exec.go (the rem<=0 duration-budget guard after acquireResources), fan_out_each.go simplification, workspace_safety.go hardening, + added tests (fan_out_each_test.go, parallel_test.go, worktree_default_test.go). Most other chunks were reviewed CLEAN with genuine cross-references (not a rubber-stamp).

Caveats / follow-ups

  1. Sandboxed runs still need the fix in-container. The first sandboxed relaunch (019f1cf7) still showed 0 gpt tool calls — the iterion __claw-runner inside the container ran a stale binary. --sandbox none (in-process, fresh binary) was used to validate + deliver. To help the operator’s normal (sandboxed) Willy runs, rebuild the sandbox image with the fixed iterion (or verify addClawBinaryMount mounts the fresh host binary and that /usr/local/bin/iterion wins on PATH).
  2. OpenAI forfait cap. reviewer_gpt hit a 429 usage-limit after ~13 passes -> recovery_pause. Resumed with --answer acknowledge_recovery=continue --force and ITERION_VIBE_MODEL_GPT=anthropic/glm-5.2 + ZAI mode (both reviewers on GLM-5.2 via z.ai) to converge. Cross-family diversity is paused in that fallback.
  3. Latent claw bug (not hit here): OpenAI chat/completions + Foundry providers drop ToolChoice (only /v1/responses + anthropic/bedrock/vertex forward it) — API-key-mode gpt with forced tools would not be forced. Fix in .works/claw-code-go internal/api/providers/openai/provider.go + foundry/.
  4. Engagement gate is still self-reported. streak_check trusts files_reviewed subset of chunk_file_list, not real tool telemetry. Forcing initial tool use makes the model actually read (observed: full 2/2-file coverage per chunk), so this is no longer urgent — but grounding the gate on real read telemetry would make it Goodhart-proof for any future backend.

Lessons for next run

2026-06-23 — code-quality axis was a no-op; fixed → 14 real cleanups + a sandbox-claw bug (runs 019ef545, 019ef550)

Run 1 (019ef545) — diagnostic: the axis was a no-op

Willy approved every chunk (0 rejections, 0 fixes, ~$2) and cruised toward a clean convergence having changed nothing. Root cause: Willy’s reviewer is a production-readiness reviewer whose anti-false-positive rule says “style/naming/minor-optimisation/missing-doc are not blockers” — exactly the class a code-quality axis targets. It found the issues (dead SwitchTeamWithCookie, drifted RotateSession, dup dotenv parser…) but filed them under fix_plan as “recommended non-blocking cleanups for a future quality pass” and set approved=true. The fixer had the same bias (pushback “not production-blocking”). So an axis-scoped run can converge on unimproved code. Cancelled after diagnosis.

Bot fix (branch fix/willy-code-quality-axis)

When improvement_prompt is non-empty the axis now redefines “blocking”: a concrete, evidence-backed on-axis finding is a blocker the fixer applies this pass; the “style not a blocker” carve-out is suspended for on-axis items. Mirrored in the fixer’s pushback rules. Convergence guards kept (concrete + smallest-change

Run 2 (019ef550) — the fix works

14 chunks rejected→fixed, real on-axis cleanups: net −259 lines across 21 files (dead code in pkg/auth, pkg/dispatcher, pkg/dsl/ir; dup slug-retry; zero-caller exported FromIdent; misleading docs; redundant wrappers). Cumulative worktree build + vet + package tests green. Cost $44.12, ~1.5h. Harvested to branch willy/iterion-code-quality-2026-06-23 (cleanups as one commit).

Engine/bug findings

  1. Bot demote-and-defer bug → fixed (above).
  2. Real bug Willy found in iterion (off-axis, separated to its own commit): pkg/backend/delegate/io.go ToIOTask/FromIOTask silently dropped CursorFragments, PresetFragment, SystemPromptMode, Ultracode, SecretFiles, UserContent, RepoRoot, Iteration. The runner builds the prompt via Task.BuildSystemPrompt() after FromIOTask and cannot re-resolve cursors/preset, so sandboxed claw silently lost cursor calibration + launch-preset bias (and zero SystemPromptModeStandalone instead of claw’s AuthoredBase → adaptivity-parity regression). Fix carries them over the wire with a proper round-trip mutation test the fixer wrote in the same commit (TestIOTaskRoundTrip asserts every new field survives ToIOTask→JSON→FromIOTask). Verified by code-trace that the runner consumes them via Task.BuildSystemPrompt() after FromIOTask.
  3. devbox couldn’t run inside the sandbox → FIXED in the engine. The devbox run -- … convention died with mkdir: cannot create directory '/home/.../.cache/devbox': Permission denied (run 019ef550), so the fixer fell back to bare go. Root cause: host_state: auto lays a user-owned tmpfs at $HOME, but the Go-cache binds nested under it ($HOME/.cache/go-build, $HOME/go/pkg/mod) made docker create the parents $HOME/.cache / $HOME/go as root:root, shadowing the writable tmpfs so devbox couldn’t mkdir its cache. Fixed by homeNestedBindParents in pkg/runtime/sandbox_mounts.go, which also lays a user-owned tmpfs at each nested-bind parent — the whole $HOME subtree is now writable and devbox is first-class. verify-build skill updated to prefer devbox run again.
  4. Reviewer context overflow on large repos → MITIGATED. Run 2 ended failed_resumable when reviewer_gpt (gpt-5.5 forfait) hit context_length_exceeded at review_loop 11. Measured: the accumulated cumulative_scanned_areas+prior_pushback was only ~4 K tokens — not the cause; the dominant input is the inline chunk_content (was ≤ 30 K tokens) vs gpt-5.5 forfait’s effective window (well below the API’s). Fix (A+B, configurable): lowered max_review_chunk_tokens default 30000 → 16000 (fits the default forfait reviewer with head-room), and added model-adaptive sizing — reviewer_context_tokens (+ reviewer_context_percent, default 45): when set, the chunk budget is capped at that %-of-window and the MAX_CHUNKS rebudget can no longer re-inflate past the ceiling (big repos take more passes instead of a bigger chunk). Deeper fix SHIPPED + validated (ADR-045, dc22b626c, v0.5.0): new context_mode (default explore) — snapshot_chunk writes a per-pass chunk INDEX markdown and the reviewer reads the listed files on demand via its read tools instead of inlining the source, removing the context-window ceiling on chunk size (a package/file bigger than the window is now reviewable). Anti-Goodhart guard folded into streak_check (no graph node): a pass counts toward convergence only if files_reviewed is a non-empty subset of the chunk’s files. inline mode kept as fallback. Validated by run 019ef5d5 (scoped pkg/log): converged in explore mode — index written, verbatim files_reviewed, engaged=true every pass, real doc-nit fixes, stop → verify → commit → done, $7.50. Convergence note: an axis-blocker quality reviewer finds a long tail of distinct genuine nits (each resets the streak) — it still asymptotes once they’re exhausted (pkg/log: 2 distinct doc fixes → converged at review_loop ~5), but a quality-axis run is thorough, not instant. (This long tail is the axis-blocker behaviour, orthogonal to explore.)
  5. .devbox/ swept into the run commit → FIXED (.gitignore). Now that devbox runs in-sandbox (operator’s HOME-nested-bind fix), it regenerates .devbox/ in the worktree and commit_changesgit add -A committed it (run 019ef5d5). Added .devbox/ to .gitignore (same family as .tmp-*/ / .gomodcache/).
  6. New capability used: iterion run --max-cost-usd (+ --max-tokens, --max-duration, --max-iterations, --max-parallel-branches) on run and resume — set the $120 ceiling without editing the bot (branch feat/budget-override-flag).

Lessons for next run

2026-06-15 — deterministic build/test gate shipped + validated live; 2 more pkg/store fixes; placeholder root cause (runs 019ec9d5, 019eca0d)

Follow-up to the 019ec7ed run below. Implemented the recommended deterministic build/test gate (the converge-on-broken backstop), re-ran Willy to exercise it, and the gate caught a real regression — its own e2e-test gap.

The gate (commit 9419b12f)

Per the CLAUDE.md “skill-guided + deterministic gate” doctrine, between streak_check’s stop and commit_changes:

Run 019ec9d5 (pkg/store re-run) — 2 more real fixes + the placeholder recurs

Run 019eca0d (pkg/clock) — gate validated live, end to end

Tiny clean scope (1 chunk → converges in 2 cross-family passes) purely to fire the gate. Converged → verify_build (prepared=true) → verify_run passed, exit_code 0commit_changesdone. The gate fired, built iterion in-sandbox (its script self-handles GOTOOLCHAIN/devbox, vendor mode, writable GOCACHE, a flake-guard), and committed. Crucially verify_build caught a real regression: commit 9419b12f added the verify nodes but never stubbed them in e2e/whole_improve_loop_test.go, so the 4 convergence e2e tests would route to fail — invisible to the chunk reviewers, but verify_build’s go test ./... caught it and authored the stubVerifyGate fix (repatriated as fb503a8f). The converge-on-broken backstop working against the gate’s own gap.

Findings / misses

  1. Sandbox image lacks iterion’s pinned Go 1.26 (full:edge ships 1.24). Blocks in-sandbox build/test for every node, makes verify_build slow+costly (~33 min / ~$3.91 fighting the toolchain on this run), and is what induced the placeholder. Real fix = publish the sandbox image with Go 1.26 (infra follow-up); the gate is sound, the environment isn’t.
  2. verify_build ran rm -f .git; git init to bootstrap a repo for the e2e worktree tests (the worktree’s .git file points outside the sandbox mount), which severed the operator’s worktree and stranded the run’s commits (recovered by hand). Guarded (commit 487b0c10): the verify-build skill now forbids destroying/recreating .git and says to SKIP git-dependent tests when git is unavailable. (Also: the sandbox should mount the worktree’s real git dir.)
  3. The placeholder is ultimately an engine bug — a claw delegate that did tool work but ends with an empty final turn should not be rendered as an applied=false “work in progress” by the formatting pass; it should reflect the work or report step-exhaustion honestly. Deferred engine follow-up (pkg/backend/delegate/parse.go + the claw backend).
  4. Gate cost on iterion: verify_build + verify_run build+test the WHOLE repo at convergence. Cheap when the toolchain is present (verify_run reused the warmed cache in ~45s); expensive when it isn’t (finding 1).

Lessons for next run

Follow-ups fixed (2026-06-15, same session)

All deferred items above are now fixed + verified:

Remaining smaller nicety (not blocking): full step-exhaustion telemetry (a distinct StepsExhausted signal on the delegate Result for the event log) — the recovery-honesty fix + the Go-1.26 image already remove the placeholder’s cause and symptom.

2026-06-14 — scope_globs shipped + pkg/store hardening + fixer-placeholder finding (run 019ec7ed, cancelled-for-value)

Value

Findings / misses (bot + engine)

  1. fix_gpt returns a “work in progress” placeholder and ships unverified edits. The fixer edited 5 files (9.3 min, 29k tokens) but its final turn produced empty output (raw_output_len: 0, formatting_pass_used: true), so iterion’s formatting pass synthesized applied=false + “validating and fixing”. Two regressions rode through unverified: B4 ReadAt on a write-only fd (EBADF → broke every event append; 6 test failures) and B1 a missed caller in pkg/cli/resume.go (build break). Root cause: the review prompt forbids provisional verdicts; the fix prompt did not. Fixed — FINALIZE guard in fix_system (3be59a70): no placeholder + self-verify with the project’s own build/test + update ALL callers.
  2. The loop cannot catch a fixer that breaks the build (no deterministic post-fix gate). Reviewers review their chunk’s source; a cross-file build break (resume.go, outside the pkg/store scope) or a runtime-only bug (B4) is invisible, so the loop rebuilt a clean streak on a broken tree and would have converged-on-broken. The fix is the CLAUDE.md “deterministic gate” pattern: run the repo’s OWN build/test as a tool/compute gate after fixes / before commit_changes, degrading the run if red. Universality-constrained (the build/test command is per-repo) → skill-guided detect + deterministic gate, like sec-audit-source’s scan_health. Recommended follow-up — not yet implemented.
  3. scope_globs ↔ out-of-scope callers: a scoped review can’t see a signature change’s callers outside the scope. The FINALIZE “update ALL callers” clause mitigates from the fixer side; the build gate (2) is the deterministic backstop.

Engine / repo hardening produced

Lessons for next run

2026-06-14 — convergence machinery re-confirmed + path-scope finding (run 019ec598, cancelled)

2026-06-13 — bounded error-handling dogfood (run 019ec0c8)

Value (the core loop works and finds real issues)

Findings / misses

  1. Willy self-kills under task studio:dev (CRITICAL — dogfood infra). Willy edits the live main working tree (it has sandbox: but no worktree: auto, and no per-run worktree was created — confirmed .iterion/worktrees/ empty for this run). task studio:dev runs the backend under watchexec -r -e go -w cmd -w pkg -w vendor. So the instant fix_claude wrote cmd/iterion/scan_shards.go, watchexec restarted the studio backend, which drained the in-flight run → context canceledfailed_resumable. Any code-editing bot that touches cmd//pkg/ on the live tree will be cancelled by its own edits under the dev server. Mitigations: run such bots against a non-watchexec studio (built iterion studio/server), or via a CLI iterion run in an independent process, or on an out-of-tree workspace copy.
  2. No worktree isolation (design tension — engine/bot). Willy mutates the operator’s actual checkout and (by design) leaves the edits uncommitted for review — but with no isolation it (a) pollutes the live tree, (b) self-destructs under file-watchers (#1), (c) risks losing edits on any restart. Billy (branch-improve-loop) and Featurly (feature-dev) use worktree: auto + a commit step. Recommendation: give Willy worktree: auto + a commit-on- convergence step (consistent with Billy), or at minimum document loudly that it edits the live tree. ADR-level decision, not a quick patch. Evaluated 2026-06-13 — deferred. The clean worktree: auto move has a real conflict: Willy’s cross-run convergence relies on .whole_improve_loop.state (cursor + clean_streak) persisted at the workspace root to amortize the num_chunks-deep sweep across re-dispatches (issue-#12 / ADR-011). A worktree: auto worktree is created fresh from HEAD and removed on finalize, so that state would vanish each run → cross-run streak amortization breaks (every run re-sweeps from cursor 0). Doing it correctly means relocating the state off the ephemeral worktree (run-store or parent repo) and adding a commit step — a genuine ADR, not a patch. Since #1 is also solvable operationally (CLI launch / non-watchexec studio), the worktree change is deferred pending that ADR rather than rushed.
  3. Chunk grouping can exceed max_review_chunk_tokens ~7× (coverage). Chunk 0 was 218K est tokens / 149 files against the 30K default budget; the renderer then hard-caps content at budget*4+4096 (~124K chars), emitting "... [chunk content truncated at the char cap] ..." — so files grouped past the cap are silently unreviewed even though they count in file_count. The per-chunk grouping (by directory) doesn’t split a group that overflows the budget. Worth bounding chunk size to the budget (or splitting oversize groups).

Engine hardening

Lessons for next run