Iterion

Appearance

Like what we’re building? Star on GitHub

← Bot runs

adr-cartograph (Adry) — bilans ​

ADR cartographer + completeness auditor. Read-only on code, reasonably idempotent: observes what the code implements and emits committable Nygard ADRs in docs/adr/, plus a feature-completeness audit; files optional handoff issues to adr-rechallenge (re-challenge) and feature-gap-fill (gap completion). Template: docs-refresh (Doki).

2026-09-21 — the #1464 receipt: two passes, the drifted lock read as noise, 15 ADRs in stride (run 01a0c5a5-b07d) ​

  • Status: validated (the receipt this run was launched for is measured, live and reproduced) — with one engine regression surfaced, ticketed (#1558).
  • Versions: bot v2.0.2 · iterion v3.177.0+8ee6f0e6d (the #1530 branch head, = main once merged as 9a9cd778e9) · host run (--sandbox none), worktree: auto, Sonnet on both agents (ITERION_ADR_MODEL_CLAUDE / ITERION_ADRY_MODEL_CLAUDE=claude-sonnet-5, the account's Opus meter was capped).
  • Method: CLI run from the branch worktree, --store-dir /home/jo/lab/ai/iterion/.iterion, --merge-into none, --no-interactive, --timeout 75m, max_passes=1 (= up to two passes), bundle_self_path=bots/adr-cartograph; account selected by CLAUDE_CONFIG_DIR=<cswap session profile> with the auth-override variables AND ZAI_API_KEY unset (a first attempt died on z.ai's [1310] weekly quota: the engine reads ZAI_API_KEY from the ambient environment as the z.ai hint and routes claude_code through the facade; a resume keeps the credential pinned at launch — a fresh run re-resolves). ~62 min wall (20:26Z → 21:07Z, survey 33 min on Sonnet).
  • Result: finished, two passes. Pass 1: scan 108 ADRs → survey → manifest → campaign authored ADRs one commit each → scope_check {scope_ok: true, out_of_scope: []} with the engine's .claude/skills/ mirror laid untracked → verify_build (the agent's devbox run -- task build) → gate not converged. Pass 2: campaign → scope_check at 21:06:28Z with devbox.lock DRIFTED (lock polls 21:05:15 → 21:07:30 all drifted) → {scope_ok: false, out_of_scope: ["docs/references/map-docs.md"]} — the drifted lock is NOT listed; the one path named is a real out-of-scope write (the agent regenerated the docs map with task map:gen) → gate not converged → mark_issue_for_review → update_cache → done. Storage branch iterion/run/01a0c5a5-… → 2c078ab74: 15 ADRs (rewind produced-scope default, repograph ×4, tree-noise canonical list, list-shaped run vars by env, credpool ×2, usagecap ×2, pool audience, credential spend attribution, schedgate overlap vocabulary, secretguard certainty/suspicion) — cherry-pick candidates; the run's own worktree preserved (see the regression below).
  • The receipt, reproduced on a real engine worktree (the preserved worktree of the earlier failed run, mirror laid by the engine): the merged scope_check command run verbatim against a hand-drifted devbox.lock — with ITERION_TREE_NOISE provisioned: {"scope_ok": true, "out_of_scope": []}; with the variable unset (the pre-#1530 floor): {"scope_ok": false, "out_of_scope": ["devbox.lock"]}. The only difference between the two runs is the drifted lock — the defect #1464 was opened for, closed.
  • Value: HIGH — the receipt itself, plus 15 undocumented decisions of the last days captured as Nygard ADRs (including the tree-noise decision this very run was validating: ADR-107 "one canonical tree-noise list, mirror-only finalizer exception").
  • Findings / misses: (1) engine regression from #1507's wip bank: git add -A -- ':/' ':(exclude,top).claude' … exits 1 on a repository that gitignores .claude/ (iterion's own **/.claude/): git reports the exclusion pathspec as "paths ignored by .gitignore" and refuses — the wip bank fails, the run's uncommitted work is left unbanked and the worktree preserved (wip bank git add -A failed: exit status 1). The finalize e2e laid an untracked mirror that was NOT ignored — a fixture that does not have the run's real shape (the same class the review-fanout exec test paid in #1530). Ticket #1558. (2) The lock drift came from the verify_build AGENT's own devbox run (the class #1464 leaves open by design) and, once, went back to identical within 45 s (mechanism not determined: the agent's inner Bash calls are not in the event stream). (3) The campaign's task map:gen is out of Adry's docs/adr scope — the gate flagged it correctly.
  • Engine hardening: #1558 (the staging gesture must not name an ignored path: drop the exclusion when git check-ignore says the path is already ignored, and the e2e's fixture must carry the ignored-mirror shape too).
  • Lessons for next run: pin Sonnet when the account's Opus meter is short (the survey is 30+ min either way); unset ZAI_API_KEY in the launch environment unless z.ai is wanted; expect map-docs.md to trip the scope gate whenever the campaign regenerates the docs map — either widen adr_dir-scoped allowances to the generated maps or tell the campaign not to regenerate them.

2026-07-07 — v2 dogfood on iterion's recent work: 4 ADRs in stride, converged first pass (run 019f3d3b-6a56) ​

  • Status: VALIDATED — first live run of the v2 shape, textbook behaviour end to end.
  • Versions: bot v2.0.0 · iterion dev+239203525cc8 · no sandbox (host run, worktree: auto).
  • Method: CLI run, --store-dir <workspace>/.iterion, --merge-into none, bundle_self_path=bots/adr-cartograph, diff_since=origin/main (prioritise the recent unpushed work), max_passes=3, --max-cost-usd 25 --max-duration 1h (mono claude, forfait). 15m39s wall.
  • Result: finished, gate.converged=true on the FIRST pass. Pipeline: scan_adrs 61 ADRs (dup '002' flagged, next NNN correct) → survey_code 4 non-mechanic decisions + 2 gaps → manifest drift=4/orphans=0/coverage 93% → campaign authored 4 ADRs, one commit each in stride on storage branch iterion/run/meteor-ray-ashglyph-fb1c @edeae90e0c: docs(adr): 061 per-backend system-prompt composition mode, 062 system-reminder envelope, 063 post-compaction todo-reseed nudge, 064 worktree finalization delegated authority. scope_check clean (adr_dir-only); deterministic verify green (devbox-pinned go build); cache updated.
  • Value: HIGH — the four ADRs document exactly the real, undocumented decisions of the last days' unpushed work; spot-read of ADR-061 shows proper Nygard form, an accurate Context (the claude_code-vs-claw composition seam), a real trade-off and correct code anchors. NNN sequencing correct (061-064 after the parallel session's 060).
  • Findings / misses: none observed — handoff issues correctly NOT filed (gaps were below the medium bar / rechallenge_after_days=0).
  • Engine hardening: none needed.
  • Lessons for next run: diff_since=origin/main is an excellent default for recurring Adry runs on an active repo — it focuses the survey on fresh decisions; cherry-picking the storage branch is the operator follow-up (commits deliberately not FF'd here).

2026-07-07 — converted to v2 minimal-framing (ADR-058 fleet rollout) — structural-validated, dogfood pending ​

  • Status: converted, dogfood pending — structural validation only this pass: iterion validate clean, catalog universality/typing/bundle-consistency green, stub e2e green where wired. NOT yet live-dogfooded in the v2 shape; treat the sections below as describing the RETIRED v1 shape.
  • Versions: bot v2.0.0 · iterion worktree branch (rollout of 2026-07-07, see git log)
  • Shape: scan_adrs + read-only survey_code + build_manifest (re-globbing the live adr_dir each pass) are kept; the round_robin review/fix relay is replaced by ONE campaign that re-applies the decision-vs-mechanic checks, authors one ADR per commit in stride, and files the rechallenge/feature-gap handoffs itself (board caps). Deterministic scope gate = adr_dir/*.md-only vs run base. 15 nodes → 10 exec; worktree: auto (v1 committed terminally on the live tree).
  • Reference proof of the shared mechanism: feature-dev v2 pilot run 019f3bb4 (one pass, 11m33s, 2 in-stride commits, deterministic gate converged — see docs/bot-runs/feature-dev.md) and the Willy/Billy v2 tours.
  • Next: a dedicated live dogfood + bilan in this file before the bot counts as validated in its v2 shape.

2026-06-22 — 13 ADRs on GLM-5.2, survived anthropic failover (run 019ef04e-a02e) ​

  • Status: validated (high value) — authored ADRs 029-041.
  • Versions: bot adr-cartograph · iterion v0.16.0 (110ea1c33)
  • Method: iterion run (CLI, branch dogfood/wave1, live-tree docs/adr scope). survey + review on glm-5.2 via z.ai; when z.ai's 5h cap hit mid-review, reviewer_claude surfaced a graceful acknowledge_recovery human-pause → resumed → review/fix loop completed on anthropic forfait (gpt-5.5 reviewer + opus). 1233 events.
  • Result: commit 86eeb81f9 "docs(adr): record ADRs 029-041 for sandbox, SSO, quota, and ultracode" — 13 ADRs, +683 lines, on dogfood/wave1 (atop current origin/main). Converged to done.
  • Value: captured the architecture decisions from the recent campaign wave (sandbox, per-org SSO, quotas, ultracode) into proper Nygard ADRs — the canonical "capture decisions after a code-mutating wave" use case.
  • Findings: (1) glm-5.2 survey is very verbose — survey_code ran ~900+ tool calls / ~20 min (many empty greps) before converging, vs opus's tighter surveys; thorough but slow-to-stop on open-ended exploration. (2) graceful rate-limit recovery (acknowledge_recovery pause) worked here — contrast Nexie's hard fail on the same 429; inconsistent rate-limit handling across nodes is the cross-cutting finding.
  • Lessons: glm-5.2 is a viable ADR author; budget more wall-clock for its survey. Recovery-pause + resume across a provider failover preserved all survey work.

2026-06-14 — finding #6 convergence tuning (runs 019ec2f3, 019ec4ec) ​

Two prompt-level fixes for #6, each validated by a live re-run on the aligned tree (8 ADRs committed, 0 ADR-drift, coverage 100%):

  • 7546d0d1 — streak_check.stop was stricter than its own approved passthrough (it omitted the low-confidence clause); aligned it so a low-confidence rejection is non-blocking (CLAUDE.md asymptote rule). run4 (019ec2f3) still failed — the blocker was confidence: high, not low, so the quick-win mis-aimed.
  • 61e995bb — the real root cause: the persistent high-confidence blocker was a GAP (the survey's file-diff-payload completeness finding), and both reviewers were treating the unfixable gap as a blocker ("file an issue for / address this gap"). Adry is read-only on code → it cannot close a gap → permanent oscillation. review_system rule 7: gaps are NEVER blockers (confirm-real → ride the deterministic handoff path to feature-gap-fill; only ADR drift / orphans / format defects block).

Result: partial. Rule 7 measurably helped — reviewers now sometimes vote approved=true, blocker_count=0 (they never did before) — but run5 (019ec4ec) still exhausted review_loop(10/10): the LLM reviewers are stochastic, and one still occasionally raises a gap as a blocker (bot-marketplace-shallow-clone) despite rule 7, resetting the cross-family streak. A prompt instruction alone is not a reliable gate.

Structural fix 09910359 — DONE, and run6 (019ec51f) CONVERGED. The fix: stop passing gaps/total_gaps into the reviewers' review_input entirely (both alt -> reviewer_* edges) and clean the review prompts of gap-review references — gaps now flow build_manifest -> prepare_commit's handoff, never through the review loop. With the reviewers structurally blind to gaps (not just instructed to ignore them) and rule 6 calibrating ADR prose, an aligned tree has nothing to block on. run6 result: reviewer_claude approved → reviewer_gpt approved (cross-family, 0 blockers, 0 fixer runs) → streak_check stop → detect_changes → prepare_commit → commit_changes → done. First clean convergence. prepare_commit ($0.83) committed no ADR change (0 drift on the aligned tree — correct idempotent behaviour, the git diff --cached --quiet guard skipped the commit) and filed the 3 type:feature-gap handoff issues to the board (source:adr-cartograph: file-diff-payload, ShallowClone-tests, marketplace-transport) — the A→Fini handoff is validated end-to-end. #6 is FIXED.

Minor follow-up: detect_changes counted the engine's bot-catalog regeneration (an unrelated side-effect) as a working-tree change, so it routed through prepare_commit ($0.83) instead of the pure update_cache -> done no-op; the outcome was still correct (no spurious commit). Filtering the catalog regen (like the cache) in detect_changes would make the aligned re-run a true zero-LLM-commit no-op.

Cost: 6 Adry runs this campaign; survey ~$3.58 + ~10 min each is the floor (survey_code always runs before the deterministic aligned-check).

2026-06-13 — first dogfood, scoped pkg/git (runs 019ec1f8 + 019ec25f) ​

  • Status: partial — core machinery validated end-to-end and 2 real bugs fixed; the bot did not converge on this run (finding #6) — since FIXED (structural fix 09910359; Adry converged on run6 019ec51f — see the 2026-06-14 section).
  • Versions: bot 0.1.0 · iterion @ f3289632 (worktree worktree-adr-bot-suite, unmerged)
  • Method: iterion run bots/adr-cartograph/main.bot --var code_scope_globs='pkg/git/**', forfait forced (ITERION_OPENAI_USE_OAUTH=1, API keys unset), claude_code opus-4-8 (scan/survey/reviewer_claude/fix_claude/prepare_commit) + claw openai/gpt-5.5 (reviewer_gpt/fix_gpt). No sandbox, no worktree:auto (the repo-root sha-cache must persist across runs). Scoped to one decision-rich package to bound cost.
  • Result: did not converge — both the authoring run (019ec1f8) and a re-run on the now-aligned tree (019ec25f) exhausted review_loop (10/10) and hit the fail backstop. Survey ≈ $3.58 and ~10 min each (pkg/git, 14 files, tool_max_steps 80). The review loop never reached the cross-family double-approval streak (see #6). But the run did the valuable work: it authored 8 genuinely good ADRs (docs/adr/020–027, committed b60106ba) recording real pkg/git decisions — arg/path-injection hardening, IsAncestor-via-merge-base, empty-tree-SHA root baseline, C-locale stderr classification, NUL-framed parsing, working-tree (not index) panel semantics, main-repo-root gitdir resolution, git-CLI-shell-out vs go-git. Format matched the repo's Nygard convention exactly.
  • Value: high — produced 8 committable ADRs from a cold read of pkg/git, with accurate Context/Decision/Alternatives/Consequences + a Re-challenge hook each. The survey's is_mechanic self-critique correctly flagged 0 mechanical decisions (all 8 were real). scan_adrs correctly detected the historical 002-* duplicate and computed next_adr_number.

Findings ​

#FindingSeverityState
1build_manifest passed survey JSON via raw {{!input.x}} wrapped in literal quotes — LLM prose (apostrophes/parens) broke bash -c (syntax error near ')')real bugFIXED 1bbca1c0 (drop ! + quotes → {{input.x}} routes through shellEscapeValue). docs-refresh shares the latent pattern (only passes apostrophe-free tokens) → cross-bot pitfall added to workflow_authoring_pitfalls.md
5build_manifest computed coverage from the frozen scan_adrs.adrs (entry snapshot), so ADRs the fixers authored mid-loop were invisible → coverage_pct froze at 71% < 80% → the gate was unreachable → loop exhausted with 8 good ADRs on diskreal bug, convergence-breakingFIXED f3289632 (re-glob docs/adr/*.md each pass, parse ../../<code> refs). Proven deterministically + LIVE on run3: coverage 71%→100%, drift 8→0
6Even with coverage=100% (aligned tree, 0 drift), the review loop never reaches cross-family double-approval — root cause: reviewers treated unfixable GAPS as blockers → oscillate to review_loop(10/10) → failreal, FIXEDstructural fix 09910359 (gaps bypass review_input); run6 (019ec51f) CONVERGED — see the 2026-06-14 section
2fix_claude (claude_code, output-schema+tools → two-pass) hit StructuredOutput — No such tool available in Pass 1 (it's a Pass-2 mechanism), then recoveredbenign engine quirk, pre-existing (all fix-nodes; claude_code.go:341)note only
3enforce_fix_scope git checkout -- <p> reverts ALL tracked changes outside docs/adr/, incl. pre-existing uncommitted WIP (it ate my own mid-run hot-fix of main.bot)known loop-bot behavior (docs-refresh idem)mitigation: commit fixes BEFORE resume (in HEAD = not "changed"); don't hot-edit tracked files during a run
4reviewer_gpt → 401 OpenAI "token is expired" mid-run (forfait ~/.codex 10 days stale)infraoperator re-authed codex → resumed on forfait. See workspace memory project_openai_oauth_token_invalidation

Engine / bot hardening ​

  • 1bbca1c0 — shell-safe data passing in build_manifest (finding #1).
  • f3289632 — build_manifest re-globs docs/adr so coverage tracks authored ADRs (finding #5).
  • b60106ba — the 8 dogfood-authored pkg/git ADRs (kept for review).

Lessons for next run (finding #6 is the gate to production) ​

The review loop must converge to an asymptote; right now it does not. The coverage gate (#5) is satisfied but the cross-family double-approval streak is unreachable because the LLM reviewers treat ADR-prose imperfections as blockers and re-litigate every iteration. Directions to try (each needs a live re-run to validate — tune one at a time):

  1. Calibrate review_system: an ADR is APPROVED when it factually and accurately records the decision the code embodies (right sections, valid Nygard format, code refs resolve). Prose polish / nuance-completeness must NOT be blockers. This is the anti-Goodhart line: the gate is accuracy, not style.
  2. Treat low-confidence rejections as non-blocking in streak_check (per the CLAUDE.md asymptote rule): add || input.confidence == 'low' to the stop condition so a noisy nitpick doesn't reset the streak.
  3. Strengthen prior_pushback feedback so a reviewer cannot re-raise a resolved item without new evidence (already wired — verify it's actually suppressing re-litigation in the prompt).
  4. Consider bounding the per-run authoring batch (8 ADRs at once gives the reviewers a large surface to nitpick); authoring fewer per pass may converge more reliably. Do not "fix" #6 by short-circuiting the review when the manifest is aligned — that trades oscillation for a bypassed quality gate (Goodhart). The fix is to make the reviewers converge, not to remove them.

Other operational notes: run Adry on a clean tree (finding #3); the forfait token expires (~hours) so long runs may need a mid-run re-auth + resume (finding #4); survey cost is the floor (~$3.58/run on pkg/git) since survey_code always runs before the deterministic aligned-check.