Iterion

Appearance

Vetty — dep-update-guard run bilans ​

Reactive security + alignment guard for automated dependency-update PRs (Dependabot / Renovate): audit the bump (supply-chain + CVE), align the consuming code, prove the tree with the deterministic verify gate, commit onto the PR branch, post the verdict comment. Never merges past a check — and only ever the commit it audited. See bots/dep-update-guard/.

2026-08-04/05 — the backlog majors + the Vite 8 alignment: capability proven, and the verify seam found its shape ​

  • Status: capability validated; landing required four bot/engine fixes found in stride. Of the 6 pre-group major PRs /vetty'd: #239, #38, #32 merged by the loop (#38 dissolved the App-workflows-permission fear — on a queue repo the QUEUE merges, not the App; #32's recreated branch re-audited clean with a pro-grade risk trade-off: the one introduced sidecar HIGH has an upstream fix while holding would keep a CRITICAL JetStream auth-bypass); #20, #21, #33 closed on grounded hold verdicts (a self-contradictory Dependabot recreate — trailer 6.0.3 vs diff 7.0.2; typescript-eslint peers capping TS <6.1; Bitnami Secure Images mutable-latest posture). The escalate lane ran its full arc live: bare-model-spec crash (fix 2.6.2 + fleet lint) → 401 on a forfait-only deployment (engine fix: llm_or_human degrades to a HUMAN PAUSE, #367) → a real operator answer resumed the run through iterion remote runs resume --answers.
  • Vite 8 (PR #19), the alignment objective jo set explicitly: the aligner performed the toolchain migration REPEATEDLY and correctly — first as manualChunks object→function, then (recreated branch) as Rolldown's output.codeSplitting.groups with the three named chunks verified, plus plugin-react ^5.2.0 chosen as the minimal peer move, lockfile regenerated, import.meta.dirname for the CJS __dirname deprecation. GitHub CI went fully green on the migrated head. What kept discarding the work was the VERIFY seam, three ways in one day: an env-dependent cost test (fix #368 — recovered from the orphaned cross-branch commit), a verify.sh omitting the repo's CI drift gate (the agent scopes "bump-relevant" and rationalises away repo-wide gates), and a verify.sh stricter than CI (eslint failing on 534 pre-existing warnings / 0 errors).
  • Fixes shipped: #368 (hermetic cost tests), #369 (skill §1c: mirror CI's exact strictness — never stricter, never looser; the litmus "name the CI step that reds on it"), #370 (the drift-gate precheck LOOPS BACK into verify_build with the exact rejection — an authorship defect self-heals in-loop instead of discarding the alignment at verdict time; budget 4h/$25 for the looped heavy case). #370 took three Revi rounds: R1acb67 (budget vs loop, real), then Rc24903 (my rc=4 loop-back would have permanently defeated a DELTA gate whose baseline swallows first-pass leftovers on retry — accepted and reverted). Revi's blocking gate caught in review what none of my local tests did, twice.
  • Lessons: an agent-authored verify script is the least deterministic node in a deterministic pipeline — pin it three ways (mirror CI exactly §1c, never scope out repo-wide gates §1b, and loop the precheck back instead of judging at verdict time); a delta-based gate must never sit inside a retry loop; suggestions in a review are hypotheses, not fixes — adopting Revi's rc=4 idea without thinking its semantics cost one full round.

2026-08-04 — the retest: all 3 PRs merged by the system, every recovery lane exercised live ​

  • Status: validated end-to-end. After the fix waves below shipped (#357 reconciler+relaunch, #358 per-bot claim, #361 budget terminal-ack; Vetty 2.6.1, arm_automerge flipped on the iterion integration), one /vetty per PR was the only human act. Final: #353 merged 10:57, #362 (recreation of #355) merged 14:27, #354 merged 15:30 — each through Vetty's own arm → merge-queue path.
  • Versions: bot 2.6.0→2.6.1 · iterion prod f272e2306 puis 5f64a87c0 · runs 019fcc30-* (first pass ×3), 019fcc7a/019fccc0/019fcd0c (#354 chain), 019fcca9/019fccd8 (#362 chain).
  • What each PR proved:
    • #353 — arm-first on a merge-queue repo: Vetty's enablePullRequestAutoMerge auto-enqueued the audited head at green (added_to_merge_queue 10:24:33); the queue survived two merge-group rebuilds caused by concurrent release pushes and merged. The merge-now-first ordering of 2.5.0 could never have landed this.
    • #354 — the full recovery gauntlet, unplanned and perfect: run 1 died on the 75m budget mid-verify → reconciler posted the synthetic failure with the budget reason + run link, relaunch lane fired in 12s (run 019fcc7a) → that run died on 75m too (verify ~30% slower on cold nix) → second death relaunched again under the per-bot key (post-#358 deploy, run 019fccc0, now 2h budget) → aligned the MCP go-sdk 1.7.0 break with a root-cause fix (fix(mcp): probe liveness with tools/list on protocol revisions without ping — revision 2026-07-28 / SEP-2575 removed ping; the health-check now probes tools/list on ≥ that revision; no test weakened) → push self-superseded → final run 019fcd0c re-audited, test+race went green on the aligned head, gate success, armed, queue, merged.
    • #355→#362 — the maintenance lesson: Dependabot refuses to rebase a branch Vetty has pushed to ("edited by someone other than Dependabot"), so the post-merge conflict path for a Vetty-touched PR is @dependabot recreate. The recreation (down to 5 bumps — #353's lockfile regen had absorbed the rest) re-triggered the whole loop from PR-open: audit → align (esbuild bundle re-staled, recommitted) → self-supersede → re-audit → arm → queue → merged.
  • Engine defects found live and fixed in stride: the per-node budget checks produced an unwrapped RuntimeError, so a budget death was naked back to JetStream — six ~40s resume/refail turns, each re-provisioning a sandbox (#361, sentinel Cause + code match in the runner carve-out). Budget 75m killed the go workload at its LAST node twice → 2h (2.6.1): the duration cap is a runaway guard, not a performance target.
  • Fresh HOLD case validated fail-closed: one run's sandbox lost the nix substituter → deterministic verify failed → hold_unstable, arm refused ("verdict hold_unstable is not green") even though the agent's own host-toolchain checks were green. Correct behavior, environmental cause — board ticket filed (verify should distinguish env-provisioning failure).
  • Still open: run cost reads None on all these runs (the delegate cost-signal chantier); the board-escalation lane (second death, same head, claim spent) armed twice but never had to complete — unit-tested, not yet live-proven.
  • Lessons: the recovery loop turns "a required gate makes every death a trap" into "every death is one relaunch away from a verdict"; supersede-on-push means the FINAL head is always the audited one — accept the double pass as the price of that guarantee; cold nix caches are the real duration driver (30-46 min verify_build), fix the cache before tightening any budget.

2026-08-03/04 — first Dependabot batch on iterion itself: 2 exemplary audits, 1 silent death, 0 merges ​

  • Status: partial — the audits were the best this bot has produced; everything around them broke. Fixes shipped as v2.6.0 + engine PR #357.
  • Versions: bot 2.5.0 · iterion prod :edge (v3.24.0 era) · runs 019fc8e2/ 019fc8e5/019fc8ed (PR-open) then 019fc924/019fc927 (post-align re-audits)
  • Method: webhook-triggered on the 3 grouped Dependabot PRs of 2026-08-03 (#353 studio npm ×29, #354 go ×15, #355 npm root ×33), gate_context: revi/review (required check + merge queue on main), arm_automerge NOT set.
  • Result, per PR:
    • #353 / #355 — audit exemplary, then stranded. Tarballs extracted and content-scanned, sha512 byte-matched against the registry, SLSA provenance predicates decoded (the react org-rename and the radix "new releaser" maintainer-change flags each investigated and cleared, not waved through), CVE differential (dompurify net −14/+3, brace-expansion ×6 HIGH resolved), esbuild 0.26 parameter-property lowering aligned (b2459da8 — the staled go:embed pi-ext bundle) and a workspace lockfile regenerated (15823337). The alignment push self-superseded the run (overlap: supersede) and the fresh runs re-audited the final head clean — the supersede property doing its job. Then: revi/review=success, PR mergeable… and nobody to enqueue it. arm_automerge was off on this repo, and would not have worked anyway (see misses).
    • #354 — the silent-death case, end to end. Usage window at launch → run_retry_scheduled (retrypolicy ✓) → resumed 36 min later → died on the bot's own max_duration: 40m mid-security_audit → failed_resumable that nothing resumes (the auto-retry covers usage windows only) → no gate status ever posted → required check absent → PR silently unmergeable, indefinitely. The MCP go-sdk 1.6.1→1.7.0 alignment (TestManagerHealthCheckHTTP, ping _meta protocolVersion) was never reached.
  • Value: the supply-chain audits are production-grade — and the batch was a live probe of every seam around the audit, which is where all four defects sat.
  • Findings / misses:
    1. Gate reconciler skipped ALL failed_resumable runs ("it will resume") — false for budget/exhausted/plain failures. The exact silent-block class the bko handover predicted.
    2. 40m/$15 budget too short for the heavy case (a CLEAN npm re-audit alone runs ~30m).
    3. arm_automerge was merge-now-first on CLEAN, which a merge queue rejects — the bot could never land anything on a queue-protected repo.
    4. Config: the iterion integration never set arm_automerge: "true" (bko had it), so green audits stopped at "ready".
  • Engine hardening (PR #357 + bot v2.6.0): reconciler skips only ARMED retries and carries the death reason on the synthetic failure; a dead gate run relaunches its bot once per (PR, head) through the webhook tail; a second death files a deduped source:gate-reconcile board card; the retry sweeper republishes the outcome on permanent abandon; autofix ignores synthetic failures; arm-first automerge with pinned enqueuePullRequest fallback; budget 75m/$20. Adversarial review (opus) caught 2 real defects pre-merge — the reconciler standing down on its own synthetic marker (the board escalation was unreachable), and an abandon-failure republish loop.
  • Lessons for next run: a required gate makes EVERY death loud or the lane is a trap — "resumable" must mean "something is actually coming back"; budget the outlier, not the median; on a queue repo, arming IS the merge path (the queue is the only door); an audit-side success is only half the loop — the merge side needs its own e2e proof per repo shape.

2026-07-31 — v2.5.0 on three heritage PRs, and the rebase that would have stalled every one of them ​

  • Status: validated, with one real gap found and closed. Two of three PRs audited and merged by the bot; the third armed and is waiting on a rebase.
  • Versions: bot 2.5.0 (first live runs of this version) · iterion v3.17.1+d337007d1, runner sha256:9c7203cc
  • Method: /vetty on the three heritage PRs of SocialGouv/buildkit-operator — those opened under the OLD github-actions[bot] identity, before the dedicated Renovate App. #6 alone first (run 019fb413), then #4 and #7 together (019fb6a0-7d9e / 019fb6a0-77d9).
  • Result: #6 audited in 16m20s and merged by the forge App at 17:46:27Z, two seconds before the run ended, on the sha it audited (ef333c66). #4 merged at 05:38:35Z. #7 posted SAFE, armed auto-merge (SQUASH, 05:37:59Z) and is not merged — see below.
  • Cost: ~$1 per PR audited ($1.31 for #6, $1.93 for the pair). Cheap enough that auditing on every dependency PR is not a budget question.

Why they needed Vetty at all, having already been reviewed ​

All four heritage PRs carried iterion/review=SUCCESS — from Revi, a code reviewer, reached by a maintainer's /revi. That is not the same question. Revi reads a diff; Vetty asks whether the artifact on the other end of a version bump is what it claims to be. Merging on Revi's verdict would have skipped the supply-chain control this whole chain exists for, while looking fully green.

The audit is real, not a shape ​

For jq 1.8.1 → 1.8.2, Vetty established that the upstream release exists (jqlang/jq tag jq-1.8.2, non-draft), that it was cut by the same automation as 1.8.0 and 1.8.1 — no unexpected-author or compromised-maintainer signal — that the locked nixpkgs commit exists, that its commit date matches the lock's last_modified byte for byte, and that its package.nix declares version = "1.8.2". For helm 4.2.1 → 4.2.3: the derivation at commit 7525d999 declares 4.2.3, builds helm/helm at tag v4.2.3 with a pinned source hash, and the upstream delta is exactly 2 commits / 3 files matching the release notes verbatim.

It also answered, with evidence, the open question Revi had left that morning on #6 about whether devbox.lock came from a real nix resolution.

The gap: a rebase silently ends the loop ​

review_on_sync was off on the integration, and Vetty's invocation is actions: [opened, reopened]. So:

  1. Renovate rebases a PR — routine, it happens whenever the base moves;
  2. new head sha, and no bot relaunches (webhooks_github.go filters a synchronize when ReviewOnSync is false);
  3. iterion/review is required and is posted on the old sha;
  4. the armed auto-merge can therefore never fire, and nothing says so.

Every Renovate PR that needs a rebase before it merges lands in this. The one proven run (#15, 2026-07-29) merged before its base moved, which is why it was never hit.

Established by reading the shipping code path and the integration config — not yet observed live, because the two PRs that looked like they were stuck in it turned out to be stuck on something else entirely (below).

Fixed by enabling review_on_sync on the integration — the pairing CLAUDE.md already prescribes for a required gate. Worth knowing: BotRule.Actions is deliberately recorded but not enforced, with "synchronize" for the merge gate named in its doc comment as the reason, so the handler's reviewability gate stays authoritative and this wiring works.

The real reason #5 and #7 were stuck: switching Renovate's identity orphans its open PRs ​

Chasing the rebase that never came produced a separate, more consequential finding. Renovate at logLevel=debug, on both branches:

branch.isModified() = true
Branch has been edited but found no PR - skipping

The four heritage PRs were opened by the old github-actions[bot] identity, before the dedicated socialgouv-renovate App. Renovate compares a branch's tip author against its own git identity; the old one no longer matches, so it reads the branch as edited by a third party. Worse, the second line: it does not associate the open PR with the branch at all. Both branches are invisible to it — never rebased, never updated, permanently stuck once they conflict.

Switching the identity a bot commits under orphans every PR it already has open. Nothing warns you: the PRs stay open, look normal, and simply stop being maintained. Half of this batch (#4, #6) hid it by merging anyway — Vetty merges through the forge, not through Renovate.

Remedy applied: close the orphans and delete their branches so Renovate re-proposes the updates under the current identity. The condition is a one-time migration artifact — PRs opened by the new App are recognised normally — but it is worth planning for whenever a bot's credentials change.

Closing carried a real risk, since Renovate normally reads a closed PR as a rejection. It did not fire here: the next debug run lists kubernetes-helm among its 33 flattened updates and processes the branch again. The update was never suppressed — it is held as "pendingVersions": ["4.2.3"] by the repo's own 14-day cooldown. Which reframes #7 entirely: it existed only because it was opened before the cooldown landed (2026-07-28). Closing it aligned the repo with the policy it had since declared, rather than contradicting it.

A correction on how this was measured ​

An earlier reading of the Renovate dry-run log claimed a real run would open eight new pull requests. It opened zero — three consecutive real runs did. DRY-RUN: Would commit files to branch X is not a PR: with internalChecksFilter: strict, Renovate creates the branch and holds the PR until minimumReleaseAge matures. Even Would create PR in a dry run is not a prediction — the dry run evaluates from a clean slate, while a real run re-checks the pending state against an existing branch. The cooldown was working exactly as designed throughout, and the log lines were read as something they do not say.

The practical consequence: the loop cannot be exercised on demand. Forcing a PR would mean defeating the cooldown, which is the one control here worth least defeating. End-to-end validation waits for an update to mature; the audit-and-merge half is provable any time with /vetty on an open PR.

Second observation: a batch of PRs touching one lock file serializes badly ​

The three PRs all bumped devbox.json + devbox.lock. #6 and #4 merged; #7 was audited and armed, then turned CONFLICTING because the two merges landed under it. Vetty was right — it armed rather than forcing a merge — but the outcome is a PR that now needs a Renovate rebase to move, and (before the fix above) would have waited forever for it.

Lessons for next run ​

  1. Reviewed is not audited. A dependency PR carrying a green review from a code reviewer has answered a different question. Do not let a shared gate context blur the two.
  2. Arming auto-merge is a weaker guarantee than merging. The merge_now path pins expectedHeadOid to the audited sha; enablePullRequestAutoMerge pins nothing, so GitHub would merge whatever the head becomes. What restores the guarantee is the required gate — provided it re-lands on each new head, which is precisely what review_on_sync buys. The two settings are one mechanism, not two options.
  3. A batch of dependency PRs touching the same lock file will conflict with each other. Expect the tail of a batch to need a rebase round.

2026-07-29 — v2.4.0: the loop closed — Renovate → audit → gate → merge, unattended (run 019faef9) ​

  • Status: VALIDATED — the first dependency PR to travel the whole chain with no human in it.
  • Versions: bot 2.4.0 · iterion v3.15.0+3a61d2da4 (runner :edgesha256:8c625432)
  • Method: a real renovate.yml dispatch on socialgouv/buildkit-operator, authenticated as the dedicated socialgouv-renovate App.
  • Result: PR #15 (go toolchain 1.26.5 [security]) created 17:43:03Z, merged 17:58:04Z by the forge App — 15 minutes from Renovate opening it to the merge, with no human in between. armed: true, reason: merged: the forge reported every required check already green. Gate iterion/review=success posted on the head at 17:57:58Z; merge commit 6d02f3e46747.
  • The App switch is what unblocked everything. Under GITHUB_TOKEN the four pre-existing Renovate PRs each had a ci run stuck in action_required with zero jobs — GitHub's anti-recursion rule. PR #15's test/lint started on their own. Nothing downstream can work without this, and no amount of bot logic substitutes for it.
  • The cooldown holds without hiding. The dry run showed 17 upgrades marked pendingChecks: true with their held versions named (undici 8.8.0/8.9.0, …), while aged updates proceeded. internalChecksFilter: strict means the branch is not created at all rather than a PR opened with a pending check.
  • The merge targets the audited commit. commit reported committed: false (nothing to align), so the pin fell back to prepare's head — and the merge went to exactly that sha.

The overclaim the run surfaced ​

The check displayed "supply-chain audit clean; alignment committed, build verified" on a PR where the alignment was a no-op (1 commit, 1 changed file, all Renovate's). The verdict is a graph PATH name stamped per-edge, so it read committed whether or not anything was.

The first fix carried the commit agent's own committed flag down to the message — which only moved the claim from one unreliable source to another, since that flag is the agent grading its own work. The shipped version derives it from two shas the run owns (commit.sha vs prepare.head_sha) and routes to the clean verdict, which existed in every string table and was unreachable. All the "committed" strings are now unconditionally true.

A required check that asserts work nobody did is the same false-statement class this bot exists to catch in other people's diffs.

Lessons for next run ​

  • A PR whose base has moved far enough to conflict can never reach the merge: cancel the audit rather than spend it on a refusal that is already knowable from mergeable. Cost saved on this session: one 14-min run.
  • Closing a stale Renovate PR is not a neutral cleanup — Renovate reads it as "this update was rejected" and stops offering it. Leave them; the bot rebases them under the new identity.

2026-07-29 — v2.1.0 live: the whole chain ran, the gate landed, and the merge never happened (run 019faad2) ​

  • Status: partial — every step validated end to end except the last one, which turned out to be structurally impossible as designed.
  • Versions: bot 2.1.0 → 2.2.0 · iterion 9d5efc6c (runner image :edgesha256:c499ba03)
  • Method: /vetty on socialgouv/buildkit-operator#5 (a golang:1.26 digest bump), cloud run on ovh-prod, sandbox iterion-sandbox-sec:edge, gate_context: iterion/review, arm_automerge: true.
  • Result: finished in 14 min, 15 nodes, no human intervention. prepare → security_audit → align → align_gate → verify_build (5 min) → verify_run exit 0 → validate_gate → commit → post_feedback → feedback_health → arm_automerge → done.
  • Value: the merge gate landed for the first time — iterion/review=success on the head SHA, posted through the server's publish endpoint. That link had never worked before (see the 401 below).

The gate: a redirect was degrading the POST ​

post_feedback had been failing with 401 authentication required on a route that is deliberately auth-exempt. Everything else had been eliminated with evidence — the route answers a bogus token differently, the URL and token were correct in the run inputs, the same request reached the handler by hand from both a laptop and a runner pod, Revi's own gate still worked. The remaining hypothesis was that urllib follows redirects, and a redirected POST becomes a GET, which misses a method-specific Go route and falls through to the auth middleware.

Refusing the redirect fixed it. The value of the fix is not only that it works: it names the URL it called and the one the server redirected to, so the next occurrence needs no investigation.

Lesson: an unexplained 401 on an auth-exempt route is worth suspecting the shape of the request before its credentials.

arm_automerge armed nothing, and could never have ​

armed: false
reason: auto-merge request refused: [{'type': 'UNPROCESSABLE',
  'message': 'Pull request Pull request is in clean status'}]

enablePullRequestAutoMerge only accepts a PR that still has something to wait for. The audit takes ~14 min and CI ~3 min, so by the time the bot decides, the PR is always already green — the arm always fails. The feature was not merely buggy on this PR; as shipped it would have merged nothing, ever, on any repo whose CI is faster than the audit. Which is every repo.

v2.2.0 merges through mergePullRequest pinned with expectedHeadOid when the forge itself reports the PR CLEAN and MERGEABLE. The invariant is unchanged — the bot never decides that checks passed, it only acts on the forge's own answer — but the guarantee is now "never merges past a check" rather than "never merges".

Lesson: a capability that only fires in a state your own latency prevents is dead code with a green test. The unit test passed because it stubbed the arm call as succeeding; nothing modelled the state the real API is in when the bot actually calls it.

Other engine defects this run surfaced ​

  • Plugin-source checkout race (fixed): git init creates .git before the fetch lands, and the fetcher treated .git as "tree complete". Five launches hitting a freshly rolled pod at once left one of them with an empty directory, and the loader reported it as "has no plugin.yaml" — a 502 that names the wrong cause, and a run row left queued forever with no error on it. The checkout is now staged and renamed into place.
  • The status description read no blocking findings (≥verdict) — the shared phrasing assumes a severity floor, while this gate turns on the audit verdict. Now written per verdict.

Lessons for next run ​

  • A green CI image build is not a deployment: iterion's CI separates the build from the finalize job that re-tags :edge. Poll until the published digest changes, then grep the fix inside the pod, and only then launch. I lost a run to trusting a green workflow.
  • The dogfood cost stayed at ~$0.60 for a digest bump with a real two-image trivy delta. The audit is not where the time goes — verify_build is (5 min of the 14).

2026-07-28 — v2.1.0: the classifier was auditing empty diffs, and the gate could never work (no run; defects found by inspection + adversarial review) ​

  • Status: partial — code validated locally and by review; no live cloud run yet (the target repo is not in the forge App's installation scope, which needs an Organization Owner).
  • Versions: bot 2.0.0 → 2.1.0 · iterion PR #306 (7 commits)
  • Method: wiring Vetty onto socialgouv/buildkit-operator's Renovate PRs end-to-end (audit → align → verdict → gate → auto-merge). Deterministic nodes exercised against real fixtures; Revi reviewed the branch.

What the attempt actually found ​

The bot did not fail loudly on this repo — it would have reported "safe" on three of the four open Renovate PRs without reading anything. prepare only recognised package manifests, so a PR moving a Dockerfile digest, a devbox.json pin or a Taskfile.yml tool matched nothing. Crucially it did not stop: is_empty means "no files changed", not "no manifest matched", so the run continued and handed the auditor an empty bump_summary. Verified on the real renovate/golang-1.26 branch: 3 Dockerfiles, a 1684-char diff where the old classifier produced an empty string.

Lesson worth keeping: a scope flag and a coverage flag are not the same flag. Conflating them is what turns "we found nothing to look at" into "we looked and found nothing".

Engine defects this surfaced ​

  • review_on_sync was unreachable — absent from the webhook API request type (a PATCH carrying it returned 200 and changed nothing) and never set by provisioning, so it could only ever be false in production. Since a commit status lives on one SHA, that made every merge gate self-defeating: the status went absent from the head after any push. Observed live on SocialGouv/iterion#300 (20 checks green, PR blocked). Now derived from the declared statuses scope.
  • A PR event could only launch one bot, via a hardcoded fallback, and the shared author allowlist was nil'd as soon as one co-enabled bot was open — so a dependency guard co-enabled with a reviewer was silently dropped along with its author filter.
  • Author routing read the event sender, not the PR author, so a human pushing a fix onto a dependency PR handed it to the wrong bot.

The adversarial review earned its keep ​

Revi found four defects on the branch, two of which would have shipped broken:

  • arm_automerge sent syntactically invalid GraphQL (a sigil swap that also rewrote GraphQL's own separators). The feature was dead, and the test certified it green — the stub answered success to any body. The fix now includes a stub that rejects what the API would, verified by reintroducing the bug and watching the test fail.
  • The ReviewOnSync derivation ran only on a fresh provision, while the already-provisioned repo — the production case — hits the idempotent short-circuit. The fix fixed nothing that was already deployed.

Lesson: a test whose stub accepts anything certifies nothing. Both of those were green in CI and broken in production; the pattern is a test that asserts on what we sent rather than on what a real peer would accept.

Lessons for next run ​

  • Point it at an npm/pypi PR: live validation is still Go/Docker-only.
  • The gate context must be pinned per repo (launch_vars.gate_context), the same value on every bot that can gate — a per-bot required check deadlocks whichever PRs that bot does not review.
  • arm_automerge is only safe on a repo that requires at least one check; with none, a forge merges an armed PR immediately.

2026-07-14 — cloud run on a real Dependabot go-minor-patch PR (#182); excellent audit/align/verify, wired via /vetty command (run 019f60cf) ​

  • Status: VALIDATED (cloud, real forge PR) — Vetty ran end-to-end on a live Dependabot PR (#182, go-minor-patch, 10 modules incl. x/crypto, mongo-driver, aws-sdk, go-selfupdate) and produced a correct verdict. The post_feedback comment step failed the first time on the Anthropic forfait's session rate-limit (failed_resumable); a fresh token + iterion remote runs resume finished it.
  • Versions: bot dep-update-guard v2.0.0 · iterion runner :edge (this session's :edge, digest 42665a30, i.e. incl. #178/#180/#184/#185) · claude_code backend on claude-opus-4-8 via the Anthropic OAuth forfait.
  • Method: wired the bot via POST /api/teams/{id}/forge/repo-bots (bot_ids:[dep-update-guard], GitHub App iterion-forge-61934180[bot], forge_token forge_github_f73ba902) → registers the pull_request + pull_request_comment webhook and the /vetty command. Triggered deliberately with gh pr comment 182 --body "/vetty" (routes only to dep-update-guard; the comment gate checks the commenter's CollaboratorPermission).
  • Result: converged. audit ($0.87) → align (no changes; all minor/patch, no breaking API in-tree) → verify_build ($1.12) wrote an out-of-tree verify.sh (go build -mod=vendor ./... + a vendor-drift go mod vendor + git diff --quiet check) → verify_run gate GREEN → verdict "safe, no alignment, nothing pushed".
  • Value: a genuine supply-chain audit — queried the OSV API per package, correctly identified that the x/crypto bump resolves CVE-2025-58181/47914, flagged a version discrepancy in the PR description, and reasoned correctly about the blocking criteria (not a new HIGH/CRITICAL → don't block). This is the reference "dependency-PR guard" behaviour working on a real PR.
  • Findings / misses:
    • #1 (FIXED, engine) — the skill mirror produces the CC-2.x directory form .claude/skills/<name>/SKILL.md, but Vetty's prompt (and ~8 other catalog bots) Reads the flat .claude/skills/<name>.md. The Read failed twice + cost a filesystem find before recovering from the baked /opt/iterion/bots/... copy — a recovery absent on a non-iterion target repo. Fix: mirrorFileSkill now writes the flat alias too (PR #187, pkg/runtime/bundle.go).
    • #2 (FIXED, security) — the forge integration auto-launched improve/review bots on every PR incl. fork PRs (untrusted code + budget-exhaustion vector) and dependency PRs. Fork PRs are now never auto-launched (a repo collaborator triggers manually via /command, gated on CollaboratorPermission); dep-bot PRs never route to the improve loop (PR #189, pkg/server/webhooks_{github,common}.go).
    • #3 — verify_build is slow (~17 min) on a cold devbox go build; ~$2 total run cost. Acceptable; a shared go/devbox cache (ADR-066-bis) would help.
    • #4 — the Anthropic forfait has a ~5h session rate-limit; a long run + many same-session runs exhaust it and the last node (post_feedback) failed rate_limited, losing an otherwise-complete verdict until resume. #4b (follow-up): make post_feedback a DETERMINISTIC tool node (compose the comment from the structured verdict + POST via the forge REST API) so it needs no LLM turn — resilient to rate-limits, faster, cheaper.
  • Engine hardening: PR #187 (skill mirror flat alias) + PR #189 (fork/dep-bot webhook guards) — both dogfood-driven.
  • Lessons for next run: keep triggering via /vetty (controlled, one bot, authz-gated) rather than the auto pull_request path; provision a FRESH forfait token before a long run (the access token is short-lived and the session limit is real); land #4b so a rate-limit at the comment step can't sink a good verdict.

2026-07-10 — first CLOUD runs on a real Dependabot PR: HOLD verdict with a real CVE finding, then safe re-verdict (runs 019f4ba8 / 019f4bcb / 019f4d3b) ​

  • Status: VALIDATED (cloud, real forge PR) — the two paths the 07-07 bilan asked for both ran live: a real Dependabot PR (#80, go-minor-patch, 22 modules) and the real post_feedback forge POST, verdict comments posted under the App identity iterion-forge-83fde406[bot] with re-fetch verify.
  • Versions: bot v2.0.0 · iterion 499957c31→6dd452c2a (fixes landed mid-session).
  • Method: /vetty PR comment → webhook command route (scope: pr, mode direct) → cloud runner (no sandbox, devbox image). ~$1.6/run, 5 min.
  • Result & value: run 019f4bcb produced an exemplary HOLD: OSV batch over all 30 bumped (name, version) pairs → zero malicious/typosquat; the vendored wails package.json bump audited for lifecycle hooks (devDependencies only); and the real finding — the PR bumps x/crypto 0.50→0.51 while the fix for 7 CRITICAL + 2 HIGH SSH/agent advisories is 0.52.0, one minor short — so the guard's tie-break (unsure → suspicious, a hold is cheap) fired exactly as designed. After Billy pushed the 0.52.0 bump, run 019f4d3b re-audited (osv-scanner v2.4.0 over all 134 pinned packages) and returned the clean ✅ safe/aligned verdict with an honest committed=false, no alignment needed.
  • Findings / misses (engine, not bot): run 019f4ba8 no-op'd (is_empty: true) because the PR-comment command path launched on the DEFAULT branch — the issue_comment payload carries no head ref. Fixed in-session: 499957c31 resolves the PR head/base via the forge API at command time (failure = loud 502, closed PR = filtered). Without local scanners on the runner image the audit adapted (OSV REST batch), matching the skill-not-DSL universality contract; osv-scanner appeared in the later run.
  • Lessons for next run: point it at an npm/pypi dep PR to exercise non-Go ecosystems; consider shipping osv-scanner in the runner-devbox image so the floor doesn't depend on the agent installing it.

2026-07-07 — first live dogfood: clean-bump path end to end, audit evidence exemplary (run 019f3d73) ​

  • Status: VALIDATED (no-sandbox variant, clean-bump path) — every stage behaved with the exact honesty the v2 contract demands; the with-alignment path and a real forge POST remain to be exercised on a live PR.
  • Versions: bot v2.0.0 · iterion dev+239203525cc8.
  • Method: CLI run FROM the PR-branch checkout of a Go fixture (bare origin + dependabot/go_modules/... branch bumping github.com/google/uuid 1.5.0→1.6.0), --sandbox none (sec image blocked by native:221edac8), pr_url empty (no forge), --max-cost-usd 12. ~11 min wall.
  • Result: finished. prepare (deterministic) correctly scoped go.mod+go.sum; security_audit verdict=safe with model evidence: govulncheck actually RUN (no reachable vulns), OSV API queried for BOTH versions with a query-shape control against a known-vuln package, go.sum hashes checked against sum.golang.org's transparency log, and the absent image scanners honestly listed not_available ×3. align: applied=false with proof (NewString() stable; build+vet run) — no invented edits. Deterministic verify_run: real exit 0 (build+vet against the bumped dep). validate_gate stable → commit node: committed=false, "no alignment needed" — the honest no-op. post_feedback skipped (no pr_url, posted=false, never pretended); feedback_health degraded=false.
  • Value: the v2 calibration is vindicated live — the read-only audit stage produced real, verifiable evidence, and the deterministic verify (which replaced the self-reporting v1 validate) gated on a real exit code.
  • Findings / misses: none on the bot. The earlier sandboxed attempt (019f3d56) fell to native:221edac8 (in-container stream zero-byte + subprocess leak) and to an operator docker-exec cleanup that killed the container — recorded there.
  • Lessons for next run: exercise (a) a breaking bump so align/commit actually land code on the PR branch, and (b) a real forge PR so post_feedback's REST + re-fetch verify path runs; both ideally back in the sec sandbox once 221edac8 lands.

2026-07-07 — converted to v2 calibrated shape (ADR-058 fleet rollout) — structural-validated, dogfood pending ​

  • Status: converted, dogfood pending — no live run yet in ANY shape (this file is new); structural validation only (iterion validate clean, catalog tests green).
  • Versions: bot v2.0.0 · iterion worktree branch (rollout of 2026-07-07).
  • Shape: the LLM validate self-report was replaced by the deterministic verify_build/verify_run gate (fail-closed — no verify.sh ⇒ no commit); the read-only security_audit DELIBERATELY stays separate from the mutating align (anti-prompt-injection separation behind a deterministic verdict gate), and commit-after-green stays (shared PR branch). align gained the G5 pre-existing-failure protocol; the dead pr_review_mode var is gone.
  • Next: first live dogfood on a real Dependabot/Renovate PR + bilan here.