Scheduling recurring bot runs (iterion schedule)

Some bots are meant to run on a clock, not on demand — a weekly security audit, a nightly docs-refreshment pass, a periodic dependency sweep. iterion schedule wires those into the host's own cron so they fire on time without keeping an iterion process resident. This is the complement to iterion dispatch: the dispatcher is an always-on loop reacting to a tracker; the scheduler is a set of cron triggers reacting to the clock.

Model

A declarative manifest is the single source of truth. schedule install materialises it into a managed block of the host crontab; each cron line calls iterion schedule run <name>, which re-reads the manifest and executes the run in-process. Because the host scheduler is the trigger, nothing iterion needs to stay running between firings.

~/.iterion/schedules.yaml          # the manifest (override with --manifest or $ITERION_SCHEDULES_FILE)
~/.iterion/logs/schedule-<name>.log  # per-schedule stdout+stderr, appended each run
host crontab                       # a managed block, markers below, regenerated by `schedule install`

The manifest is host-wide (a host has one crontab), and every entry carries its own workdir, so a single manifest can schedule bots across several repositories.

Manifest format

version: 1
schedules:
  - name: sec-audit-source-weekly      # unique; used in the crontab line + log filename
    cron: "0 2 * * 1"                   # standard 5-field expression, passed opaquely to host cron
    bot: bots/sec-audit-source/main.bot
    workdir: /home/jo/lab/ai/iterion    # cd here before running; bot path resolves against it
    store_dir: ""                        # optional override; empty uses project-store resolution
    sandbox: ""                          # optional --sandbox override (none|auto)
    timeout: "2h"                        # optional max run duration (guards a hung run)
    vars:                                # optional --var overrides (commas kept verbatim)
      label_source: sec-audit-self
    description: Weekly SAST self-audit  # optional; emitted as a crontab comment
    disabled: false                      # keep in the manifest, leave out of the crontab
    overlap: skip                        # optional: skip (default) | allow | keepalive — see "Overlap policy"
    max_concurrent: 0                    # optional, with overlap: allow — cap on live runs (0 = unlimited)
    guard: ""                            # optional pre-launch sh -lc gate — see "Guard command"
    guard_timeout: "30s"                 # optional guard subprocess timeout
    guard_var: guard_output              # optional var name receiving the guard stdout

With an empty store_dir, the run uses the normal resolution anchored at the bot path: an existing managed project .iterion, otherwise the deterministic slot under $ITERION_HOME/projects/ (normally ~/.iterion/projects/). Set an explicit <workdir>/.iterion when scheduled runs must appear in a studio bound to that workspace store.

Commands

Command	What it does
iterion schedule add <name> --cron … --bot … [--workdir …] [--var k=v]… [--store-dir …] [--sandbox …] [--timeout …] [--description …] [--disabled] [--overlap skip|allow|keepalive] [--max-concurrent N] [--stale-after 5m] [--guard …] [--guard-timeout 30s] [--guard-var guard_output]	Add or update an entry (upsert by name). Overlap, keepalive (--stale-after), and guard flags mirror the manifest fields — see "Overlap policy", "Always-on agents", and "Guard command".
iterion schedule list [--json]	List manifest entries.
iterion schedule remove <name>	Delete an entry from the manifest.
iterion schedule run <name> [--dry-run]	Execute one entry now — what cron invokes. Applies the overlap policy and the guard before launching. --dry-run prints the resolved iterion run command without executing.
iterion schedule audit [--name X] [--surface host-cron|trigger|cloud] [--since 24h] [--tail 50] [--json]	Show the tick-decision history: every fired / skipped / guard outcome with its reason — the deterministic answer to "why didn't my scheduled bot fire last night?".
iterion schedule install [--print] [--tz UTC]	Sync the manifest into the host crontab. --print renders the block to stdout without touching the crontab (works even where crontab is absent).
iterion schedule uninstall	Remove the iterion-managed block from the host crontab (manifest left intact).

--manifest <path> is available on every subcommand.

add/remove only edit the manifest — run schedule install afterwards to push the change into the crontab.

The managed crontab block

schedule install reads the current crontab, replaces (or appends) the block delimited by:

# >>> iterion schedules (managed by `iterion schedule install`) >>>
…
# <<< iterion schedules <<<

User-authored crontab lines outside the markers are preserved untouched, and re-installing is idempotent (the block is replaced, never duplicated). Two details make scheduled runs actually work in cron's minimal environment:

• CRON_TZ=UTC (override with --tz) — honoured by cronie/Vixie cron so schedules fire in the intended zone regardless of host local time; on other cron implementations it is a harmless env var, so verify the firing time against your cron's timezone semantics.
• PATH=<install-time PATH> — captured when you run schedule install, so docker, git, and any scanner binaries the bot needs are reachable from cron.

A rendered block looks like:

# >>> iterion schedules (managed by `iterion schedule install`) >>>
# Managed by iterion — edit the manifest then `iterion schedule install`.
# Remove with `iterion schedule uninstall`.
CRON_TZ=UTC
PATH=/usr/local/bin:/usr/bin:/bin:…
# Weekly SAST self-audit
0 2 * * 1 cd /home/jo/lab/ai/iterion && /usr/local/bin/iterion schedule run sec-audit-source-weekly --manifest /home/jo/.iterion/schedules.yaml >> /home/jo/.iterion/logs/schedule-sec-audit-source-weekly.log 2>&1
# <<< iterion schedules <<<

Example — weekly self-audit

iterion schedule add sec-audit-source-weekly \
  --cron "0 2 * * 1" --bot bots/sec-audit-source/main.bot --workdir "$PWD"
iterion schedule add sec-audit-deps-weekly \
  --cron "0 3 * * 1" --bot bots/sec-audit-deps/main.bot --workdir "$PWD"

iterion schedule run sec-audit-source-weekly --dry-run   # sanity-check the resolved command
iterion schedule install                                  # write the crontab block
crontab -l                                                # verify

The audit bots label their findings source:sec-audit-self on the native board (see Security in CLAUDE.md). They pin the iterion-sandbox-sec image via sandbox.image, so the host needs that image present (CI publishes it; for a local loop, docker tag your build to ghcr.io/socialgouv/iterion-sandbox-sec:edge).

Note: sec-audit-source (SAST) is production-ready. sec-audit-deps (SCA) now has a real CVE floor — run_generic_heuristics runs trivy fs --scanners vuln over the workspace from a bare checkout, matching pinned versions against the OSV/GHSA/NVD DB. The per-ecosystem npm/pip-audit and code-pattern/typosquat malware signals remain partial, so a run still self-labels with a "⚠ Coverage" banner — but it is no longer a zero-finding scaffold.

Overlap policy — skip is the default (behavior change)

Since the schedgate integration, iterion schedule run does not fire while a previous run of the same schedule is still live (any non-terminal status: running, queued, paused_*). Before, every tick launched unconditionally — a nightly bot that overran its window silently piled up concurrent runs on the same repo. Skip-by-default fixes that latent bug; the trade-off is deliberate and loud:

• Every decision is recorded in the tick audit (<manifest-dir>/logs/tick-audit.jsonl, read with iterion schedule audit). A skipped tick writes skipped_overlap with the blocking_run_id, so you find out which run held the slot instead of wondering why nothing ran.
• A stale run stuck in running (crashed host) blocks its schedule until it is reconciled or cancelled. iterion does not guess a staleness cutoff: the audit names the run — inspect it, then iterion resume or cancel it. The studio/server's orphan reconciliation flips flock-releasable orphans automatically.
• Opt out per entry with overlap: allow (unbounded) or overlap: allow + max_concurrent: N (fire while fewer than N runs of this schedule are live, counting the one about to start).
• After upgrading, watch iterion schedule audit --since 48h once: a schedule that used to rely on implicit overlap shows up as skipped_overlap there.

Overlap counting keys on run provenance: schedule-launched runs are stamped source.kind: schedule + source.schedule_id in run.json, which also makes them attributable in the studio and queryable via the store.

Always-on agents — overlap: keepalive

overlap: keepalive runs a bot continuously: every tick relaunches it, but with at-most-one-live semantics and a staleness cutoff. It is how you keep an agent (a watcher, a poller, your own long-lived bot) alive as a stream of fresh, individually-budgeted runs rather than one immortal run that fights max_duration, per-node deadlines, and the cloud budget ceiling.

The only difference from skip: a run that is silent past stale_after (default 5m; a running run whose last progress is older than the cutoff) stops counting as live, so the next tick relaunches a fresh run and the zombie is reaped (running → failed_resumable, so it is still inspectable/resumable). This closes the "crashed run stuck in running blocks the schedule forever" gap above — a dead always-on agent recovers on its own within one tick.

# schedules.yaml — an always-on watcher, relaunched every minute,
# recovered if it goes silent for 2 minutes.
- name: watcher
  cron: "* * * * *"           # host crontab floors at 1 minute
  bot: bots/my-watcher/main.bot
  overlap: keepalive
  stale_after: 2m

• Sub-minute cadence is not expressible on the host crontab (its floor is 1 minute). For a faster pulse, run the resident scheduler (iterion studio / iterion server) and author the bot with a kind: keepalive invocation carrying an interval: (see below) — the in-process scheduler ticks every 15s by default (ITERION_SCHEDULER_INTERVAL).
• max_concurrent is invalid with keepalive (at-most-one-live is the point).
• stale_after should be ≥ the bot's max_duration so a long-but-alive run is never falsely reaped. Staleness only ever applies to running runs — a paused_waiting_human run is legitimately idle and never reaped.

Authoring an always-on bot (kind: keepalive)

A bot declares its always-on capability in its manifest.yaml; the cadence lives on the invocation (not baked into the bot's logic):

invocations:
  - kind: keepalive
    keepalive:
      interval: 30s            # >= 5s; sub-minute needs the resident scheduler
      stale_after: 5m          # optional (default 5m)

Enable it one-click from the bot's studio home (the /bots/{name}/triggers/from-invocation route), or toggle Always-on on a schedule in the studio Schedules tab. To also steer each launched run mid-flight (correct it, nudge it), give the bot a supervisor: block — the engine auto-attaches a supervisor coordinator to every keepalive run, no extra wiring.

Guard command — fire only when there is work

guard: is an optional sh -lc snippet executed before any launch, in the entry's workdir, with ITERION_SCHEDULE / ITERION_SCHEDULE_BOT in the environment and a hard guard_timeout (default 30s) on its own context:

• exit 0 → the run fires, and the guard's stdout becomes the run's vars[guard_var] (default guard_output) — a cheap way to hand the run "the work found" (an issue list, a diff summary).
• exit non-zero → the tick is skipped (guard_blocked in the audit, exit code + stderr tail captured).
• guard breaks (spawn failure, timeout) → guard_error in the audit — deliberately distinct from guard_blocked, so "the guard said no" never masks "the guard is broken".

Example — only run the fixer when ready-labeled issues exist, and pass them in:

- name: fix-ready-issues
  cron: "*/30 * * * *"
  bot: bots/feature-dev/main.bot
  workdir: /home/jo/proj
  guard: 'out=$(gh issue list --label ready-for-agent --json number,title); [ "$out" != "[]" ] && printf %s "$out"'
  guard_var: issues_json

Idempotence stays with the workflow (mutate the state you poll — e.g. the bot relabels the issue), exactly like the dispatcher's tracker loop: the guard is a gate + input source, not a dedup engine.

Retry — a provider quota window is waited out, not re-attempted

A scheduled run that dies because the LLM provider's quota window is exhausted (the Anthropic forfait 5h / session / daily / weekly caps) is not a failure of the run: nothing about it is wrong, it just arrived while the door was shut. Retrying it inside a node's retry budget, or handing it back to the work queue, cannot help — a weekly reset can be seven days away, and each attempt costs a fresh pod against a wall that will not move.

So iterion waits for the reset and resumes, and the wait is durable (cloud mode: the intent lives in the run document, a server-side sweeper acts on it). The schedule itself is unaffected: it keeps firing on its own cadence, and the retry is about the run that already failed.

retry:
  usage_window: resume   # resume (default) | off
  max_attempts: 5        # over the run's WHOLE lifetime; never reset
  max_wait: 192h         # cap on how far ahead a retry may be scheduled (8d)
  jitter: 10m            # spread runs that share one reset instant

Where to declare it. The same block is accepted on four layers, and each one overrides only the fields it actually sets:

priority	layer	where
1	per-run override	launch API / CLI
2	launching surface	a cloud schedule row, a trigger subscription, a webhook config, a schedules.yaml entry — each exposes the same four retry_* fields
3	the bot	retry: in the bundle's manifest.yaml
4	machine default	ITERION_RETRY_USAGE_WINDOW / _MAX_ATTEMPTS / _MAX_WAIT / _JITTER

A platform ceiling (ITERION_CLOUD_RETRY_MAX_ATTEMPTS, ITERION_CLOUD_RETRY_MAX_WAIT) is applied last and can only lower a resolved policy, so a tenant cannot reserve a hundred attempts over thirty days on their own schedule.

Local vs cloud — the wait is held differently. In cloud mode the intent is durable, so nothing extra is needed. On a host-crontab schedule the wait is held in-process by the bounded auto-resume loop, which stays opt-in: the retry_* fields on a schedules.yaml entry shape that wait, they do not enable it. Add --auto-resume N (or ITERION_AUTO_RESUME) to turn it on. The reason it is not on by default is that a local iterion run blocks your terminal, and silently sleeping it for 33 hours is not a default anyone would want.

Choosing a value. The question a bot author is answering is is this output still worth having late? A weekly digest is (resume); a "what changed in the last hour" report is not (usage_window: off — let the next tick produce a fresh one). max_wait bounds one scheduled sleep, not the run's total lateness: if the parsed reset is farther away, iterion schedules the retry at that horizon and re-evaluates when it wakes. For example, 36h means "wake no later than 36 hours from now and check again"; max_attempts is the separate whole-lifetime bound on repeated waits.

What you see. The run stays failed_resumable while it waits, with a run_retry_scheduled event naming the instant, the attempt number and how the instant was derived. When the window reopens the run emits run_auto_resumed and continues from its checkpoint — the same pair the CLI's in-process --auto-resume loop writes, so the timeline reads identically local and cloud. A run that stops retrying records why (budget spent, admission denied, source no longer resolvable) rather than going quiet.

Not covered by this: a budget cap (max_cost_usd and friends) still needs a human to raise the cap and resume — retrying the same cap would re-fail instantly. Nor an auth failure, which is a credential problem time does not fix.

Retention — pair recurring schedules with iterion runs prune

Every scheduled run persists forever under <store-dir>/runs/<run_id>/; the store has no built-in retention. A weekly bot alone adds ~50 run directories a year, and a fleet of hourly/daily bots pushes that into the low thousands per month. Cap disk usage by running iterion runs prune on its own crontab line — the schedule manifest only runs bots, not arbitrary commands, so this one belongs outside the managed block:

# Weekly retention sweep — keep the last 100 runs and prune anything
# finished/failed/cancelled that is older than 30 days.
30 3 * * 1 /usr/local/bin/iterion runs prune --store-dir /path/to/workspace/.iterion --older-than 720h --keep-last 100 >> "$HOME"/.iterion/logs/runs-prune.log 2>&1

Flags mirror the semantics of the shipped statuses — see iterion runs prune --help for the full list. --dry-run is the safe way to preview what a candidate retention policy would delete before committing to it in the crontab. The command only removes <store-dir>/runs/<id>/ directories; it never touches <store-dir>/worktrees/ or anything else.

failed_resumable runs are excluded by default (they are recoverable); opt in with --status finished,failed,cancelled,failed_resumable when you have accepted their loss.

Notes & limits

• Cron expressions are passed through verbatim. iterion only checks the field count (5); range validity is the host cron's job.
• Host cron only. schedule install/uninstall shell out to crontab. On a host without it (e.g. a minimal container, Windows), use schedule install --print and wire the block into whatever scheduler you run (systemd timer, Task Scheduler, a CI cron). The manifest + schedule run <name> work everywhere.
• No TTY at firing time. Scheduled runs are non-interactive; a bot that pauses for human input will pause and persist a resumable checkpoint (resume it later with iterion resume), not block the cron job.

Implementation

• CLI wiring: cmd/iterion/schedule.go
• Logic + manifest + crontab splice: pkg/cli/schedule.go
• Tests (manifest round-trip, splice idempotency, install/uninstall via injected crontab seam, dry-run resolution): pkg/cli/schedule_test.go