Iterion

Appearance

Like what we’re building? Star on GitHub

Cloud — public exposure checklist ​

This page is a pre-flight checklist for opening an iterion deployment to traffic outside your private network (i.e., putting it behind a public Ingress, on the internet, or accessible to users you don't fully trust). Every item below is a hard prerequisite — skipping any of them is how incidents happen.

If you are running iterion only on a private network (VPN, internal cluster, single-tenant lab), this page is informational. The checklist is geared toward multi-tenant or internet-exposed deployments.

This is not a substitute for a full security review. It is the minimum bar.

1. Authentication is enforced on every public route ​

  • [ ] Auth enforcement on in the server config: confirm DisableAuth is not set in the iterion server config (cloud mode requires auth by default; see pkg/config/config.go). There is no AUTH_REQUIRED env var — auth is gated by the DisableAuth config field, which must remain false/unset in any public deployment.
  • [ ] Every /api/* route is gated by requireAuth (verified in pkg/server/server.go). The health endpoints (/healthz, /readyz) and auth bootstrap routes are the only intentional exceptions; both are read-only and reveal no tenant state.
  • [ ] JWT signing key rotated from the chart default. ITERION_JWT_SECRET must be a base64-encoded signing key with at least 32 decoded random bytes (for example, openssl rand -base64 48), held in a Kubernetes Secret, not the values file.
  • [ ] SSO / OIDC wired if you have ≥ 2 users. See cloud-admin.md for OIDC bootstrap.
  • [ ] Super-admin account created with a strong password and 2FA enabled where the IdP supports it.

How to verify: try curl https://<host>/api/runs with no token — must return 401. Try with an expired token — must return 401. Try with a valid token from tenant A reading tenant B's run — must return 404 or 403.

2. Multi-tenant isolation is real ​

  • [ ] All Mongo queries include the active tenant filter. Sample by inspecting the access log: every query line should show a tenantID=<id> matching the JWT.
  • [ ] All blob keys embed the run-id as a top path segment (artifacts/<run-id>/<node>/<v>.json, attachments/<run-id>/..., tools/<run-id>/..., ir/<run-id>.json, runfiles/<run-id>/...) and the run-id is itself a tenant-scoped value. No cross-tenant prefix collision possible. The one family keyed by something else, uploads/<upload-id>/<filename> (an upload staged before any run exists), is reachable only through its staged_uploads record, which is read under the tenant filter; it is never presigned, and it is swept after an hour.
  • [ ] Audit trail requirement is covered for your deployment: the tree ships a dedicated persistent audit-log subsystem (pkg/audit — tenant + platform Mongo TTL store, surfaced at /api/teams/{id}/audit, /api/orgs/{id}/audit — where accepted sign-ins are read — and /api/admin/audit) recording control-plane mutations, every sign-in accepted or refused (auth.login.succeeded / auth.login.failed, every provider, the attempted address only as a peppered digest — ADR-157), API-key issuance (pat.created) and team-membership changes (member.*). Run launches are the second half of ADR-157, not yet recorded. Confirm your retention (Mongo TTL ≥ your compliance window), and know that no setting trusts a forwarded header yet (#2673): every row records the immediate peer's address and port — the ingress pod's behind an ingress — and the login rate limit is per connection, not per client (admin guide §1.5); supplement with server/runner structured logs in your aggregator as needed.
  • [ ] Audit trail requirement is covered for your deployment: the tree ships a dedicated persistent audit-log subsystem (pkg/audit — tenant + platform Mongo TTL store, surfaced at /api/teams/{id}/audit and /api/admin/audit) recording control-plane mutations and every run launch (one run.launched row per launched run — user, webhook or daemon actor, run id, bot, trigger, delivery / schedule id, IP and user-agent when a request served it — and run.forked for a run forked from another). Confirm your retention (Mongo TTL ≥ your compliance window) and that login, API-key issuance, and team-membership changes are covered; supplement with server/runner structured logs in your aggregator as needed.
  • [ ] Admin-only endpoints (/api/admin/*) require both requireAuth AND a role check (requireSuperAdmin).

How to verify: log in as user A on tenant A, take their JWT, try to call /api/runs/<a-run-id-from-tenant-B> — must return 404 or 403, never 200.

3. Network isolation ​

  • [ ] NetworkPolicy enabled on the iterion namespace. Use examples/networkpolicy-egress.yaml as a starting point and trim to the minimum your workflows need.
  • [ ] Egress allowlist denies the public internet by default; only the LLM provider endpoints, package registries (if sandbox.image isn't pre-baked), source forges (if workflows clone repos), and in-cluster data-plane services are reachable. This policy selects the release's app pods (server, runner, pools, backup) — never the per-run sandbox pods, which carry iterion.io/* labels only.
  • [ ] CNI plugin is NetworkPolicy-aware. Calico, Cilium, Antrea, and Weave all enforce; flannel-only clusters silently ignore NetworkPolicy. Verify with kubectl describe networkpolicy … or by observing that kubectl exec into a runner pod cannot reach 1.1.1.1.
  • [ ] Ingress controller terminates TLS with a valid certificate (Let's Encrypt, internal CA, etc.). HTTP→HTTPS redirect enforced.
  • [ ] Per-run sandbox egress is open by default. The CONNECT proxy filters hosts only when the workflow declares sandbox.network with an allowlist or denylist mode (a block without mode: stays open, C312, except on a repo-defined sandbox, where it is completed to an allowlist) — or on a repo-defined sandbox (auto + a found devcontainer.json), where the iterion-default preset applies. The proxy also runs — admitting every public host the workflow's own !host exclusions do not refuse — when the run carries secrets Layer 2 inspects or a gateway credential; the per-run NetworkPolicy follows the proxy, not the mode. There is no deployment-wide setting: a deployment that wants every run filtered has every bot declare such a network: block, and refuses a preset host per workflow with a !host exclusion. While the proxy runs, the per-run NetworkPolicy locks the sandbox pod to the runner (proxy) + kube-dns and the proxied traffic leaves through the runner pod, under the chart's egress policy; with no proxy (no network:, no inspected secret, no gateway credential), no per-run NetworkPolicy is applied and the sandbox pod's egress is whatever the cluster allows. The iterion-default preset names the hosts of the providers the engine routes to by name, z.ai and Moonshot included — see the verbatim list. The two layers do not compose: with networkPolicy.enabled on an enforcing CNI, the chart's Ingress rules admit only the server and metrics ports on the runner pod, and the runner's CONNECT proxy binds an ephemeral port — a sandbox pod cannot reach it, so every run that needs the proxy (allowlist/denylist, repo-defined default, Layer 2, gateway credential) fails at its first egress, and networkPolicy.ingress.extraAllow does not open that port (cloud-deployment). Verify on your cluster which layer you actually run before relying on either.

How to verify: from the sandbox pod of a run whose workflow declares network: {mode: allowlist, preset: "iterion-default"}, attempt curl https://example.com/ — must fail. Attempt curl https://api.anthropic.com/ — must succeed. From the sandbox pod of a run declaring no network:, the first curl succeeds: that is the default posture, not a misconfiguration.

4. Secrets management ​

  • [ ] No secrets in values files. Every secret (Mongo URI, S3 keys, JWT secret, OAuth client secrets, BYOK API keys, OIDC client secret) is held in a Kubernetes Secret or external secret store (Vault, AWS Secrets Manager, etc.) and pulled via valueFrom / external-secrets.
  • [ ] Per-tenant BYOK keys are encrypted at rest in Mongo (envelope encryption, key in a Secret outside Mongo). The plaintext API key is never logged. See cloud-admin.md.
  • [ ] Secret rotation has a documented cadence. JWT signing key, S3 credentials, and OIDC client secret are rotated at least quarterly. The rotation runbook is captured in cloud-deployment.md.
  • [ ] No consumer subscription serves the deployment's production runs. A Claude Pro/Max or ChatGPT Plus/Pro forfait is an individual licence under the vendor's consumer terms; the tree's reading is "development and testing, not automated production" — a paraphrase to check against the current terms, not a quotation (oauth-forfait.md). Production automation runs on API keys (BYOK, platform tier, gateway), Bedrock or Vertex. If the operator keeps a forfait configured anyway (the SocialGouv deployment keeps the Codex one, under legal review), that is a declared choice in its architecture file and privacy policy, not an oversight. Users' personal forfaits for developing their own bots are the licence's own use and are not what this item is about. There is no single switch: the levers, per delivery channel, are in cloud-admin.md §7.

How to verify: kubectl describe pod <iterion-server-pod> | grep -i secret — env vars must reference valueFrom.secretKeyRef, never literal value:. For the forfait item: secrets.forfait.enabled is false in the production values (kubectl get secret iterion-forfait finds nothing, or the runner has no envFrom on it); kubectl get secret iterion-llm -o json | jq '.data | keys' (key names only, never values) lists no CLAUDE_CODE_OAUTH_TOKEN — that Secret is mounted on the runner regardless of the forfait flag; iterion remote admin llm oauth lists no platform-tier forfait; ITERION_FORBID_SUBSCRIPTION_OAUTH=1 is set on the runner if claw/pi nodes must not spend a tenant's Claude subscription; the credential pool is disabled (iterion remote pool policy --enabled=false) or its audience excludes production bots, since a member's lent forfait serves any run in that audience; and the publisher log matches no grep -E 'credentials GRANTED for run=.*\(oauth:(claude_code|codex) ' for production bots — that one line per run names the tier of every forfait it carried (oauth-forfait, org, pool, platform; cloud-admin.md §9) and is the evidence, since nothing refuses a forfait on claude_code. A token in the runner's env never reaches that log, which is why the two Secrets' key names are checked above.

5. Image supply chain ​

  • [ ] Trivy CI runs on PRs, pushes to main, weekly schedule, and post-image builds; the workflow publishes SARIF/results summaries for HIGH/CRITICAL findings but intentionally does not fail the PR or image release by itself. If you require a hard gate, configure branch protection or a code-scanning rule outside the current workflow. See .github/workflows/trivy.yml.
  • [ ] Image references in production values are digest-pinned, not floating tags like :latest, :edge or :v1. Set image.digest: sha256:… (it wins over image.tag, and the chart refuses any other shape); pin runner.image separately when the runner pool has its own image, since it falls back to the shared reference; set image.requireDigest: true so the render refuses any iterion pod image that is not repo@sha256:… (server, runner, pools and the sandbox: auto image — not the helm test pod nor the devStack images); keep the platform setting sandbox.default_image unset or a digest reference, it overrides the env at publish time outside the render guard. Note the form is repository@sha256:… — a digest written into the tag: field renders …iterion:@sha256:…, which only kubelet rejects. See cloud-deployment.md.
  • [ ] Sandbox image is pre-built and digest-pinned. The Kubernetes driver rejects sandbox.build: by design — production deployments reference a CI-built iterion-sandbox-slim:<version> digest.
  • [ ] Cosign / sigstore signatures verified at admission for production. (Optional but recommended; falls outside the chart's scope.)

How to verify: helm template ./charts/iterion -f values-prod.yaml | grep image: — every image string ends in @sha256:….

6. Observability + alerting ​

  • [ ] Prometheus scrapes /metrics on the server. The chart's metrics.podMonitor.enabled switch wires this when prometheus-operator is installed.
  • [ ] OTLP traces exported to a collector if you have one. Set OTEL_EXPORTER_OTLP_ENDPOINT (or the traces-specific OTEL_EXPORTER_OTLP_TRACES_ENDPOINT) on server + runner.
  • [ ] Log format is JSON in production. ITERION_LOG_FORMAT=json set on server + runner; verified by piping a server log line through jq.
  • [ ] Alert rules exist for: /readyz 503 > 1m, /readyz 200 with "status":"degraded" > 5m (a NATS/S3/Valkey outage no longer 503s by design — see probes-and-graceful-shutdown.md — so without this rule it is invisible), NATS queue depth > 100 sustained, runner OOM kills, Trivy failed scan, JWT signing-key proximity to expiry.
  • [ ] Runbook linked from each alert. The runbook is cloud-troubleshooting.md plus alert-specific notes.

How to verify: kill a runner pod; /api/runs/<id>/events should not stream new events (run hangs); the alert for queue-depth or stale-run should fire within its configured window.

7. Probes + autoscaling ​

  • [ ] livenessProbe and readinessProbe wired on server AND runner deployments. Both shipped in the chart; verify they appear in kubectl get deploy -o yaml | grep -i probe.
  • [ ] KEDA ScaledObject drives the runner pool from NATS queue depth. Consumer name is parameterised (runner.keda.consumerName); two parallel deployments do not collide on the same JetStream consumer.
  • [ ] PodDisruptionBudget for the server (chart ships one). Cluster auto-upgrades won't take down all server replicas at once.

How to verify: kubectl get scaledobject -n <ns> -o yaml | grep consumer shows the parameterised name. Drain a runner node; KEDA recreates pods on a remaining node within ~30s.

8. Resource limits + cost guards ​

  • [ ] CPU / memory limits on server + runner pods. Without limits, a runaway claude_code session can OOM the node.
  • [ ] Per-run budget caps enforced via the workflow's budget: block AND a server-side default override (operator can refuse to launch a run with no budget).
  • [ ] Per-tenant launch rate cap set: LaunchRatePerMin on each team, or the platform default ITERION_ORG_DEFAULT_LAUNCH_RATE_PER_MIN; 0 leaves launches unlimited. The launch gate already enforces it as a per-team token bucket, on POST /api/runs, POST /api/runs/{id}/resume and every other launch surface — see quotas-and-limits.md.
  • [ ] Storage quotas on the S3 bucket (lifecycle policy on objects > 90 days unless explicitly retained).

9. Backup + recovery ​

  • [ ] Mongo is a replica set with point-in-time recovery enabled (Atlas, MongoDB Cloud Manager, or in-cluster operator with continuous backup).
  • [ ] S3 bucket has versioning enabled; the off-cluster copy (backup.* for the bucket, plus the Mongo dump CronJob) is what holds the published RPO 24 h / RTO 8 h (cloud-backup.md § Recovery objectives).
  • [ ] NATS JetStream state is not the source of truth — runs and events are durable in Mongo. NATS reseed from scratch is acceptable; document the procedure.
  • [ ] Disaster recovery drill run at least once before public exposure, following the restore drill (objects first, then the base, erasures replayed, a paused run resumed from the restored checkpoint) and logged in the drill log.

10. Documentation + runbooks ​

  • [ ] On-call runbook linked from your alert manager. Minimum contents: how to scale the runner pool manually, how to release a stale lock, how to rotate the JWT signing key, how to disable a misbehaving tenant.
  • [ ] Privacy / data residency policy documented. What data does iterion store about a run? Where? For how long? How can a tenant request deletion? Published as the privacy policy below.
  • [ ] Legal pages published (config.legal.*, cloud-admin.md): legal notice (publisher, publication director, hosting provider), privacy policy (controller, purposes, legal bases, recipients and processors incl. AI providers and transfers outside the EU, retention, rights, DPO, no trackers), terms of use, and a mailto: contact (ITERION_LEGAL_CONTACT_EMAIL) that a named person reads. Texts reviewed by the privacy / legal reviewer below — templates in docs/legal-templates/.
  • [ ] Accessibility statement published (accessibilite document + ITERION_LEGAL_ACCESSIBILITY_STATUS): non_compliant until a full RGAA audit of the studio exists, then partially_compliant with the rate and the date; feedback contact and recourse (Défenseur des droits) in the text.
  • [ ] Incident response plan with named owner, paging escalation, and post-mortem template.
  • [ ] Public status page (or equivalent) committed to. If /readyz 503s, downstream users need a way to know without paging the on-call manually.

Sign-off ​

RoleNameDateNotes
Operator (deploys + monitors)
Security reviewer
Privacy / legal reviewer
On-call manager

A deployment is ready for public exposure when all four sign-offs are present and dated, and every checkbox above is ticked. Half-completed checklists are how incidents are made.

If a box can't be ticked, do not open the deployment to traffic until it can — or accept the risk explicitly, in writing, with a named owner.