Flow — memory and knowledge: four writers, one seam, two adapters

A memory space is a quota-governed tree of Markdown documents addressed by a visibility and the qualifiers that visibility needs, and the four surfaces that write one — the REST routes, an agent's memory tool, the auto-memory mirror folding a node's notes back, and the local CLI import — meet at a single store seam, each carrying a different share of the refusals a byte passes on its way in.

Drawn from the code at commit f564846d3. No gate compares this page's prose with the code: the commit above is the claim a redraw must re-verify by hand. Conventions.

A write, from the four writers down to the bytes

The surfaces that write a space meet at knowledge.MemoryStore.WriteDocument, which two adapters implement: Mongo in cloud mode, the filesystem everywhere else. Three of them carry refusals of their own; iterion memory import is a fourth, entering at the seam with nothing above it but the per-entry checks of ImportSpace. The refusals before the seam belong to the caller, the ones after it to the adapter — and the two adapters do not refuse the same things.

The diagram lives on the feature page: a write, end to end.

Steps

Actor	Action	Code
studio	Team page, Memory tab: one Load fetches usage and the document list, opening a row reads its body, and the export link streams the archive. The tab is read-only — no client writes, deletes or imports — and bot with no bot id is refused locally before any request	MemoryTab, getMemoryUsage, listMemoryDocuments, readMemoryDocument, memoryExportURL
server	refuses a state-changing /api/ request whose Origin the deployment does not recognise, before any credential is read — the repository's one CSRF boundary, which on this surface means PUT /doc, DELETE /doc and POST /import	authMiddleware, originGateAllows, requireSafeOrigin
server	resolves the bearer, then stamps the identity's TeamID as the store tenant — the only source of tenancy a memory space has	requireAuth, stampAuthedContext, tenantFreePrefixes
server	builds the SpaceRef from the query plus the identity: there is no ?user=, so a user space is always the caller's own, and global is forced tenant-less	memoryRef, ResolveSpaceRef, SpaceRef.Validate
server	gates the one cross-tenant space on IsSuperAdmin, on PUT, DELETE and POST /import only	requireMemoryWriteAuth
server	validates the document path at the boundary, with the same function both adapters re-run, then reads at most the compiled 2 MiB plus one byte of the body — io.LimitReader reports EOF at the bound, so an over-size body reaches the store truncated rather than refused	memoryDocPath, knowledge.ValidateDocPath, handleMemoryWriteDoc
store, cloud	the ordered write: tenancy, the per-document cap, the space row, the two byte counters under conditional updates, then the revision compare-and-swap with counter compensation on a lost race	MongoMemoryStore.WriteDocument, ensureSpace, bumpTenant, bumpSpace, bumpCounter, compensate
store, cloud	a delete reads the size in the same atomic operation that removes the document, so a racing write cannot make the credit stale; an absent document is a 204, and a failed compensation only logs that used_bytes now drifts	MongoMemoryStore.DeleteDocument, clampUsage
store, local	one coarse cross-process lock, a per-space JSON sidecar and one tree-wide aggregate sidecar, all beside the space tree rather than inside it	FSStore.WriteDocument, acquireQuotaLock, checkCeilings, commitDelta
server	export walks the index, reads every body and refuses a literal credential — after the gzip headers are already set; a legacy non-canonical path is listed as unaddressable instead of aborting the archive	handleMemoryExport, knowledge.ExportSpace, ScanForSecret
server	import keeps only docs/ entries, caps each at 2 MiB, scans each, and stamps UpdatedBy as import rather than the operator	handleMemoryImport, knowledge.ImportSpace
cli	iterion memory du, export and import are hardwired to the local filesystem store, so import reaches WriteDocument with none of the REST refusals above it; iterion remote memory is the HTTP client over the same seven routes	memoryRefFromFlags, memory.DefaultFSStore, knowledge.ImportSpace, remoteMemoryCmd
engine	a memory: block resolves the node's space — structured from the visibility, else LegacyBotRef on the repo root — then installs the index block, the autoload block, the tools and the pre-compact hook	applyMemorySpec, installWorkspaceMemory
bot	reads and lists behind read:, writes behind write: stamped bot: plus the bot id; before a compaction the index and autoload set are re-read and prepended as an injected user turn	memoryReadTool, memoryListTool, memoryWriteTool, memoryPreCompactInjector
runner	auto_memory: resolves the reserved auto-memory bot space, guards the per-space leaf against a symlink, gives each node its own mirror directory and materialises the space into it	applyAutoMemory, autoMemoryLocation, PrepareStateRoot, NewNodeDir, Mirror.Hydrate
runner	the deferred sync folds the node's edits back and deletes only what the walk actually saw; every body first passes the run's secret unmaterializer, so both the untouched comparison and the secret scan run on placeholders rather than on materialised values	Mirror.SyncBack, Mirror.SetUnmaterialize, walkPaths, readMirror
ops	the nightly org purge hard-deletes the memory rows of every team of an org past its grace	Purger.PurgeOrg, teamScopedCollections
boot	the two memory indexes are created idempotently at start, with the other schemas	MongoMemoryStore.EnsureSchema

The auto-memory round trip

One space, one mirror directory per node, and a diff on the way back: an untouched space costs no write at all.

State	Vocabulary, and what holds it	Entered by, left by
absent, revision N	memDoc.Revision — absent reads as 0, and the composite _id is the only uniqueness the collection has	a CAS upsert filtered on the revision just read inserts 1 or advances to N + 1; a delete returns it to absent; no TTL exists, so a document lives until something deletes it
space row absent, present, purged	memSpaceDoc	ensureSpace upserts it on the first write with the visibility's default sub-cap; UsageBytes on an absent row answers 0 against that default without creating it; the org purge removes it
within cap, denied, drifting	used_bytes against quota_bytes, on memory_spaces and memory_tenant_usage	a positive delta applies only if the conditional update matches, and quota_bytes 0 means unlimited for a space; a denial rolls the other counter back; a non-positive delta always applies and clampUsage resets a negative counter to 0; nothing recomputes a counter from the documents
off, refused, hydrated, synced, released	this page's names for the mirror round trip: automemory.Mode has off and on and contributes only off here, while Mirror carries no phase field — its one real marker is a hydrated map that stays nil until Hydrate succeeds, which is what SyncBack refuses on	resolved per node from the precedence chain; every failure is a warning, never a failed node
imported, skipped, renamed, replaced	knowledge.ImportStrategy, per archive entry	absent is imported; existing plus skip, the default, is skipped; existing plus rename is written beside it and counted in both renamed and imported; existing plus overwrite — or any value the switch does not recognise — is replaced
active, pending deletion, purged	identity.Org status, as it governs memory data	a soft delete sets the grace; the nightly sweep past it hard-deletes memory_docs and memory_spaces for each team; a restore before the sweep makes the purge a no-op

Guards that refuse

• originGateAllows — 403 cross-origin request rejected, the repository's single CSRF boundary, taken in authMiddleware ahead of requireAuth. It covers the three state-changing routes and none of the four reads; an absent Origin passes, a non-browser caller having no ambient credential to ride; ITERION_REQUIRE_ORIGIN=0 disables it for a rollback without a redeploy, and the same check is also callable per handler as requireSafeOrigin.
• requireAuth — 401 authentication required when no bearer reaches it at all, token expired or token invalid for a JWT that fails, and the PAT store's own text (token revoked or expired, token owner unavailable) for an iap_ one. A ?ticket= is never accepted here: isWSTicketPath admits /api/ws, /api/ws/* and a run's /browser/cdp only, so a ticket on a memory route falls through to the bearer path and 401s.
• stampAuthedContext — 403 credential has no active team, because /api/memory is deliberately absent from tenantFreePrefixes. There is no canViewTeam or canViewOrg anywhere on these routes: the space is derived from the token's active team, never from a path id, so there is no membership probe to pass.
• SpaceRef.Validate — 400 invalid space: a name that is not a single segment, an unknown visibility, a missing bot or project for the visibilities that need them, and any qualifier containing / or \, or that is . or .. — the dot is not in the cutset, so ?project=my.app validates. The clamp is what keeps an untrusted ?project= out of scopeFor.
• requireMemoryWriteAuth — 403 writing the global memory space requires super-admin, on PUT /doc, DELETE /doc and POST /import only. Reading and exporting a global space stay open to any authenticated caller, and no audit row and no log line records either outcome.
• knowledge.ValidateDocPath — 400 from ErrInvalidDocPath: relative, no NUL byte, no .. segment, not the bare ., and canonical — one document, one spelling, which is what makes the two adapters agree.
• validateCloudTenant — a store error, requires a tenant in cloud mode, for any non-global ref with no tenant. A legacy memory: scope: block resolves to exactly such a ref, so that form fails the node in cloud mode.
• The per-document ceiling — a QuotaError the handler renders as 422. The cloud adapter uses the compiled 2 MiB; the filesystem adapter honours ITERION_MEMORY_MAX_DOC, and the cloud one honours no ITERION_MEMORY_QUOTA_* variable at all. That override widens the store, not the REST surface: PUT /doc reads at most the compiled 2 MiB plus one byte and never reports the cut, so against an adapter cap raised past 2 MiB an over-size body is stored truncated under a 200 instead of being refused — ImportSpace, on the archive path, compares each entry against the same cap and errors instead.
• The two byte ceilings — 422 carrying used, delta and quota. The aggregate is skipped when the ref has no tenant, so a global write in cloud mode passes no aggregate ceiling, while the filesystem adapter still charges its tree-wide sidecar: the two disagree on the same space.
• The revision CAS — memory: revision conflict when the caller pinned ExpectedRev, otherwise a retry bounded only by the context, each round with its counter bumps compensated. The filesystem adapter ignores ExpectedRev and always reports revision 0.
• The quota lock — memory: quota lock busy after 5 s, on the local adapter only.
• ErrUnsupportedVisibility — private has no filesystem home, while UsageBytes on the same ref validates and answers, so one ref gets two different verdicts on a local server.
• Export and import — a literal credential is *ErrSecretInExport, but the export handler has already set the gzip headers and only logs, so the client receives a successful response with an empty body; an import refuses an unsafe tar path and any entry over 2 MiB, and never validates ?strategy=.
• The compiler — C170 for an unknown visibility: and C171 for visibility: together with the legacy project_root: are errors; C047 for a memory: block on a backend other than claw and C132 for auto_memory: on a backend that ignores it are warnings.
• Status shape — a write maps every store error to 422, a document read maps every one to 404, and a delete, a document listing and a usage read map theirs to 500: the same refusal reaches the client as three different codes depending on the verb.

Collections touched

memory_docs (bodies stored inline, written by the three writers that reach the configured store through the server or the runner — the local CLI import writes the filesystem twin instead — and deleted explicitly or by the mirror's inferred delete) · memory_spaces (upserted on the first write, carrying the per-space sub-cap and its running counter) · memory_tenant_usage (the enforced aggregate counter, whose _id is the team id — it carries no tenant_id field, so the org purge's field filter cannot match it) · orgs (MemoryQuotaBytes, read to compute the ceiling) · teams (enumerated to propagate that ceiling per team, and again by the purge) · audit_events (org.updated from the admin quota change — no memory mutation writes one).

Without Mongo the same three live on disk: the space tree under the global data dir, plus .quota/spaces/<checksum of the ref>.json and one .quota/aggregate.json.

Related

• memory-and-knowledge.md — the feature's home page: the visibility table, the two quota levels, the REST surface, export and import, the CLI, how bots use the tools, and the whole auto_memory: mechanism.
• data-model-knowledge.md — memory_spaces, memory_docs, memory_tenant_usage and their edges, with the key lines and the filesystem twins.
• quotas-and-limits.md — where the memory ceiling sits among the other quotas, and the org usage view that reports it.
• cloud-rest-api.md — the seven routes in the API reference; dsl.md and settings-precedence.md — the memory: block and the auto_memory: precedence chain.
• flow-org-team-lifecycle.md — the soft delete and the nightly purge that takes these rows; data-model-identity.md — orgs, teams, audit_events.
• access-control.md — the principal kinds and the gate matrix these routes sit in.