Lifecycle & Async Writes
The six atomic operations of memory and how to observe them via the SSE lifecycle stream.
CortexDB keeps heavy derivation off the write path: capture is durable within milliseconds, while extraction and consolidation run asynchronously. Writes are async by default, and you observe progress via a Server-Sent Events (SSE) lifecycle stream.
A write appends an event to the WAL — acknowledgments are fsync'd by default, so an acked event survives
power loss — and can return at the durable capture point in ~10 ms. The heavier work of extracting entities
and consolidating beliefs happens strictly in the background.
The six atomic operations
These are observable lifecycle events, not separate endpoints. (Retrieve is not in the list — recall is a request/response operation.)
| Operation | Fires when | Visible via |
|---|---|---|
| Capture | WAL append succeeds | event: captured; wait=captured returns |
| Index | Event indexed in BM25 + HNSW | event: indexed; wait=indexed returns |
| Update | Fact ADD / UPDATE / NOOP decision | event: extracted / event: consolidated |
| Consolidate | Beliefs revised; Understanding touched | event: consolidated; wait=consolidated returns |
| Forget | Records deleted | event: forgotten |
| Compress | Episodes sealed; Understanding versions bumped | event: compressed |
Choosing a write barrier — ?wait=
Pass ?wait= to POST /v1/experience to pick the stage the connection is held for:
| Value | Returns when | Typical latency |
|---|---|---|
| (omitted) | at capture — status captured (202). Read-your-writes holds: /v1/recall sees the event immediately, but BM25/HNSW indexing may still be in progress. | ~10 ms |
captured | Durable accept point (WAL fsync'd) — same as omitted, made explicit | ~10 ms, flat in payload size |
indexed | BM25 + HNSW insert confirmed | ~100–500 ms |
consolidated | Beliefs / Understanding touched | ~500–3000 ms |
Omitted ≠ fully indexed
Omitting ?wait= returns at captured, not "fully indexed" (a common mislabel in older docs). The
event is durable and immediately visible to recall, but vector/BM25 search may lag briefly. If you need the
event to be search-indexed before you return, pass ?wait=indexed. accepted is an alias for the
omitted default. A waiting capture has one budget of about 30 s (a waiting bulk,
min(30 s + 1 s per item, 600 s)); when it runs out the server answers 408 WAIT_TIMEOUT and the
event keeps processing — see
Experience.
Subscribing to the lifecycle stream
The recommended pattern for UIs/agents is fire-and-forget writes + an SSE subscription:
GET /v1/lifecycle/stream?scope=org:acme/dept:eng&events=indexed,consolidated
Accept: text/event-stream
Authorization: Bearer ...
X-Cortex-Actor: user:alice
event: indexed
data: { "event": "indexed", "scope": "...", "data": { "event_id": "evt_01HX...", "layers_indexed": ["events"] } }Use the Last-Event-ID header (or ?since_lifecycle_id=<lce>) to resume after a disconnect — the server
replays missed events.
Async jobs
Lifecycle events cover async jobs too:
| Job | Events | Status endpoint |
|---|---|---|
| Bulk write | captured per item + aggregate | — |
| Import | import_progress, import_complete, import_error | GET /v1/import/{id} |
| Erasure | none; poll the status endpoint | GET /v1/erasures/{id} |
| Synthesize | synthesis_progress, synthesis_complete | GET /v1/understanding/synthesize/{id} |
| Export | none — POST /v1/export is synchronous and returns the data inline | — (GET /v1/export/{id} is 404) |
A lagging event fires when the consolidator queue depth exceeds a configurable threshold — surface it to
detect index drift.
FAQ
How fast is a write? ?wait=captured (or omitted) returns at the durable accept point in ~10 ms, flat
in payload size. Pass ?wait=indexed to additionally block until search indexes have the event.
Does CortexDB include Retrieve in the lifecycle? No — retrieve is a synchronous request/response, not a background stage.
Why async by default? So the agent stays responsive while extraction/consolidation run in the background instead of blocking on an LLM.
See also
- The Experience Envelope — the payload that enters the lifecycle.
- The Five Memory Layers — where data lands after consolidation.