Changelog

DATA-SAFETY: selective erasure fix (upgrade advised for anyone using /v1/erasures)

• POST /v1/erasures erased the ENTIRE scope, ignoring the selector. The request carried no selector field, so a caller-supplied selector.memory_ids was silently dropped and the whole scope was permanently, irreversibly destroyed — returning 202. An empty, absent, or typo'd selector all wiped the scope, on the very endpoint operators are told to use for GDPR erasure requests. /v1/erasures now honors selector.memory_ids (erasing ONLY the named records and their derived data), requires confirm_all: true for a deliberate whole-scope erasure (matching /v1/forget), rejects a typo'd selector key with 422, and reports the true deleted count. A GDPR request targeting one subject no longer destroys the tenant.

• Cluster mode silently disabled ~half the API. The distributed coordinator implemented only 8 of 75 storage methods and fell through to inert defaults for the rest — so on a multi-node deployment, layer reads returned empty for locally-owned data, fact/selective deletion no-op'd, GDPR forget-tombstones were not durable (erased events resurrected on restart), usage/billing reported zero, and the index-audit always reported "clean". All storage operations now execute against the local engine; a delegation-completeness test prevents recurrence.

• Back-compat: top-level metadata is accepted again. Pre-v0.8.0 clients that tagged events with a top-level metadata: {labels: [...]} object were hard-broken by the v0.8.0 strict envelope (422). That shape is accepted again and folded into context.labels. New clients should write context.labels directly.

• Strict unknown-field rejection now covers /v1/recall, /v1/forget, the bulk request envelope, and budgets — a typo'd top-level key (e.g. temporl for temporal) returns 422 instead of silently dropping the temporal pin and answering with current-time data.

• directives rejects unknown keys and validates ttl_for_belief_layer as an ISO-8601 duration (a typo no longer silently skips the directive).

• Triple predicate is validated (non-empty, length-bounded) like subject/object ids.

• CORTEX_EXTRACTION_BATCH_EVENTS startup logging corrected — the knob was always wired, but the log printed an unrelated fixed constant, making it look dead.

SECURITY: cross-tenant authorization fix (upgrade advised for token-auth deployments)

• Cross-tenant authorization on scope-supplied routes. Several endpoints acted on the caller-supplied scope with no membership check, while their read/write siblings (recall, events, by-id reads) correctly enforced scope-membership roles. On deployments using per-actor token auth (PASETO/JWT via the minter), any authenticated caller could target another tenant''s scope — one that tenant had written to (and thus auto-registered) — to delete/erase its data (/v1/forget, /v1/erasures), export its memories (/v1/export), take over the scope by rewriting its member roster (PUT /v1/scopes/members), delete/enumerate registrations (/v1/scopes, /v1/scopes/prune), revoke another caller''s token (/v1/auth/revoke), or read cross-tenant fact timelines (/v1/facts/timeline). All of these now enforce the same owner/reader membership checks as the rest of the surface. Deployments running with auth disabled, or with only the static operator key, were never exposed (the operator key is a deployment-wide secret by design, not a per-actor credential). Token-auth deployments should upgrade.

• docker run -v ./data:/data failed at first boot with Permission denied (a host bind-mount is root-owned while the server runs as uid 1000). The container now chowns the data directory at startup (via a privilege-dropping entrypoint) so the documented Docker quickstart works out of the box. docker run --user <uid> continues to work unchanged.

• Ingestion connectors (Slack, Jira, Notion, and every connector using the shared framework) were broken by the v0.8.0 strict write envelope: they emitted context.thread_id / meta / preceded_by, which now return 422 and stalled the sync cursor. This metadata now rides context.labels. Connector insight detection was rewired off the retired /v1/entities route onto /v1/facts.

• /v1/answer temporal valid_during — a query with only a valid_during window was treated as un-pinned, so derived-layer context was injected without applying the requested window. It is now a proper temporal pin, and the response as_of reflects the window.

• CLI: forget --query now works (was silently dropped → confusing 422); bulk writes warn on a 207 partial commit instead of reporting success; --ordering help no longer promises unimplemented enforcement.

• MCP: erasure dry-run now reports the real affected count (was always 0).

• Python SDK README forget() example corrected (reason=); TypeScript bulk response type now surfaces results / failures / partial.

• GETTING_STARTED recall example uses budgets.max_tokens (the prior tokens_total was silently ignored) and the correct Cortex SDK class name. API reference: context.observed_at and idempotency_key corrected to optional; idempotency replay is signaled by a response body flag, not a header.

strict write envelope, honest bulk semantics, unified idempotency, fact-forget repair

• The write envelope (POST /v1/experience, bulk items) is now strict: unknown or misplaced top-level and context fields return 422 instead of being silently ignored. Clients that were unknowingly sending typo'd or misplaced fields (most commonly observed_at at the top level, which silently lost the occurrence timestamp) will now get a clear validation error. As part of this, top-level observed_at is accepted as a first-class, validated alias for context.observed_at — the common placement now works correctly instead of corrupting the bitemporal timeline.

• POST /v1/experience/bulk responses changed: accepted and event_ids now count distinct persisted events (idempotency-deduplicated items are no longer double-counted), and a new index-aligned results[] array reports each item's event_id and replayed_from_idempotency. A batch where some items committed and some failed returns 207 Multi-Status (partial: true, per-index failures[]); any 4xx/5xx bulk response now guarantees nothing was committed — whole-batch retries are safe.

• Selector-scoped fact deletion now works on single-node deployments. Since v0.7.5, five coordinator methods (get_fact, delete_derived_by_source, delete_derived_facts, delete_derived_by_attributes, delete_facts_in_scope) were served as inert defaults through the production wrapper — fact-targeted forgets silently deleted nothing. If you issued fact-selector forgets on v0.7.5–v0.7.9, re-run them on v0.8.0.

• Bulk envelope validation is atomic: every item is validated before anything commits, matching the schema-error path.

• Single and bulk experience writes share one idempotency bucket: the same (scope, key, body) replays across endpoints instead of double-storing; a different body under a reused key returns 409 from either endpoint. Import sources remain separate buckets.

• Reusing an idempotency key with the same content but different context.labels is now reliably a 409 IDEMPOTENCY_CONFLICT (labels are part of the write identity), never a silent replay that drops the new labels.

• CI gate repaired end to end (embedded-docs parity, TypeScript SDK suite actually executing, runner disk headroom) and the ci workflow is now a required merge check on main.

per-tenant idempotency keys and true point-in-time answers

• Idempotency keys are now namespaced per tenant. The dedup scope is (scope root, caller, endpoint family, key), where the scope root is the top-level segment of the write's scope path (org:acct_46, user:alice). Two tenants reusing the same idempotency_key — even under one deployment API key — no longer collide: the second tenant's write is accepted normally instead of failing with 409 IDEMPOTENCY_CONFLICT, and a conflict response can no longer reveal another tenant's event_id. Within a tenant nothing changes: an identical retry still replays the original event, and reusing a key with a different body is still a 409. Upgrade note: idempotency records written by earlier versions do not carry across this upgrade — a retried pre-upgrade write re-processes (safely) instead of replaying.
• POST /v1/answer now honors temporal.as_of (and the as_of_valid / as_of_recorded axis pins). Point-in-time questions are answered from the state at the pinned instant: retrieval, the evidence block, and the answer prompt all reflect the pin, and the response's as_of echoes the requested instant instead of the current time. Note that as_of pins both bi-temporal axes ("true at T" and "known at T") — use as_of_valid to ask what was true at T regardless of when the system learned it.
• GET /v1/experience/status accepts an optional scope= parameter for an exact tenant-scoped probe by idempotency_key; without it, the probe searches all of the calling actor's own tenants (never another caller's).
• Clearer validation errors:
  • An empty (or whitespace-only) content.text is rejected up front with 422 INVALID_ENVELOPE — previously it was accepted and then surfaced as 502 INDEXING_FAILED after the write was already durable.
  • Unsupported pagination parameters on POST /v1/recall (limit, offset, page, cursor, page_size) return 422 with guidance (use budgets to bound the pack; GET /v1/events to paginate raw records) instead of being silently ignored.
  • An empty filter value list (e.g. filters.metadata.labels: []) returns 422 instead of silently matching nothing — a footgun for dynamically-built filters.

Fixes the two remaining findings (and the minor validation gaps) from the v0.7.8 external bug report.

complete-events packs keep their newest events under the default recall budget

• Exhaustive and metadata-filtered recall packs no longer lose their newest events to the default token budget. The cross-layer knapsack (introduced in v0.7.6) applied its default 4,000-token budget to the events layer of complete-corpus packs and evicted from the tail — which, on a chronological corpus, deleted the newest events. With grounded citations enabled, the model could neither see nor cite the latest entries of a changelog-style corpus even though the context block carried them. Complete-events packs are now exempt from default-budget event eviction; derived layers still drain toward the budget, and an explicit budgets.max_tokens from the caller still applies to every layer.

This completes the fix started in v0.7.7: v0.7.7 corrected the citation renderer (recency window, byte-budget direction, marker parsing), and v0.7.8 corrects the upstream pack assembly so the renderer actually receives the newest events it is meant to protect.

grounded citations keep the newest sources; consolidation controls for curated corpora

• Grounded [S#] citations on exhaustive (whole-scope) packs now always include the newest sources: both the source-count window and the prompt byte budget trim from the oldest side of a chronological pack. Previously, "what is the latest X?" could be answered from the middle of a chronological corpus because the newest chunks fell outside the cited window even though retrieval had returned them.

• Citation markers above the source cap now parse correctly. Recency windows can legitimately start above [S1], and a model citing the newest source in a large pack no longer has that citation silently dropped from the response.

• CORTEX_ANSWER_MAX_CITED_SOURCES (default 20): per-deployment cap on the grounded-citation source window, for corpora whose answers legitimately draw on more than 20 chunks (for example, a full release history).

• CORTEX_CONSOLIDATION_ENABLED (default on): first-class switch for background memory consolidation. Set 0, false, or off to disable it entirely — previously the only way to stop consolidation was to raise its thresholds out of reach.

• Consolidation now skips curated content automatically: events written with context.intent: "documentation" or directives.extract: [] are never folded into summaries. Summaries of curated chunks go stale the moment the source documents change, and consolidation archives the sources it folds in — documentation corpora should never pay that trade.

• Consolidation summaries now carry a source=consolidation label, making them queryable (GET /v1/events?labels=source=consolidation) and auditable instead of appearing as anonymous events in the scope.

the 13-issue defect-report response: forget that forgets, fact history, deterministic recall, secure defaults

Closes every open finding from the v0.7.5 external report (DISC-006 through DISC-028):

• forget: fact ids and attribute selectors (about_subject / about_entity / predicate) now delete the derived records they name, with belief/concept cascade and accurate all-layer matched counts; a selector matching nothing says so explicitly. Superseded facts stay fetchable — GET /v1/facts/{id} is new, and as-of queries return the facts that were true then.
• beliefs: genuine same-key conflicts (support_tier, plan, status, …) reconcile to stance:contradicted; per-fact cardinality metadata can mark any predicate single- or multi-valued.
• deterministic recall: pack composition is now reproducible for a fixed query and store — derived-layer listings and rankers use total orderings instead of hash-map iteration order.
• typed triples: deterministic writes are excluded from LLM enrichment (no more duplicated or hallucinated facts), and ?wait=consolidated resolves immediately for them; write status reports the furthest completed stage, consistent with stages_completed.
• erasure: preview and completion counts now match the records actually deleted, including facts.
• security: a keyless boot on a network-exposed bind (the Docker quickstart) generates and persists an API key on first start instead of serving an open store; CORTEX_INSECURE_NO_AUTH=1 is the explicit opt-in for open dev mode.
• ollama: the native answer provider works out of the box (the default URL now targets Ollama's OpenAI-compatible /v1 surface; bare daemon addresses are normalized).
• events: /v1/events and /v1/events/{id} return one canonical body for the same id.
• performance: default embedding concurrency raised 4→8 and the write admission cap 64→128, recovering bulk-write throughput without touching the WAL fsync durability default.

Also included, disabled by default: fact-as-key retrieval expansion and question-shape-routed layer injection (CORTEX_FACT_INDEX_KEYS, CORTEX_LAYER_INJECTION_GATE) — bench-validated, awaiting production routing hardening.

No breaking changes.

forget selector safety, idempotency under concurrency, enrichment robustness

Fixes for four issues reported against v0.7.4:

• forget: cascade=derived_only with a memory_ids selector now deletes exactly the derived records (facts, beliefs, episodes, concepts) traced to the selected events — never the whole layer. A whole-layer wipe requires an explicitly empty selector plus confirm_all:true. Deleted counts in the response are now accurate, and wiping a derived layer no longer touches event storage.
• experience: concurrent writes with the same idempotency_key now collapse to a single event; retried and racing requests all receive the winning event_id. Applies to both single and bulk writes, and composes with the persistent idempotency store introduced in v0.7.4.
• enrichment: entity extraction no longer silently contributes an empty graph on large, dense documents. The output cap is configurable (CORTEX_ENTITY_MAX_TOKENS, default 4096), truncated responses retry automatically at a higher cap, and internal provider limits no longer clamp dense fact extraction.
• concepts: every concept now carries a walkable provenance trail — supported_by links the facts and beliefs that fed synthesis, and synthesis_inputs.layers reports only layers that actually contributed.

No breaking changes. All response-shape additions are backward compatible.

recall integrity, index self-healing, ranking, and performance hardening

This release merges the recall-quality workstream on top of the v0.7.3 security patch. It closes a silent index-loss class, hardens recall scope/temporal correctness, adds index self-audit + repair, and lands cross-layer budgeting and de-crowding — validated to move the full LongMemEval-S server run up from the prior baseline with no regression.

INDEX INTEGRITY (the headline)

• Fixed a silent search-index loss: a derived-layer forget with an empty selector and confirm_all could strip a scope's BM25/vector documents while the WAL kept the events, so recall went blind on already-stored memories with no error. Derived-layer forgets now purge only derived state and never touch event documents.
• New WAL<->index self-audit: the server can reconcile every durable event against the search indexes and repair any that are missing. Exposed as POST /v1/admin/index-audit (diagnostics.read; {"repair": bool}) and run report-only at startup (CORTEX_INDEX_AUDIT_STARTUP, default on). Filtered/exhaustive recalls now flag divergence between the listing and the ranked pool in diagnostics, so this failure mode can never be silent again.
• Enrichment scanner: fixed a cursor-promotion bug that could mark un-applied events done across a restart, and a livelock that drained only one chunk per boot.

RECALL CORRECTNESS

• Holistic recall at a member (leaf) scope no longer leaks sibling-member content; holistic/descend at a parent scope now blends member data upward (self + ancestors + strict descendants). Current-state answers no longer return a superseded belief when no as_of is given (bi-temporal as_of/valid-time queries unchanged).
• Exhaustive recall no longer treats a scan-capped page as "drained" (no more silently-truncated exhaustive blocks); graph and sibling channels enforce scope isolation.

RANKING + BUDGETS

• budgets.max_tokens is now a real cross-layer token budget (priority spend across layers, tail eviction, diagnostics.knapsack_evictions) — on by default; set CORTEX_CROSS_LAYER_KNAPSACK=0 to restore the prior behavior.
• Per-document pack cap (default on): stops one multi-chunk document from flooding the pack on documentation corpora; a no-op on conversational memory. Optional exact-title topic boost and derived-layer embedding rerank ship default-off (CORTEX_TOPIC_MATCH_BOOST, CORTEX_DERIVED_EMBEDDINGS).

PERFORMANCE

• Recall read path: removed a per-request O(corpus) WAL clone, a term-matching allocation storm, and a serialized retrieval stage; fixed a BM25 group-commit reader race. Warm p95 on a normal-sized corpus improved ~36%. Recall now emits per-phase timings in diagnostics.time_ms (recall.<phase>) for latency attribution. NOTE: on very large corpora under heavy concurrency the dominant recall cost is external calls (query embedding, reranker, query-expansion LLM) — the target of a follow-up performance pass.

UPGRADE NOTES

• No storage-format changes; volumes migrate in place from v0.7.x.
• The cross-layer knapsack and per-document cap are ON by default; both have kill switches (CORTEX_CROSS_LAYER_KNAPSACK=0, CORTEX_PACK_PER_DOC_CAP=0). Both were validated non-regressing on the eval corpora.
• Includes all v0.7.3 security fixes (cross-tenant IDOR on by-ID routes, triple-enrichment integrity, extraction parsing).

security: cross-tenant isolation, deterministic-fact integrity, recall scope correctness

This is a security and correctness patch. It closes a cross-tenant data-disclosure class on by-ID fetch routes, stops the direct-fact path from corrupting deterministic truth, and fixes recall scope resolution so member data neither leaks sideways nor goes missing.

SECURITY — CROSS-TENANT ISOLATION (please upgrade)

• Direct by-ID fetch routes (GET /v1/events/{id}, /v1/understanding/{id}, /v1/erasures/{id}, /v1/understanding/synthesize/{id}, the /v1/lifecycle by-id routes, /v1/blobs/{id}, /v1/import/{id}) resolved a record without a caller-supplied scope, so the scope authorization that /v1/recall enforces never ran. An authenticated caller could read another tenant's record by id. Every by-ID and direct-fetch route now performs object-level authorization against the resolved record's own scope (or, for scope-less staged objects like blobs and import jobs, against its owner). A caller with no access receives 404 — the endpoint never confirms that another tenant's id exists. The by-id write twins (erasure cancel, lifecycle memory-event cancel) authorize before mutating.

DETERMINISTIC-FACT INTEGRITY

• Typed triples written through the synchronous direct-fact path are no longer also run through async LLM enrichment. Re-enriching an already-deterministic fact re-derived it from prose and could hallucinate adjacent attributes (an observed case turned a known-good plan=Pro into account_status=active). Triple-origin events are now skipped by the enrichment scanner.

ENTITY EXTRACTION (Anthropic and other providers)

• Knowledge extraction no longer stalls at facts=0/beliefs=0 when the LLM wraps its JSON reply in markdown code fences or surrounds it with prose (which some providers do even when a JSON schema is requested), and no longer truncates large batched extractions. The parser now recovers the JSON payload from a fenced/prose reply on both the batched and per-event paths, and batches get a larger token budget. Completes the bug_03 series (v0.7.0 response_format type -> v0.7.1 json_schema strict field -> v0.7.3 lenient parsing + batch budget).

RECALL SCOPE + TEMPORAL CORRECTNESS

• Holistic recall at a member (leaf) scope no longer pulls in sibling members' private content. Holistic resolution is now self + ancestors + strict descendants: a leaf has no descendants, so it can never surface a sibling (siblings are descendants of the shared parent, not of the leaf).
• Holistic/descend recall at a household (parent) scope now blends member data upward (member beliefs and events surface at the parent) — previously these questions abstained even though the data existed one scope down. The two fixes are one model, so they cannot reopen each other.
• Current-state answers (no as_of) no longer return a superseded value: when no time is pinned, recall keeps only currently-valid beliefs/facts (valid_to = null or in the future). Bi-temporal as_of / valid-time queries are unchanged — the guard is a no-op whenever a time pin is present.

UPGRADE NOTES

• No storage-format changes; volumes migrate in place from v0.7.0–v0.7.2.
• Behavior change to be aware of: a by-ID fetch of a record you cannot access now returns 404 instead of 200. Integrations that fetched cross-tenant ids (which should not have worked) will see 404.
• If you self-host with auth disabled (dev_local) or a shared static API key, object-level authorization is not enforced (identities are operator-chosen) — this matches the existing membership model and is unchanged.

forget safety, Anthropic-only extraction, honest filter docs

Three fixes from self-host evaluation reports against v0.7.1.

FORGET SAFETY (critical)

• POST /v1/forget with confirm_all: true sent alongside a non-empty selector used to run BOTH the targeted delete AND the scope-wide redaction — "delete these 3 records" silently wiped every event in the scope. The ambiguous combination is now refused with HTTP 400 AMBIGUOUS_SELECTOR_CONFIRM_ALL and an actionable message. Targeted deletes need only the selector (no confirm_all); scope-wide operations still require an EMPTY selector plus confirm_all: true, unchanged.
• deleted.events no longer reports ~2x the true number (targeted deletes were double-counted through the storage propagation).
• The forget response now carries requested and matched so callers can cross-check destructive results: matched < requested means some ids did not exist in the scope.

ANTHROPIC-ONLY DEPLOYMENTS

• Entity extraction no longer 400s when CORTEX_LLM_URL points at Anthropic's OpenAI-compatible endpoint: the negotiated json_schema response format now includes the strict field Anthropic requires (response_format.json_schema.strict: Field required). Extractions that failed 6/6 with blocked_reason: enrichment_failing recover without re-ingest. Providers that reject the new shape degrade one rung further (format omitted), so no configuration can strand.

DOCS

• The API reference's recall filters example was un-runnable (it pre-dated the shipped schema). The real shape is {"filters": {"metadata": {...}}} with exactly five attributes (observed_actor, caller, modality, labels, intent); attributes AND, list values OR, labels matches the exact stored string. The boolean/operator DSL was never in the binary and is no longer documented. Note the asymmetry: labels are STORED top-level in context.labels but FILTERED under filters.metadata.labels.

UPGRADE NOTES

• If you scripted confirm_all: true together with selector.memory_ids, the call now returns 400 instead of deleting the whole scope. Drop confirm_all for targeted deletes.
• No storage format changes; volumes migrate in place from v0.7.0/v0.7.1.

durability, honest failure reporting, and a working out-of-the-box path

This release hardens the single-node engine end to end: acknowledged writes now survive power loss, forget/GDPR operations report failures instead of swallowing them, corrupt state fails loudly instead of silently becoming an empty database, and the documented quickstart works exactly as written.

DURABILITY & CRASH SAFETY

• WAL acknowledgments are fsync'd: an acknowledged write now survives power loss, not just process crashes. Benchmarks/bulk loads can opt into the faster non-synced mode with CORTEX_WAL_SYNC=0.
• The v1 accept point is durable: ?wait=captured acks land on stable media before returning, idempotency records persist across restarts (a retry after a crash replays the same event id instead of re-processing), and events caught mid-crash are re-mirrored automatically on the next boot.
• Every WAL read path now verifies checksums; corrupt rows are quarantined and reported instead of silently replayed. Fact snapshots carry an integrity digest — a truncated or tampered snapshot triggers a full rebuild instead of silently suppressing data.
• Persisted index formats are now version-enveloped with automatic legacy fallback, so future format changes migrate cleanly instead of breaking existing data dirs.

FORGET / GDPR

• Forgetting multiple memory ids now purges every id (previously only the first was physically deleted while the API reported success).
• Targeted forgets purge all chunks of a multi-chunk memory, remove the memory from /v1/events listings, and delete associated blobs even for memories written before the last restart.
• Backend failures during forget/erasure now FAIL the request with a retriable error and an audit id — deletion is never reported complete while copies remain. Deletes are idempotent, so retries are safe.

RECALL CORRECTNESS

• Recall responses now honor the caller's contract end to end: layer includes, per-layer limits, exclude_content, and temporal windows apply to every result, including cross-layer and lineage injections.
• Degraded recalls are visible on the wire: diagnostics carry _partial: true and degraded_sources naming what failed, so a backend outage is distinguishable from an empty corpus.
• Fixed three inverted similarity comparisons that made write-path dedup treat maximally-DISSIMILAR facts as duplicates.

SAFETY & OPERATIONS

• CORTEX_API_KEY is now a real credential: its value authenticates as a bearer token (constant-time compared). With a key set, anonymous signup is disabled. Without one, the server runs in an explicit local no-auth mode bound to loopback.
• New readiness probe GET /v1/admin/ready checks data-dir writability, storage handles, and provider state (and reports degraded=true on mock-embedding corpora). Container healthchecks now use it.
• Embedding provider identity is pinned to the data dir: a lost API key can no longer silently downgrade a real corpus to mock vectors — the server refuses to start with a clear re-index message. CORTEX_EMBEDDING_PROVIDER=mock is the explicit dev mode.
• Corrupt vector-index state now fails startup and preserves the on-disk files for repair; it is never replaced by a silent empty index.
• Verified cold backup/restore: scripts/cold_backup.py archives a stopped data dir with per-file SHA-256 manifests and restores only after full verification (tampered/truncated archives are refused). An end-to-end drill validates ids, recall, and forget markers across the round trip.
• Anthropic-only deployments: entity extraction no longer 400s on the Anthropic /v1/chat/completions endpoint — the LLM router negotiates the response_format shape per provider at runtime.

PACKAGING & DOCS

• docker compose up now starts the supported single-node deployment (no shipped default API key; loopback-published by default). The experimental cluster topology moved to docker-compose.cluster-experimental.yml and is clearly labeled non-operational.
• The Docker build uses cargo-chef dependency caching and no longer masks build failures.
• Quickstart, API reference, SDK defaults (port 3141), preset names, and reranker configuration now match the runtime exactly; docs served by the binary are byte-identical to the repo and are secret-scanned before shipping.
• The API reference no longer claims unimplemented features (boolean filter trees, cross-layer token knapsack, backup endpoints, TTL auto-expiry); TTL inputs now log a clear not-enforced warning.

UPGRADE NOTES

• First boot after upgrading writes an embedding-provider pin into the data dir (one log line; no action needed). If you later change embedding provider/model/dims deliberately, set CORTEX_EMBEDDING_ALLOW_REPIN=1 for one restart after re-indexing.
• WAL writes are now synced by default; if ingestion throughput matters more than power-loss safety on a given host, set CORTEX_WAL_SYNC=0.
• Forget/erasure APIs can now return 502 FORGET_BACKEND_FAILED where they previously reported false success — treat it as retriable.
• Clients that parsed the recall filters docs literally: the accepted shape is the five metadata attributes (observed_actor, subject, modality, labels, intent); boolean AND/OR trees were never accepted and are no longer documented.

bi-temporal recall for real, deterministic facts, and honest operations

The headline: the three deep external audits of v0.6.9 (19-item readiness review, 8-item personal-vault audit, self-host defect report) are fully closed in this release — every finding fixed or formally dispositioned, each fix verified live before shipping.

BI-TEMPORAL RECALL

• temporal.as_of_valid and temporal.as_of_recorded are now independent query axes: ask "what was true on June 1, as we knew it on June 15" directly. as_of still pins both.
• Recorded-time and valid-time range filters now apply to every layer — facts, beliefs, episodes, and concepts all honor the same temporal predicates. A miss-range returns empty across the board instead of leaking out-of-window records.
• Date-only range endpoints (2026-07-01) are accepted and end-inclusive; unknown temporal keys are rejected with a 422 instead of silently ignored.

DETERMINISTIC FACTS

• content.kind: "triple" writes a fact directly — no LLM extractor in the loop. Load known-good facts with exact subject/predicate/object and they are queryable immediately.
• Predicates are canonicalized against the ontology at fact-minting time (raw surface form preserved in metadata, plus a predicate_mapped coverage flag). Extraction now steers to one canonical vocabulary, so "billing contact" and "billing contact for" land on one queryable key.
• Python SDK 0.10.0 and TypeScript SDK 0.8.0 support triple writes and the new temporal axes natively.

TRUSTWORTHY WRITE SEMANTICS

• ?wait=consolidated is now a true barrier: it returns only after enrichment for that event is durably applied.
• The write ack's event_id is the stored id — fetchable and citable immediately.
• Idempotent replays are visible: acks and status responses carry replayed_from_idempotency.

ERASURE THAT STICKS (GDPR)

• Scope-wide forget tombstones are now durably persisted and honored by every read path, including the raw event listings the background episode/concept builders consume — previously a rebuild running seconds after a forget could resurrect erased content into a fresh derived record, and tombstones did not survive restarts. Caught by the live deployment suite; fixed at three layers (durable write, listing filter, write-time race guard) and covered by a repetition test.
• A structural fix rode along: the server's coordinator wrapper now delegates every trait method, with a build-failing enforcement test — this class of silently disabled feature (forget tombstones, parent-document hydration, cognitive-state persistence, importance decay, admin compact) cannot recur.

PREDICTABLE LLM SPEND

• Enrichment unset now means zero background LLM calls, exactly as documented: background concept reflection follows the enrichment switch (CORTEX_V1_LAYERS_AUTO=1 explicitly opts events-only reflection back in; manual synthesize is never gated). layers/stats reports the live verdict under health.concepts_gated_by.
• Models that reject the temperature parameter (newer reasoning-generation models) are handled automatically — version-aware defaults plus a self-healing retry that remembers the verdict per model.
• The Ollama answer path works out of the box: bare :11434 URLs are normalized and no cloud key is required. Answer-provider failures surface as proper 502 errors, never as prose in the answer body.

OPERATIONS

• /v1/admin/version reports the exact git commit and build date for Docker images and release binaries alike.
• GET /v1/usage — durable per-scope recall counters for dashboards.
• /v1/openapi.json serves the machine-readable API contract; the route inventory in the docs is generated from the live router.
• Secure-by-default binds: an auth-disabled bare-metal start binds to localhost unless explicitly overridden (Docker images keep publishing on the container interface as before).
• Event listing pages reliably through artifact-heavy stores; modality round-trips verbatim; the synthesis scheduler heartbeats even when idle so "alive but idle" is distinguishable from "stuck".
• dev_local single-user deployments no longer require scope-membership bookkeeping; parent-scope answer/compose automatically descend into child scopes when the parent holds no direct memories (capability-gated).

Known gaps (tracked, not silent)

• The legacy enterprise admin surface (enterprise status, backup, legal hold, encryption rotation, residency, consent/DSAR) was removed in the May v1-only cutover and has not yet been rebuilt on v1; the corresponding config profile feature-gates are parsed but not enforced. The live-suite tests for that surface are retained, skipped-with-reason, as the spec for the v1 replacement.

Upgrade notes

• No storage migration required; new column families are created on first open.
• If you relied on background concept reflection with enrichment disabled, set CORTEX_V1_LAYERS_AUTO=1 — the new default gates that LLM spend off.
• Clients: Python cortexdbai 0.10.0, TypeScript cortexdbai 0.8.0, MCP cortexdb-mcp 0.6.0.

metadata filters for recall

POST /v1/recall (and /v1/answer, /v1/compose) accept a new filters block — hard eligibility predicates over stored event attributes:

{"filters": {"metadata": {"labels": "primary_inbox"}}}

Relevance decides ORDER; filters decide ELIGIBILITY: a candidate that fails the filter never enters the pack (or the answer context), no matter how well it ranks. No more stuffing tag names into the query text and hoping BM25 notices.

• Keys: labels (containment in the event's labels), intent, modality, caller, observed_actor (equality). A list value means any-of; multiple keys AND.
• Fail-closed by design: unknown filter keys are rejected with 422 — a typo'd filter can never silently return unfiltered results.
• Applied at every event-producing path, including the rendered context block that grounds /v1/answer and /v1/compose.
• Recall diagnostics report dropped_metadata_filter.

Also in this release: hydrated events now carry their original labels and intent (previously dropped in projection), and a workspace-wide test-hygiene pass.

Python SDK 0.9.0 (recall/answer/compose(..., filters=...)), TypeScript SDK 0.7.0 (RecallFilters on all three calls), CLI 0.4.0 (cortexdb recall --filter labels=primary_inbox, repeatable).

searchable cold tier, the scope contract, and /v1/compose

Demoted vectors now stay fully searchable. CORTEX_EVICT_VECTORS_ON_ENRICH moves enriched events' chunk vectors out of RAM into an mmap'd cold tier that every search still scans exactly — demotion is now a pure memory/latency trade with zero recall loss. The OS page cache decides what stays resident. Benchmarked on a real 22k-vector index: identical latency to the pre-demotion baseline. Escape hatch: CORTEX_COLD_SEARCH=0.

• Cross-scope synthesis now persists at the nearest common ancestor: memories from user:you/source:chatgpt and user:you/source:whatsapp synthesize into knowledge at user:you instead of failing with "mixed-tenant provenance" warnings. The leftmost scope segment is the isolation boundary: synthesis never crosses tenant roots, and such skips are now a counted health signal (synthesis_cross_tenant_skips_total).
• New view=lineage for POST /v1/recall: child-scope recall can opt in to ancestor-scoped derived knowledge its own data contributed to — gated by provenance intersection AND per-ancestor credential checks. Raw ancestor events never surface.
• forget now cascades to ancestor-scoped derived artifacts via a provenance index — synthesis whose evidence you erased is tombstoned and regenerates from surviving sources.

The synthesis-permitted counterpart to /v1/answer: digests, reports, and overviews as structured markdown (headings, tables, mermaid diagrams), with grounded [S#] citations on by default. Synthesis across memories is the job; fabrication stays forbidden; thin evidence gets per-section "insufficient evidence" notes instead of a global abstention. Parameters: view (incl. lineage), time window, depth (standard|exhaustive), length (brief|standard|deep), section outline, free-text instructions. Available in the Python SDK (compose()), TypeScript SDK, CLI (cortexdb compose), and the Explorer UI's new Compose card.

• CORTEX_ENRICHMENT_DELAY_SECONDS replaces the old realtime/deferred split: every enrichment now flows through the durable WAL-cursor scanner (N seconds cadence; 0 clamps to a 5s floor). A restart can no longer orphan queued enrichment. Legacy CORTEX_ENRICHMENT_DELAY_MINUTES still honored (x60).

• CORTEX_ENRICHMENT_CONCURRENCY default raised 2 -> 16 (the old default made bulk backfills crawl at ~4 events/min).

• /v1/admin/layers/stats now separates raw ingestion health from derived enrichment health: enrichment_status (off|healthy|degraded|failing), per-cause fallback counters (date grounding, batch timeouts, invalid LLM JSON, missing batch indexes), and raw_indexing_healthy.

• context.intent="documentation" routes enrichment to a normative-claims extractor: product facts become memory, example blocks and placeholder names never do.

• Documentation predicates (defaults_to, released_on, listens_on, ...) joined the builtin conflict ontology — competing normative claims across doc revisions now open conflicts for human review instead of silently coexisting.

• Beliefs pagination implemented; layer stats counts no longer clip at 1,000.

Python SDK 0.8.0, TypeScript SDK 0.6.0, CLI 0.3.0 (all with compose). MCP server unchanged (0.5.1).

Compaction dedup: unbounded background growth fixed

• Duplicate abstraction artifacts eliminated — background compaction re-persisted every derived procedure, knowledge summary, and concept on every cycle, growing storage without bound, degrading recall precision, and re-billing LLM generations. A durable dedup registry now guarantees each derived artifact is persisted exactly once. Long-running deployments were accumulating hundreds of thousands of duplicates from a few thousand raw events.
• Repair endpoint for existing stores — POST /v1/admin/compaction/dedup removes accumulated duplicates in place (keeps the original copy of each artifact, deletes the rest from the search indexes, and seeds the registry so they cannot return). Idempotent and safe on a live instance.
• Honest compaction stats — the compaction log now reports newly persisted artifact counts instead of re-derived counts.
• Knowledge synthesis also fingerprints its LLM input, so unchanged entity communities no longer trigger a fresh generation every cycle.

Bulk ingestion fix for Unicode content, honest version reporting

• Unicode chunking fix — bulk ingestion no longer fails on multi-byte characters (em-dashes, non-Latin text) in long boundary-free content such as exported chat transcripts. Chunk overlap now always lands on character boundaries.
• One bad item no longer fails the batch — a chunking failure on a single item is contained to that item; the rest of the bulk request indexes normally and the failed item is reported individually.
• GET /v1/admin/version implemented — the documented endpoint now returns release version, crate version, git SHA, and build date instead of 404.
• Consistent version reporting — the startup banner and usage telemetry now report the real release version (previously both said "v0.1.0" regardless of release), matching /v1/admin/health and the image labels.
• cortexdb-mcp 0.5.1 — the CLI --version output is now derived from the package version instead of a stale hardcoded string.

Answer controls, grounded citations, forget safety

• Answer style controls — POST /v1/answer accepts answer_instructions, a prompt addendum (up to 4000 chars) that controls style, verbosity, and output format ("be terse", "answer as markdown with curl examples", "respond in German") without affecting retrieval. A new question_type "docs-assistant" produces thorough, example-rich documentation-support answers.
• Grounded citations — pass cite_sources: true and the answer cites [S#] source markers inline; the response citations array reflects exactly the sources the model used, with real event ids and full support strength, instead of a fixed top-of-pack list.
• Forget safety — a memory_ids selector now deletes exactly the named events. Scope-wide deletion of derived layers (beliefs, episodes, understanding) requires naming the layer or confirm_all, and scope-wide redaction requires confirm_all; the API warns instead of over-deleting.
• Recall reliability — writes made after a forget are recallable again (erasure tombstones respect their cutoff on every read path); recall context no longer starves on heavily-enriched topics; ranked recall hydrates events reliably across restarts.
• Leaner responses — recall packs no longer carry internal embedding vectors (responses shrink by ~6 KB per fact/belief).
• Blob upload contract enforced — POST /v1/blobs takes the raw file bytes with Content-Type; multipart/JSON wrappers now return 415 unsupported_media_type as documented, instead of being stored verbatim.
• Deployment recall tuning (opt-in) — CORTEX_RECALL_DEFAULT_MAX_TOKENS (deployment-wide context budget), CORTEX_RECALL_RELEVANCE_FLOOR (drop weak context matches instead of budget-filling), CORTEX_RECALL_SUPERLATIVE_RECENCY ("latest/newest" questions prefer the newest evidence). All off by default. Startup now warns when COHERE_API_KEY is set but CORTEX_RERANKER_PROVIDER is not.
• Chunking fix — Markdown numbered-list markers are no longer treated as sentence boundaries, so document chunks stop ending on a dangling "1.".
• Python SDK 0.7.0 — answer(..., answer_instructions=..., cite_sources=True) on sync and async clients.

Write reliability: status probe, async accept, honest indexing errors

Reliability and observability release driven by an external integration audit. All changes are additive — no breaking API changes, no data migration.

Write reliability

• GET /v1/experience/status — write-status probe by idempotency_key or event_id: was a write stored, is it queryable yet, did it fail? Turns an ambiguous client timeout into a safe resume: "not found" or "failed" always mean re-sending is safe (it re-processes, never duplicates).
• ?wait=captured is now a true async accept — the response returns at the durable accept point (~10ms, flat in payload size) and indexing continues in the background: the fast path for large transcripts and chat snapshots. Default behavior is unchanged (fully indexed + read-your-writes when the response returns).
• Honest indexing failures — a write whose indexing fails now returns 502 INDEXING_FAILED (retriable: true) instead of reporting success; its idempotency record is dropped so a retry re-processes instead of replaying. Bulk requests map per-item indexing failures back onto the response, and ?wait=indexed barriers fail fast instead of timing out on events that will never index.

Developer experience

• Structured 413 — PAYLOAD_TOO_LARGE with details.max_body_bytes and remediation pointers (bulk / blobs); the limit is tunable via CORTEX_MAX_BODY_BYTES (default 32 MiB) and documented with recommended chunk sizes.
• Field-level validation errors — 422 INVALID_BODY carries details.field_path (e.g. items[3].context.observed_at), the parser reason, and a pointer to the matching schema.
• GET /v1/schemas — the normative JSON Schemas served by the binary itself, for client-side preflight validation.

Observability

• /v1/admin/metrics now reports version, per-route p50/p95/p99 latency, errors by error_code, indexing-failure count, and the on-disk storage footprint. New Prometheus series: cortexdb_indexing_failures_total, cortexdb_errors_by_code_total.

Clients

• Python SDK write_status(), TypeScript SDK writeStatus() + bulk wait option, MCP server memory_status tool.

3× write throughput, enrichment scheduling tiers, event-driven layers

Breaking default change: enrichment (facts / beliefs / concepts extraction) is now off unless CORTEX_ENRICHMENT_DELAY_MINUTES is set. Set it to 0 for the previous always-on realtime behavior, or N greater than 0 to batch-enrich every N minutes via a crash-safe WAL-cursor scanner. Deployments that upgrade without setting this variable stop extracting new facts; recall over raw memories is unaffected.

Performance

• Write path: ~50 → ~155 writes/s at high concurrency. BM25 group-commit removes the per-document fsync; a new embedding coalescer batches vectors with a tunable accumulation window and in-flight call count. Zero errors through 256 concurrent clients.
• Event-driven derived layers — facts land ~5s and beliefs ~15s after a write in realtime mode (previously up to 90s of tick polling).

Enrichment scheduling

• Deferred mode: a durable per-partition WAL cursor drives batch enrichment — kill-9 tested, at-least-once with dedup, and forgotten or redacted events are never re-enriched.
• New tuning surface: CORTEX_ENRICHMENT_CONCURRENCY, CORTEX_ENRICHMENT_SCAN_CHUNK, CORTEX_ENRICHMENT_BACKFILL, CORTEX_BM25_COMMIT_INTERVAL_MS, CORTEX_EMBEDDING_COALESCE_WINDOW_MS, CORTEX_EMBEDDING_INFLIGHT.
• Enrichment mode and deferred backlog are reported on /v1/admin/layers/stats.

Fixes

• Concurrent bulk ?wait= barriers no longer time out with 408 — the stage handoff backpressures instead of dropping events.
• recall / forget flush group-committed BM25, preserving read-your-writes and GDPR erasure guarantees.

Bundled /status page

• Every instance now serves a /status page showing live ingestion progress and a conflict-review queue — watch a backfill move through enrichment and resolve detected contradictions from the browser, no separate console needed.

Ingestion progress gauges + client releases

• Enrichment queued / completed / pending gauges on /v1/admin/layers/stats, so operators can see exactly how far a backfill has progressed.
• Clients published with the new conflicts/claims surfaces: Python SDK cortexdbai 0.5.0, TypeScript SDK 0.4.0, MCP server cortexdb-mcp 0.4.0.

Bi-temporal knowledge & conflict management

CortexDB now tracks two time axes for every derived fact — when it was true in the world (valid-time) and when the system learned it (record-time) — and detects, queues, and resolves conflicting knowledge on top of that model.

• Bi-temporal records — facts carry valid_from/valid_to plus a record-time interval. Corrections never erase history: superseded records stay queryable as-of any past date.
• Conflict detection, queue & resolution — contradicting claims are detected on ingest, auto-resolved when the dates make the answer unambiguous (temporal split, correction, source priority), and queued for review otherwise. New API: /v1/conflicts and /v1/claims (two-axis "what was true at X, as known at Y" queries).
• GDPR erasure closure — erasure now redacts WAL payloads in place with checksum recompute, so replay can never resurrect erased content.
• Semantic-delta versioning for beliefs and concepts — layers version on meaning changes, not on every re-synthesis.
• Predicate ontology — canonical predicates with synonym mapping lifted extraction canonicalization coverage from 35% to 84% on the quality harness.
• Fix: the maintenance scheduler no longer holds the global coordinator write lock across cycles (removes periodic write stalls).
• Ships in shadow mode by default (CORTEX_BITEMPORAL_MODE=shadow: observe and measure); enforce is opt-in per deployment.

Undated-relationship dating fix

• Time-invariant relationships (kinship, identity) stated without a date are no longer stamped with the observation date as valid_from — an undated fact now stays undated instead of appearing to begin the day it was written down.

Recall & artifact-quality hardening

• New artifact-quality evaluation harness scoring facts, beliefs, episodes, concepts, and recall output — quality regressions are now measurable.
• Extraction fixes: correct first-person attribution, correct valid_until dating, retry on empty extraction, and relative dates anchored to the event's own timestamp.
• Concept synthesis: domain-focused questions, real provenance, no internal-id leaks, no invented content.
• Recall: fail loud on embedding dimension mismatch; unified graph edge endpoints; human transcript speaker lines are no longer stripped from returned context.

Bulk backfill builds the full memory stack

• Fix: POST /v1/experience/bulk batches above the synchronous extraction cap now dispatch async enrichment per memory, so large historical backfills build facts, beliefs, episodes, and concepts instead of raw events only. Single writes and managed connectors were never affected.

Multimodal ingestion, client transcripts, knowledge graph view

• End-to-end file ingestion — upload via POST /v1/blobs, reference as blob_ref content, and the file is extracted, transcribed, or described, then indexed like any memory. Validated for text, PDF/Office (Apache Tika), archives, images (vision models), audio (ASR), and video (keyframes + audio transcript). Processors are opt-in per deployment.
• Client-provided transcript fast path — send an authoritative transcript inline with the blob reference and CortexDB indexes it directly, skipping re-transcription; original bytes are retained as provenance.
• recall(view="structured") returns a JSON-LD knowledge subgraph (entities, predicates, confidence) built from the recalled facts.
• Fixes: ffmpeg bundled in the runtime image; image/audio processor URLs accept either the API base or the full endpoint.

Operational polish: request IDs, honest health, idempotency 409

• Health endpoints report the real release version instead of an internal crate version.
• Every v1 error body carries a request_id, and every response gets a matching X-Request-Id header (client-supplied IDs are honored) — a correlation ID to quote in support requests.
• Reusing an idempotency key with a materially different body returns 409 IDEMPOTENCY_CONFLICT. The conflict check hashes a stable write identity that excludes server-defaulted fields, so genuine retries and connector re-syncs still replay correctly.

Scope inheritance + durable GDPR erasure

• Holistic recall inherits ancestor scopes — knowledge written at an org or region scope is folded into child-scope recall with zero per-user duplication.
• Durable erasure — GDPR/DPDP forget now removes events from every read path and survives process restart; derived indexes (search, vector, content, blobs) are physically purged.
• The write envelope is fully forgiving: context is optional, and a minimal {scope, modality, content} experience is accepted.
• Structured error envelopes on write endpoints; small API papercuts closed from a full docs-vs-deployed-code audit.

Forgiving write API

• POST /v1/experience no longer requires hand-built envelopes: bare {"text": ...} content, observed_at defaulting to now, and auto-generated idempotency keys.
• Bulk accepts experiences as an alias for items; the granular recall view now populates context_block; new unauthenticated GET /v1/health.
• Driven by a live pilot audit — backward compatible throughout.

Recall & answer quality + 29 framework integrations

• Shape-aware reranking — the reranker adapts to question shape (single-fact vs enumeration/aggregation), with coverage-seeking facet merge and fan-out for enumeration questions.
• Fact-join multi-hop retrieval for questions that join across memories, plus a relevance-ranked entity-to-facts channel.
• Premise-gated abstention — the answer stage pushes back on false-premise questions instead of confabulating, grounded in retrieved evidence.
• Stratified evidence pack (belief sheets + fact ledger) passed to the answer stage; real token usage surfaced in /v1/answer diagnostics.
• The Python SDK bundles 29 framework integrations as optional extras — pip install cortexdbai[crewai] (LangChain, LlamaIndex, CrewAI, LangGraph, Letta, AutoGen/AG2, Agno, DSPy, and more).

Unified codebase + managed connectors

• One build everywhere — the former enterprise edition's hybrid recall merged into main; a single binary now serves community and enterprise deployments.
• Six managed connectors (Slack, GitHub, Jira, PagerDuty, Confluence and more) integrated across bulk, streaming, and webhook ingestion, with a no-credentials test harness.
• Per-instance dashboard branding (CORTEX_INSTANCE_LABEL, default scope, auth labeling) and an observed_actor ingestion fix.
• MCP server: configurable recall view via CORTEXDB_VIEW, corrected recall rendering, and a rewritten insights engine.
• Event labels and intent are preserved through /v1/events.

Opt-out usage telemetry

Default change: the community binary now sends a small periodic anonymous usage beacon (install UUID, version, OS/arch, endpoint call counts and latencies, data size). Never sent: request/response bodies, query text, stored memory content, IPs, or scope names. Opt out with CORTEX_TELEMETRY=0 or DO_NOT_TRACK=1 — nothing is collected or transmitted.

• The beacon is best-effort and entirely off the request path — it never adds latency, and a failed send is silently dropped.
• Build hygiene: clippy correctness lint fixed; benchmark-specific markers removed from shipped source.

One coherent API, a real admin console, on-prem first

Breaking: the legacy /v1/remember write path is retired — use POST /v1/experience. Legacy SDK clients are replaced by the realigned cortexdbai packages (0.3.0). No data migration required.

• The /v1/* surface is now the only public API: write via experience, read via recall/answer, per-layer GET endpoints, and the admin surface.
• Bundled admin console — all five memory layers with per-row drilldown, stance pills on beliefs, confidence on facts, cursor pagination, and an inline answer panel with citations and a raw evidence-pack inspector.
• On-prem first — boots with no API key for local use (with a loud warning) and flips to enforced Bearer auth the moment a key is set. No cloud dependency, no phone-home.
• Five-layer recall in the stratified pack; per-layer lexical ranking in community recall; concept synthesis merges instead of overwriting.
• New /v1/admin/metrics: request/error/rate-limit counters + uptime. 17 findings from an external API audit resolved.
• Native per-arch Docker builds cut release build time from ~5 hours to ~6 minutes; operations docs bundled in every tarball.

Benchmark-proven defaults, baked in

• First tagged release: the Docker image ships with the recall configuration proven on long-horizon memory benchmarks as its out-of-the-box defaults — what we benchmark is what you run.