Operator Modes and Break-Glass

The static operator key (CORTEX_API_KEY) is the deployment's own credential. Since 0.14.0 its operator mode decides whether it can also read the memories it stores for others: CORTEX_OPERATOR_DATA_READS is allow, audit or deny. The mode governs only requests the operator key makes (and a deployment minter's mints); no other token is affected.

The three modes

"Counted" means the access report counts what deny would refuse (check operator_access), so audit shows when an application has stopped depending on the key. "Needs the header" means 403 BREAK_GLASS_REQUIRED unless the request carries a break-glass header.

The presets' defaults

Unset, the mode is the deployment preset's (CORTEX_DEPLOYMENT_PRESET):

So a multi-tenant deployment keeps the operator key out of its tenants' memories unless an operator says why, and a single-organization deployment behaves as before. Set the variable to choose otherwise; any other value refuses the start (exit 78).

Break-glass

Under deny, an operator-key request that needs the header sends:

X-Cortex-Break-Glass: reason=support; ticket=SUP-1142

reason is one of incident, legal, support, security, migration, restore or other; ticket is your case reference (1 to 64 characters of [A-Za-z0-9._:-], no personal data); an optional note (percent-encoded, at most 256 bytes) goes to the decision log only. Each such request is audited with its reason and ticket. A read of a private region tells the region's people first (a notice they read at GET /v1/access/notices), and is not served until that notice is stored. The header is checked for its form, not its truth: it is an accountable override, not an authentication.

# A cloud_shared_saas server, the operator key, no header:
curl -sS -X POST "$CORTEX_URL/v1/recall" \
  -H "Authorization: Bearer $CORTEX_API_KEY" -H "Content-Type: application/json" \
  -d '{"scope": "org:acme/user:alice", "query": "standups"}'

{"error_code": "BREAK_GLASS_REQUIRED",
 "message": "the operator key's stored_rows action on /v1/recall needs X-Cortex-Break-Glass under the tenant's operator mode deny",
 "details": {"class": "stored_rows", "mode": "deny", "route": "/v1/recall"}, "retriable": false}

# The same request with the header is served:
curl -sS -X POST "$CORTEX_URL/v1/recall" \
  -H "Authorization: Bearer $CORTEX_API_KEY" -H "Content-Type: application/json" \
  -H "X-Cortex-Break-Glass: reason=support; ticket=SUP-1142" \
  -d '{"scope": "org:acme/user:alice", "query": "standups"}'

A malformed header is 400 INVALID_HEADER in every mode, naming what is wrong (an unknown reason, a ticket outside its grammar, a repeated header). The embedded console asks the operator for a reason and a ticket only after a request answers 403 BREAK_GLASS_REQUIRED.

A tenant's own mode

A tenant can be stricter, or, with a break-glass header, looser than the deployment:

curl -sS "$CORTEX_URL/v1/admin/access/tenant-settings?scope=org:acme" \
  -H "Authorization: Bearer $CORTEX_API_KEY"
# {"scope":"org:acme","operator_data_reads":{"mode":"deny","source":"deployment"},"deployment_mode":"deny"}

curl -sS -X PUT "$CORTEX_URL/v1/admin/access/tenant-settings?scope=org:acme" \
  -H "Authorization: Bearer $CORTEX_API_KEY" -H "Content-Type: application/json" \
  -H "X-Cortex-Break-Glass: reason=migration; ticket=OPS-1142" \
  -d '{"operator_data_reads": "audit"}'
# {"scope":"org:acme","operator_data_reads":{"mode":"audit","source":"tenant"},"deployment_mode":"deny",…}

Lowering a tenant from deny is itself an in-band action and needs the header; raising it back needs none.

Moving an application off the operator key

An application that reads, writes or mints with the operator key on a cloud_shared_saas deployment is refused under deny. To move it without an outage:

1. Set CORTEX_OPERATOR_DATA_READS=audit (or the tenant's own mode to audit). Everything is served as under allow, and each request deny would refuse is counted.
2. Move the application to per-person tokens (Identity and Tokens). While it still uses the key, X-Cortex-Subject-Token: <a person's token> on its requests evaluates each decision for that person too and counts the result (check subject_shadow), without changing any answer: a dry run of the move.
3. Read the counts, then set deny once operator_access stays at zero:

curl -sS "$CORTEX_URL/v1/admin/access/report?scope=org:acme" -H "Authorization: Bearer $CORTEX_API_KEY"

{"scope": "org:acme",
 "checks": [{"check": "operator_access", "requests": 1, "outcomes": {"match": 0, "would_deny": 1},
             "routes": [{"method": "POST", "route": "/v1/recall", "requests": 1, "would_deny": 1,
                         "reasons": {"stored_rows": 1}, "…": "…"}],
             "classes": {"stored_rows": 1}}, "…"],
 "tenant_settings": {"scope": "org:acme", "operator_data_reads": {"mode": "audit", "source": "tenant"}, "deployment_mode": "deny"}}

The report's counts are in memory and restart with the process (counting_since). Every refused request also writes one line under the log target cortex_api::decision (CORTEX_DECISION_LOG, default denials), naming the link that failed.

See also

• Deployment Presets · Multi-Tenancy
• Upgrading to 0.14.0: the operator key on cloud_shared_saas