Embeddings

CortexDB calls an embedding service on every event ingest (one vector per chunk) and on every recall query (one for the query, plus N for HyDE-expanded queries). Embeddings are the single largest line item in the inference bill for most workloads, and vector recall does not work without a real provider configured.

The core constraint

The embedding service's output dimension must match the engine's vector index dimension. They are configured separately and are not cross-checked at startup — a mismatch produces silent recall failures (everything returns 0 results).

# cortex.toml
[engine]
vector_dimensions = 1536

export CORTEX_EMBEDDING_MODEL=text-embedding-3-small
export CORTEX_EMBEDDING_DIMS=1536

Provider selection

export CORTEX_EMBEDDING_PROVIDER=<empty> | mock | cohere | ollama

The provider pin

On first boot the server writes an embedding_provider.pin file (recording provider:model:dims) into the data directory. On every later start, the configured provider is checked against the pin — a mismatch is startup-fatal, with guidance to re-index. A missing API key can no longer silently downgrade a real corpus to mock vectors.

To deliberately migrate a data directory to a different provider/model, set CORTEX_EMBEDDING_ALLOW_REPIN=1 for one restart (and plan to re-embed — vectors from different models are not comparable).

When the API key is missing

If no OPENAI_API_KEY / LLM_API_KEY / COHERE_API_KEY is set and Ollama isn't detected, the binary boots on mock embeddings (384-d) with a loud warning, and the data directory is pinned as mock. GET /v1/admin/ready reports degraded: true and the pinned provider. Mock embeddings produce meaningless recall — if you didn't ask for mock, check your startup logs and readiness output.

The full env-var surface

Choosing a model

Batch and retry tuning

The service packs pending requests into provider calls of up to CORTEX_EMBEDDING_MAX_BATCH_ITEMS items. Larger batches improve throughput but raise the latency of the first request in a batch.

OpenAI's text-embedding-3-* models accept up to 2048 inputs per call — setting the batch higher doesn't error (the client splits transparently) but stops amortizing per-batch overhead.

Local-first with Ollama

ollama pull nomic-embed-text

export CORTEX_EMBEDDING_URL=http://localhost:11434
export CORTEX_EMBEDDING_MODEL=nomic-embed-text
export CORTEX_EMBEDDING_DIMS=768
export CORTEX_EMBEDDING_API_KEY=ollama          # non-empty required

# cortex.toml — match the dim
[engine]
vector_dimensions = 768

Expect ~30 ms / embedding on a modern CPU. See Self-hosting defaults for the full content-only setup.

Caching

HTTP embedding services are wrapped in an in-process LRU keyed by (model, text_hash). The cache survives the process lifetime; it's lost on restart, and its size is not env-configurable (~10K entries compiled default). Highly effective for re-embedding the same text (eval reruns), near-useless for cold ingest.

Diagnostics

On startup the binary logs which service it picked:

info  Using HTTP embedding service model=text-embedding-3-small dims=1536
info  Using Ollama local embedding service model=nomic-embed-text dims=768 url=http://localhost:11434
warn  No OPENAI_API_KEY or LLM_API_KEY set -- falling back to mock embeddings.

Grep for "embedding service" after changing config, and check GET /v1/admin/ready — it reports the pinned provider and degraded: true when on mock. Recall failures are usually a missing API key or a dim mismatch.

Next steps

• LLM & Answer Generation — the extraction and answer LLMs
• Recall Tuning — how embeddings feed the recall pipeline
• Storage & Cluster — the HNSW index that consumes these vectors