Architecture

Memory Model

Storage is organized as three layers of decreasing resolution over one durable ground truth.

Canonical turns    ground truth: every message, stored verbatim

Layer 0: Raw recent turns            active memory, in the context window
Layer 1: Segment summaries + facts   compressed pages, per-topic
Layer 2: Tag summaries               working set descriptors

Canonical turns are the durable record. Every user and assistant message is persisted as its own row with a content hash, a sort key defining conversation order, sender and channel provenance, and its compaction lifecycle state. Everything else, segments, facts, tag summaries and embeddings, is derived from canonical turns and can be rebuilt from them.

Segments are per-topic summaries produced by compaction: turns are grouped by tag and summarized independently, with structured facts extracted alongside. Tag summaries are one-per-tag digests, so the topic list the model sees is complete. The _general fallback tag, used for content the tagger could not classify, is excluded from tag summaries and tracked as a quality signal.

Request Pipeline

Each request resolves its conversation identity, then passes a content-based routing gate.

Client request
  -> Format detection (Anthropic / OpenAI Chat / OpenAI Responses / Gemini)
  -> Envelope extraction (sender, channel, reply metadata preserved;
                          transport wrappers removed from model-visible text)
  -> Identity resolution (explicit id -> label -> chat id -> system-prompt hash,
                          read through the alias table)
  -> Routing gate: does this conversation have retrievable content?
       no  -> Passthrough, forwarded mostly untouched
       yes -> Active path:
                wait for the previous turn’s background work
                canonical ingestion (reconcile payload against stored rows)
                inbound tagging
                retrieval (3-signal RRF: IDF tag overlap + BM25 + embedding cosine)
                assembly (recent turns + summaries + facts + hint, within budget)
                inject <virtual-context> into the system prompt
                forward upstream, stream the response back
                background: persist turn, LLM tagging, TurnTagIndex update,
                            compaction check, tag summaries, fact extraction

Because the gate is content-based, a conversation with no compacted content and no indexed turns costs almost nothing. Enrichment begins once there is something to enrich.

The REST Path

Besides the proxy, the engine serves a prepare/ingest REST surface used by hosting services. Its shape differs in one critical way: there is no response hook. The proxy sees the model’s response on the same connection and can run completion work immediately. A REST client calls prepare to enrich a request, then later and separately calls ingest with the assistant’s reply. That ingest may arrive late, or never.

The engine is built around the asymmetry: prepare persists the user half of the turn, ingest reconciles the assistant half against the stored tail, and finalization tolerates ingests that never arrive. Every write is idempotent against redelivery.

Canonical Ingestion

Ingestion reconciles each incoming payload against the rows already stored, on every request rather than only the first.

• Alignment: incoming messages are matched to stored rows by content hash. A tail-hash fast path recognizes the common case, where the payload extends the stored conversation by one turn, without rewriting anything.
• Sort keys: rows carry spaced numeric sort keys so mid-history inserts need no renumbering. When repeated inserts exhaust a gap, the reconciler shifts later keys to restore spacing.
• Fragment guard: payloads that look like a fragment of a different conversation, with no overlap against the stored tail, are rejected rather than appended, preventing cross-conversation contamination.
• Provenance: sender, channel, actor identity and reply-target metadata extracted from transport envelopes are stored on each row.
• Turn groups: canonical rows are per-message, so one logical turn spans several rows. A derived turn-group number ties them together, recomputed after ingestion, so compaction and windowing operate on whole logical turns and never split a reply from its prompt.

Multi-Worker Coordination

Multiple proxy or REST workers can serve the same conversation against a shared PostgreSQL store. Coordination is explicit rather than incidental.

• Compaction is fenced. A compaction runs under a leased operation row; a worker that loses its lease has every subsequent write rejected, so a stalled worker cannot clobber a takeover.
• Lifecycle epochs version a conversation’s identity lifecycle. Writes carry the epoch they began under and are rejected if the conversation was reset or merged in the meantime.
• Schema bootstrap is serialized under an advisory lock, so workers starting simultaneously against a fresh database do not race the DDL.
• The backlog sweeper finds conversations whose tagged-but-uncompacted backlog has grown past a threshold and queues them for compaction.

Identity and Aliases

Identity resolution tries, in order: an explicitly supplied conversation ID, a conversation label, a transport chat ID, and finally a hash of the system prompt. IDs in the reserved sk: namespace are caller-asserted and passed through verbatim.

Every resolution step reads through the alias table. VCATTACH writes a durable alias redirecting one conversation identity to another; a predecessor link records an idempotent relationship when a client’s session identity rolls over but the conversation logically continues. Stale conversation markers embedded in old assistant responses keep resolving because they follow the alias chain.

Attribution

Group conversations carry per-message sender identity. The envelope parser claims the sender, channel and reply target from transport metadata before it is stripped from model-visible text, and these land as columns on the canonical turn.

On top of that sit actor profiles and person cards, durable per-actor fact digests injected into assembly for the requester, and speaker-conditioned retrieval: search tools accept a speaker selection, so a question about what one person said resolves against rows attributed to that actor rather than the whole conversation. The full subsystem, its gates and its operator surface are documented on the attribution page.

Storage Backends

Backends share a common store protocol, but not every backend hosts every table; capability checks let callers degrade per backend.

Provider Adapters

Four API formats are supported with automatic detection and no configuration.

Threading Model

The main request path is synchronous within the async server handler. Each conversation owns two background pools: a single-worker pool that serializes tagging and turn persistence, and a compaction pool so a long compaction does not block the next turn’s tagging.

Each new request waits for the previous turn’s background work on the same conversation before ingesting, so reconciliation always sees a consistent tail. Coordination across processes is handled at the storage layer through fencing, epochs and advisory locks, not by in-process locks.