Speaker Attribution
Who said what, and how the system proves it.
The Problem
In a group conversation, many people talk to one assistant. Raw chat history flattens them into an undifferentiated stream of user messages, so the model cannot answer what one particular person said, cannot tell whose preference a stored fact records, and will cheerfully attribute one member’s statement to another.
Virtual-context answers this with a layered attribution subsystem: durable per-message provenance, stable actor identity, authorship-safe fact extraction, per-person memory cards, and speaker-aware search.
The whole subsystem ships dark. Every gate defaults to off, and with them off the rendered output and tool schemas are byte-identical to a build without the feature. Nothing below changes your deployment until you turn it on.
Provenance on Canonical Turns
Attribution starts at ingestion. The envelope parser claims identity metadata from the transport wrapper before the wrapper is stripped from model-visible text, and the claims land as columns on the canonical turn row.
| Column | Meaning |
|---|---|
sender | Display label of the human who sent the message |
sender_actor_id | Stable actor ID resolved for the sender |
origin_channel_id, origin_channel_label | Which channel the message arrived on |
reply_target_message_id | The message this one replies to, when the transport carries it |
reply_subject_actor_id | The actor whose message is being replied to |
audience_conversation_id | The audience this row is proved to belong to |
Because provenance is stored on the durable row rather than inferred later from text, every downstream layer can attribute content without trusting model output.
Actor Identity
An actor is a durable identity for one human, keyed by a stable key derived from the transport identity. Display labels change; the actor ID does not. Sender labels can also be recovered from actor profiles when a transport stops sending them.
Reply Lanes and Fact Authorship
Fact extraction is authorship-safe by construction.
Facts are attributed from canonical rows, never from model text. The author of a fact is the actor recorded on the row it came from, not whatever name the summarization model happens to emit.
Reply lanes separate the two people involved in a reply. When one member replies to another, the requester lane carries the replier’s own statement attributed to their row actor, while the subject lane carries the quoted material attributed only to the resolved reply subject. A requester lane never contains its own quote block, and an unresolved actor stays empty rather than being guessed.
This is what makes a question about who said something answerable from storage instead of from the model’s recollection.
Person Cards (Actor Cards)
A person card is a small per-actor digest of durable identity facts, rebuilt from that actor’s own recorded speech and injected into assembly for the requester. When a member sends a message, the model receives that member’s card; a new member with no history gets nothing and starts clean.
Person card and actor card are the same thing, used interchangeably here and elsewhere. The configuration keys and the admin command use the actor spelling: every setting below is prefixed actor_card_, and the rebuild command is admin rebuild-actor-cards.
- Candidate extraction: the fact pipeline proposes card-worthy entries from the actor’s own speech.
- Semantic admission: a dedicated admission model decides whether a candidate is a durable identity fact rather than a passing remark. Admission requires citations to the segments the evidence came from, rejects ambiguous entries, and preserves qualifiers.
- Curation: a curation pass reads the admitted evidence under per-actor and per-audience limits, so a busy channel cannot crowd out an actor’s other evidence, and emits the card.
- Consolidation: cards marked dirty by new evidence are consolidated after compaction, batched with a per-run limit.
- Refresh safety: during a live refresh the last good card keeps serving, malformed responses are retried, and a no-op evidence update does not dirty the card.
Admission is deliberately fail-closed. Enabling the card gate without configuring an admission model is a configuration error, not a silent fallback: the cheap fact extractor must never be the judge of what becomes durable identity memory.
Injection also fails closed. Nothing is injected when the gate is off, the requester is unknown, the audience is unproved, the card is invalid, or no store is available. Cards are fenced by audience, tenancy and lifecycle, so one can never leak across tenants, across unproved audiences, or across a conversation reset. Direct-message evidence and public-channel evidence sit behind a privacy boundary that fails closed.
assembly:
actor_card_enabled: false # master gate; ships dark
actor_card_max_tokens: 400
actor_card_fact_limit: 60
actor_card_turn_limit: 500
actor_card_entries_per_kind: 3
actor_card_curation_model: "" # unset = general compaction model
actor_card_admission_model: "" # REQUIRED when the gate is onSpeaker Roster
The roster is an audience-scoped list of the participants in the current conversation, with durable handles, rendered into assembly so the model knows who is present and how to refer to them. It is independent of the person card and also ships dark: with the gate off there is no roster read at all.
assembly:
speaker_roster_enabled: false
speaker_roster_max_tokens: 300Unified Guilds
A server-style community can be unified into one conversation deliberately. Conversation keys in the caller-asserted namespace carry a kind segment, and guild is a first-class kind alongside channel, group and direct message. A guild-kind key makes every channel of the server share one conversation owner: one memory, one fact base, one actor space for the whole community.
That sharing is bounded by two privacy rules, and both fail closed. They are structural, not filters applied after the fact.
1. Direct-message content can never render into guild context. A stored row is eligible for guild-visible context only when it carries both an origin channel and a proved audience. A direct-message row has no origin channel, so the eligibility check excludes it structurally. Every guild read path shares this boundary, native requester replay and peer reference context alike, so there is no route by which a DM reaches a server channel.
2. Audience proof is per-row, not per-conversation. Owner-scoped lookup alone does not qualify a row: each physical row must carry its own audience proof. Audience equality is deliberately not required, because peer guild channels, and routes retained from before an alias change, legitimately carry distinct audience IDs inside the same owner-scoped conversation.
Within those bounds a unified guild gets:
- Native requester replay: the requester’s own guild history is replayed into the model-visible window as native turns, correctly paired and ordered against the payload, so the assistant remembers its exchanges with that member whichever channel they happened in.
- Peer-channel visibility: recent activity from sibling channels is merged in as reference context through the cross-channel setting, with instruction authority reserved to the current requester.
- Actor continuity: one identity, one card and one fact history per person across every channel of the server.
- Guild-appropriate search scope: setting the audience scope to
conversationtreats the origin channel as provenance rather than a filter, matching the unified model.
Speaker-Conditioned Search
Two independent gates make the search tools speaker-aware.
Annotations (speaker_annotations_enabled) add speaker labels to search results, so the model sees who said each returned excerpt. Selection (speaker_selection_enabled) activates the speaker-input unit as one atomic gate: a request-local speaker hint, a strict speaker-only filter, validation of the selection against the roster snapshot, affinity ordering, and requester-intent conditioning, so a first-person question resolves to the requester. While selection is off, an arriving speaker argument is never consumed and results carry no conditioning metadata.
With both on, a question about one person resolves against rows attributed to that actor rather than the whole conversation, and the strict filter guarantees no other speaker’s content appears.
Audience scope controls the read boundary. The default channel preserves a strict per-channel boundary. conversation keeps the proved owner boundary but treats the origin channel as provenance rather than a filter, which fits servers where every public channel deliberately shares one conversation.
search:
speaker_annotations_enabled: false
speaker_selection_enabled: false
speaker_audience_scope: "channel" # or "conversation"Operator Surface
Existing conversations predate these columns, so the admin CLI ships guarded, idempotent backfills.
| Command | Restores |
|---|---|
admin backfill-senders | Sender labels on canonical turns |
admin backfill-channels | Channel provenance |
admin backfill-actors | Durable actor IDs from retained raw user text |
admin backfill-reply-roles | Reply roles and audience provenance |
admin backfill-fact-authors | Re-distills facts with canonical actor provenance |
admin reattribute-audience | Corrects audience attribution |
admin normalize-canonical-actor-ids | Normalizes actor ID formats |
admin rebuild-actor-cards | Rebuilds card caches |
Telling It Works
- With annotations on, quote-search results carry speaker labels.
- With the card gate on and a known requester, the assembled context contains that requester’s rendered card within its token budget; an unknown requester gets none.
- With the roster on, the assembled context contains the roster block for the current audience.
admin rebuild-actor-cardsreports which cards it rebuilt, and a rebuild-status table records due and completed cards.- New canonical turns in group conversations carry non-empty
senderandsender_actor_id. Rows written before the backfills show empty ones until backfilled.