vocero-client/docs/client/context-injection.md

# Context injection

Context reaches the assistant through browser-built instructions, server-patched
realtime session instructions, and HTTP reasoning context. The implementation is
in the server's [context services](../../vocero-api/vocero_api/services/context.py),
[realtime proxy](../../vocero-api/vocero_api/realtime/proxy.py),
[context.ts](../../vocero-frontend/src/lib/context.ts), and the
[WebSocket client](../../vocero-frontend/src/realtime/client.ts).

## Independent controls

`S2S_INJECT_PROFILE` defaults to false and gates server loading of caller,
episode, report and prior-conversation context. It also controls whether the
browser appends caller profile text to its realtime instructions.

It is not a master switch for every context source. Operator-supplied RAG text
and configured ApiPilot discovery work independently. `S2S_INJECT_TODAY`
defaults to true; date and voice-output guards are built on paths that invoke
instruction augmentation or HTTP context assembly. A WebSocket session with no
profile/episode/report/summary or ApiPilot block skips proxy augmentation.
Direct browser connections bypass server-side WebSocket injection entirely.

| Setting | Default | Effect |
|---|---|---|
| `S2S_INJECT_PROFILE` | false | Enable server-resolved caller, episode and history context |
| `S2S_INJECT_REPORTS` | false | Allow scheduled-report fetch/injection; also requires the profile gate |
| `S2S_REPORTS_INJECT_MODE` | `minimal` | `minimal`: labels/dates; `full`: scripts; `off`/`none`: no report prompt block |
| `S2S_REPORTS_QUICK_HINT_MAX` | 5 | Keep at most this many newest reports, clamped to 1–50 |
| `S2S_INJECT_TODAY` | true | Add the current server-local date guard when its context builder runs |
| `S2S_DEBUG_PROMPT` | false | Log complete prompt/context text instead of only summaries |

## Context sources

| Block | Trigger and source | Size/behavior |
|---|---|---|
| Podcast episode | `episode_id` page/query/body value and profile gate | Up to 24,000 characters of serialized episode content |
| Caller profile | `phoneNumber` and `VOCERO_USER_PHONE_AUTH_URL` | Up to 1,600 serialized characters |
| Scheduled reports | `S2S_REPORT_LIST`, profile gate and report gate | Newest N labels/dates by default; full mode includes scripts with a 12,000-character block limit |
| Latest conversation | `S2S_LATEST_SUMMARY_TEXT`, or `S2S_LATEST_SUMMARY` with a phone number | Up to 1,600 characters; static text overrides HTTP fetch |
| Custom RAG | Operator applies text through the RAG dialog | Up to 12,000 characters; appended in browser instructions and HTTP `user_context` |
| ApiPilot discovery | Configured `S2S_APIPILOT_TOOLS_URL` | Tool names/descriptions/auth requirements and call-marker rules, independent of the profile gate |

### Episode

The current episode fetch uses the deployment-specific URL:

```text
https://aquachile.azurianlab.com/api/v1/podcast/episodes/{episode_id}
```

It follows redirects with a 15-second timeout, accepts JSON or plain text,
unwraps a `result` field, and includes arbitrary episode fields. The prompt
labels it as the primary reference for the user's current episode. The endpoint
is hardcoded in the fetch helper rather than selected through an environment
variable.

### Caller profile and login

For caller-specific context, the proxy can first post
`{"phoneNumber":"+56912345678"}` to `S2S_LOGIN_BY_PHONE`, extract a bearer token,
and forward it to subsequent profile/history requests. Profile retrieval uses
`GET VOCERO_USER_PHONE_AUTH_URL?phoneNumber=...` with a 10-second timeout and
accepts an object, optionally wrapped in `result`.

Encode the leading plus sign in page links, for example
`?phoneNumber=%2B56912345678`. Caller information is also fetched through
`/api/user/phone` for the UI. An available profile can update live browser
instructions when profile injection is enabled.

### Scheduled reports

`S2S_INJECT_REPORTS=false` skips report fetching and prompt content, including
the table of contents. When enabled, the report endpoint may be called with or
without a phone number. The resolver keeps dictionary entries, reads
`sourceScheduleLabel` or `sourceSchedule.label`, reads `date` or
`sourceSchedule.date`, and optionally keeps `podcastScript` in full mode.

Entries are sorted newest first using ISO/date/timestamp handling and an
ISO-like string fallback. Empty/unparseable dates sort last. The newest-item
limit applies before constructing the prompt block. `minimal` exposes only the
labels and dates; `full` explicitly opts into script content.

### Latest conversation summary

`S2S_LATEST_SUMMARY_TEXT` takes priority over `S2S_LATEST_SUMMARY` and needs no
phone number or network request. Literal `\n` sequences in the environment value
become newlines. Otherwise the proxy fetches
`S2S_LATEST_SUMMARY?phoneNumber=...` and accepts text or JSON with an optional
`result` wrapper. Both sources pass through the same cap and history wrapper.

### Custom RAG

The operator can paste reference material and explicitly apply it to the
session. The browser appends it to instructions even with profile injection
disabled, includes it in HTTP `user_context`, and updates an open realtime
session. This context does not require an external retrieval endpoint.

## Application timing and order

At local WebSocket connect, the server resolves enabled context and stores it in
`session_ctx`. Each client `session.update` then passes through
`_patch_client_session_update_with_profile()`. Other client frame types are not
modified by this helper. The broader relay still performs ApiPilot handling and
text/audio filtering.

The current augmentation function **keeps base instructions first**, then
appends surviving voice-output guards, date context, episode, profile, reports,
and summary blocks. ApiPilot discovery is appended separately. If a block type
is already present in the base, deduplication may omit its server candidate.
This describes string assembly, not a guarantee of model attention or priority.

Every HTTP reasoning request rebuilds enabled context. Its `user_context`
assembly orders voice guard, date, episode, profile, reports, summary, ApiPilot
and browser-provided context, then deduplicates recognized types and strips
internal markers. Server instruction augmentation remains separate, so avoid
assuming the WebSocket and HTTP prompts have identical ordering.

Context markers support internal deduplication and diagnostics; ordinary
profile/report/episode/history sentinels are stripped before final augmentation.
ApiPilot call/catalog markers have their own handling. Treat the source helpers
as the reference for the exact final payload.

## Failure behavior and inspection

Optional context-fetch errors are logged and the affected context is skipped.
The speech conversation normally continues without that reference material.
This does not mean all API calls ignore errors: missing reasoning configuration,
invalid tool arguments and tool authorization can return explicit errors.

Inspect `/api/config` for active gates, the actual outbound `session.update` in
browser DevTools, and `[INJECT-*]`/`[DEBUG-PROMPT]` server logs. Full prompt
logging includes caller data and conversation text. See
[troubleshooting](troubleshooting.md) for a step-by-step diagnostic flow.