vocero-client/vocero-api/CHANGELOG.md

# Changelog — vocero-api (Azurian Vocero)

All significant client changes are recorded here. This English history
supersedes the former Italian and Spanish versions. Historical entries describe
the implementation at the recorded date; current operating instructions live in
[the project documentation](../docs/README.md).

## Unreleased

### Fixed

- Consolidate responsive transcript layout, keep export controls visible on narrow screens,
  and place feedback and version details below the conversation. Add jump-to-latest,
  semantic message metadata, state-aware empty hints, shared sanitized Markdown rendering,
  and reduced-motion support. Remove retired camera, account, and overlay styles.

- Use native dialogs for startup and mobile reports, restore focus on dismissal,
  and make duplicate transcript bubbles inert. Require explicit wake-word opt-in,
  show when it is listening, and turn it off when the conversation stops.

- Isolate microphone meter subscriptions from the application tree. Apply draft settings only
  on Save, preserve explicitly live noise calibration, and persist/send only relevant changes.

- Keep the voice UI speaking until buffered playback finishes, flush short final audio chunks,
  and exclude response processing from the user inactivity timeout.

- Make frontend configuration failures retryable, bound startup requests, and
  load optional reports independently while waiting for caller context before
  starting a session. Cancel superseded external reasoning and reject stale
  answers after a new turn, interruption, or session teardown.

### Changed

- Start `TALK` and `TALKFIRST` conversations as soon as configuration and caller
  context are ready. Remove the startup dialog and two-second delay; use an
  inline hint and the existing microphone button when a browser requires a tap.

- Replace the frontend legacy stylesheet with Tailwind v4 semantic theme tokens,
  shared form/dialog controls and component-owned layout utilities. Preserve both
  configured themes, responsive layouts, sanitized Markdown and audio-driven
  visuals in focused stylesheets, and remove unused historical selectors.

- Renamed the API and browser projects to `vocero-api/` and `vocero-frontend/`.
  Python uses the `vocero-api` distribution and `vocero_api` import package;
  npm uses `vocero-frontend`. Updated the launcher, Docker setup, tooling,
  tests, and documentation while preserving HTTP/WebSocket contracts and
  existing environment variables, browser preferences, and PM2 process names.
- Removed the optional workflow reasoning service, its configuration, transcript
  forwarding, and browser integration. Reasoning supports `PROXY` and `OPENAI`;
  unsupported mode values fall back to `PROXY`.
- Filter the specific Starlette TestClient / AnyIO `BlockingPortal` deprecation
  in pytest while retaining the runtime versions used for compatibility captures.
- Added a uv-managed Python 3.12 environment and Ruff, mypy, and pytest quality
  checks, all contained in `vocero-api/`. Startup and Docker use the committed
  lockfile. Split the FastAPI service into configuration, route, provider,
  context/reasoning, and realtime modules while retaining `server:app` startup.
- Added Pydantic response contracts with permissive upstream fields and original
  payload serialization. Captured HTTP, request-schema, and realtime regression
  fixtures preserve existing API behavior, including known failures.
- Removed the legacy browser site in favor of the React application in
  `vocero-frontend`. FastAPI now serves only backend and documentation routes.
- Retired daily talk-time accounting and the associated Hugging Face account
  UI. Load-balancer allocation, queue lifecycle, caller profiles, upstream
  authentication, and voice-session cleanup remain available.
- Consolidated technical documentation under the root `docs/` directory and
  replaced translated changelog copies with this English history.
- Moved the agent-profile and reference-audio examples into `docs/examples/`.
- Moved the historical architecture proposal into `docs/proposals/` and retained
  the existing document URLs as redirects.
- Removed the obsolete `index_video.html` page, which no longer matched the
  elements required by the shared client script.
- Translated developer comments, diagnostics, internal model instructions, and
  local identifiers into English. Sanitization accepts both the former Spanish
  and new English internal tool-result wrappers.
- Standardized browser labels, accessibility text, account/profile dialogs,
  session feedback, and built-in API errors in Spanish. The interface declares
  `lang="es"`; internal instructions and diagnostics remain English.

## 2026-09-25

### Fixed — ApiPilot tool discovery in realtime sessions

- Fetch the tool catalog at the start of each realtime WebSocket connection,
  before opening the upstream connection. Previously the catalog was injected
  only through HTTP reasoning, leaving voice sessions without it.
- Inject ApiPilot independently of `S2S_INJECT_PROFILE`, including sessions with
  profile injection disabled.
- Cache discovery using the authenticated identity or an anonymous session key.
  Save the tool definitions and wrapped context in the session context and reuse
  them when patching later `session.update` messages.
- Wrap the catalog with `APIPILOT_TOOLS_LIST_BEGIN/END` markers and include it
  after the other context blocks. Record catalog readiness, count, timing,
  authentication state, and injection length; full prompt dumps remain controlled
  by `S2S_DEBUG_PROMPT`.

### Fixed — Two-pass execution, authentication, and marker handling

- Strengthen the first-pass prompt so a tool selection emits exactly one CALL
  marker line, without a spoken preamble or trailing explanation.
- Replace exposed CALL markers in chat with a Spanish waiting message.
- Detect and execute CALL markers before sanitizing them. Sanitizing first had
  erased the signal required to start the second pass.
- Correct the monotonic-clock helper used by second-pass processing.
- Reject illustrative phone values from model output and fill missing values
  from the actual session context.
- Accept empty arguments and normalize malformed argument representations using
  JSON, single-quoted mappings, key/value pairs, missing braces, or empty input.
- Initialize default arguments on all discovery and exception paths.
- Normalize execute endpoints, supporting an explicit execute URL and avoiding
  an incorrect `/tools/execute` route when the backend expects `/execute`.
- Send empty arguments for authentication-only tools whose data is derived from
  the session token, avoiding conflicts with a separate phone argument.
- Reuse the discovery authorization and demo-session identity for execution.
- Add the configured API-key override for deployments using that authentication
  path, ahead of the service-token and session-token paths. The historical
  override sent `X-API-Key`, `Authorization: Bearer`, and `x-demo-session`
  together for discovery and execution, and bypassed phone login entirely.
  Documentation uses placeholders rather than retaining the credential from
  the old history.
- Add diagnostics for discovery, execution, schema validation, login-token
  shape, default arguments, and upstream HTTP errors.

The historical authentication priority was: configured API key, configured
service token, discovery authorization, forced phone login when required, and
anonymous access to tools that do not require authentication. This records the change rather than
recommending a credential or deployment configuration.

### Fixed — Prevent internal markers from reaching speech

- Sanitize CALL and RESULT markers in the relay, LLM output processor, and TTS
  handler, including truncated, inverted, and partially wrapped forms.
- Use the Spanish spoken placeholder
  `Voy a consultar tus datos, un momento por favor.` for pending calls.
- Add phone-context fallback and prompt rules preventing internal marker syntax
  from being spoken. Suppress the first-pass audio while preserving the second
  pass that contains the actual tool result.
- Remove leaked Unix, Windows, and TRAE workspace paths. Preserve ordinary text
  containing phone numbers when no internal marker context is present.
- The original offline sanitizer check passed 7 of 11 cases, with four
  artificial edge cases still failing and end-to-end voice verification pending.
  This is historical verification, not a claim about later changes.

## 2026-09-24

### Added — ApiPilot reasoning in two passes

- Discover tools dynamically and present their names and descriptions to the
  reasoning model. A matching request produces
  `<<<APIPILOT_CALL:NAME=tool_name,ARGUMENTS={...}>>>`.
- Parse the marker, execute the selected tool, and return its text to the model
  in a second reasoning request. The model then produces the spoken answer;
  an empty second-pass answer falls back to the raw tool-result text.
- Support `S2S_APIPILOT_TOOLS_EXECUTE_URL` as an explicit execute-endpoint
  override. Normalize the configured endpoint instead of hardcoding a backend
  route.
- Supply known caller-context values to argument construction and use bounded
  catalog/context text rather than exposing authentication material to the model.

### Added — Browser fetch and WebSocket tracing

- Trace fetch calls with semantic endpoint tags, method, URL, timing, response
  status, and bounded body/response previews. Distinguish configuration,
  reasoning, profile, reports, search, ApiPilot, queue, and session operations.
- Trace WebSocket connection, handshake, close, session, and tool events while
  limiting noisy audio-frame logs and masking secret URL parameters.
- Prefetch ApiPilot tools at every realtime connection in parallel with audio
  and WebSocket setup. Connection setup awaits these operations while isolating
  discovery failures. Cache per user, invalidate on identity changes, and
  refresh after successful phone login.
- Provide the browser demo-discovery path and server-side discovery logs so
  operators can follow both halves of a proxied request.

### Fixed — Discovery refresh after login

- Refresh the tool catalog after successful phone authentication, rather than
  retaining the catalog fetched for the anonymous identity.
- Store client discovery state per identity and expose cache invalidation for
  identity changes and explicit refresh requests.

### Added — Dynamic ApiPilot proxy

- Add `GET /api/apipilot/tools` and `POST /api/apipilot/execute` with a consistent
  `{success, data, error}` envelope.
- Configure discovery URL, service-token override, cache TTL, and timeout through
  environment variables. Cache by authorization/demo-session fingerprint and
  allow a logged stale-cache fallback when discovery is temporarily unavailable.
- Re-discover missing tools once before returning a not-found response, enforce
  `requires_auth`, validate required arguments, and strip unknown arguments when
  the schema is closed. Tools with an open schema retain flexible arguments.
- Present only the minimal tool definition to the model. Return user-facing
  errors in Spanish and keep error codes/status/raw details available for
  diagnostics.
- Expose ApiPilot availability through `/api/config` and record configuration
  availability in the startup banner without printing token values.

## 2026-09-23

### Added — Conversation downloads

- Add the `Descargar conversación` action to the chat header.
- Export JSON and text entirely in the browser, including user/assistant
  messages, tool calls/results, timestamps, display names, and snapshot metadata.
- Preserve initials, interruption flags, and snapshot image `dataUrl` values.
  Export persistent conversation history, excluding ephemeral orb bubbles.
- Include a schema version, export timestamp, message count, and conversation
  entries in the JSON export. Build the text export from the same conversation
  entries so both formats agree.
- Download through temporary Blob URLs, revoke those URLs afterward, and show
  success or empty-conversation feedback. No server export endpoint is required.

### Added — Static latest-summary override

- Add `S2S_LATEST_SUMMARY_TEXT`, with escaped newline handling, ahead of the
  `S2S_LATEST_SUMMARY` HTTP source.
- Accept the static summary without a phone number and apply it in both HTTP
  reasoning and realtime session context. Preserve the summary wrapper and cap.
- Include override availability in startup/context diagnostics and update the
  injection guide.

## 2026-09-07

### Added — Podcast episode context

- Read `episode_id` from the page query and pass it to realtime and HTTP
  reasoning requests.
- Fetch the selected podcast episode independently of phone context, normalize
  dictionary/list/string responses, and wrap the serialized episode with
  `PODCAST_EPISODE_BEGIN/END` markers.
- Cap episode context at 24,000 characters and prioritize it before caller
  profile, scheduled reports, and latest-summary context.
- Store episode context at WebSocket connection and patch subsequent client
  `session.update` messages. Skip unavailable context without aborting the call.

### Added — Session-start modes

- Keep manual orb activation as the default.
- Add `S2S_START_SESSION_MODE=TALK`, which waits for the microphone/audio unlock
  gesture and greets after session configuration.
- Add `TALKFIRST`, which attempts automatic activation and falls back to the
  unlock overlay when browser permissions require a gesture.
- Delay automatic activation by two seconds. Use the existing `hola vocero`
  greeting path rather than a separate greeting protocol.

### Changed — Debug-only reports controls

- Gate the reports-sidebar toggle and suggestions behind `?debug=true`.
- Apply the visibility gate in HTML, CSS, and client lifecycle handling so
  controls do not flash into view before configuration loads.

## 2026-09-04

### Changed — Inline conversation and visual design

- Replace the overlay chat panel with a permanently visible inline conversation
  below the voice stage and above the sticky footer; remove the chat toggle.
- Introduce the historical aurora background and glass surfaces using shared
  violet, pink, and cyan brand tokens, including the animated voice orb.
- Make the chat surface continuous with the page, distinguish message roles,
  and maintain safe spacing from viewport edges and the footer.
- Put reports in a 320-pixel desktop sidebar and an off-canvas mobile panel.
- Use safe viewport units and responsive layouts to improve mobile stability.
- Keep the text-entry handler connected while allowing the input row to be
  hidden in voice-only layouts.

### Fixed — Chat scrolling and optional controls

- Schedule repeated scrolling during token streaming and after response/tool
  completion to account for delayed line wrapping and layout changes.
- Disable scroll anchoring/smooth scrolling and reserve bottom space so the last
  assistant line remains visible above the footer.
- Make removed chat-toggle, badge, backdrop, and close-button references optional
  while keeping required chat/history elements strict.
- Synchronize stylesheet/script/import version query parameters and add static
  no-cache handling for development updates.

### Added — Caller avatar and profile dialog

- Replace the header name with a circular initials avatar.
- Open a profile dialog showing contact details, account identifiers, role,
  organization, locale, and available creation/login timestamps.
- Accept nested profile payloads, hide empty fields, support `mailto:`/`tel:`
  links, and close through Escape, backdrop interaction, or the close control.

Historical context limits were 24,000 characters for an episode, 1,600 each for
profile/summary, and 12,000 for a full reports list. Context behavior has evolved;
consult [context injection](../docs/client/context-injection.md) for the current
configuration and ordering.