vocero-client/vocero-api/CHANGELOG.md

Changelog — vocero-api (Azurian Vocero)

All significant client changes are recorded here. This English history supersedes the former Italian and Spanish versions. Historical entries describe the implementation at the recorded date; current operating instructions live in the project documentation.

Unreleased

Fixed

• Consolidate responsive transcript layout, keep export controls visible on narrow screens, and place feedback and version details below the conversation. Add jump-to-latest, semantic message metadata, state-aware empty hints, shared sanitized Markdown rendering, and reduced-motion support. Remove retired camera, account, and overlay styles.

• Use native dialogs for startup and mobile reports, restore focus on dismissal, and make duplicate transcript bubbles inert. Require explicit wake-word opt-in, show when it is listening, and turn it off when the conversation stops.

• Isolate microphone meter subscriptions from the application tree. Apply draft settings only on Save, preserve explicitly live noise calibration, and persist/send only relevant changes.

• Keep the voice UI speaking until buffered playback finishes, flush short final audio chunks, and exclude response processing from the user inactivity timeout.

• Make frontend configuration failures retryable, bound startup requests, and load optional reports independently while waiting for caller context before starting a session. Cancel superseded external reasoning and reject stale answers after a new turn, interruption, or session teardown.

Changed

• Replace the frontend legacy stylesheet with Tailwind v4 semantic theme tokens, shared form/dialog controls and component-owned layout utilities. Preserve both configured themes, responsive layouts, sanitized Markdown and audio-driven visuals in focused stylesheets, and remove unused historical selectors.

• Renamed the API and browser projects to vocero-api/ and vocero-frontend/. Python uses the vocero-api distribution and vocero_api import package; npm uses vocero-frontend. Updated the launcher, Docker setup, tooling, tests, and documentation while preserving HTTP/WebSocket contracts and existing environment variables, browser preferences, and PM2 process names.

• Removed the optional workflow reasoning service, its configuration, transcript forwarding, and browser integration. Reasoning supports PROXY and OPENAI; unsupported mode values fall back to PROXY.

• Filter the specific Starlette TestClient / AnyIO BlockingPortal deprecation in pytest while retaining the runtime versions used for compatibility captures.

• Added a uv-managed Python 3.12 environment and Ruff, mypy, and pytest quality checks, all contained in vocero-api/. Startup and Docker use the committed lockfile. Split the FastAPI service into configuration, route, provider, context/reasoning, and realtime modules while retaining server:app startup.

• Added Pydantic response contracts with permissive upstream fields and original payload serialization. Captured HTTP, request-schema, and realtime regression fixtures preserve existing API behavior, including known failures.

• Removed the legacy browser site in favor of the React application in vocero-frontend. FastAPI now serves only backend and documentation routes.

• Retired daily talk-time accounting and the associated Hugging Face account UI. Load-balancer allocation, queue lifecycle, caller profiles, upstream authentication, and voice-session cleanup remain available.

• Consolidated technical documentation under the root docs/ directory and replaced translated changelog copies with this English history.

• Moved the agent-profile and reference-audio examples into docs/examples/.

• Moved the historical architecture proposal into docs/proposals/ and retained the existing document URLs as redirects.

• Removed the obsolete index_video.html page, which no longer matched the elements required by the shared client script.

• Translated developer comments, diagnostics, internal model instructions, and local identifiers into English. Sanitization accepts both the former Spanish and new English internal tool-result wrappers.

• Standardized browser labels, accessibility text, account/profile dialogs, session feedback, and built-in API errors in Spanish. The interface declares lang="es"; internal instructions and diagnostics remain English.

2026-09-25

Fixed — ApiPilot tool discovery in realtime sessions

• Fetch the tool catalog at the start of each realtime WebSocket connection, before opening the upstream connection. Previously the catalog was injected only through HTTP reasoning, leaving voice sessions without it.
• Inject ApiPilot independently of S2S_INJECT_PROFILE, including sessions with profile injection disabled.
• Cache discovery using the authenticated identity or an anonymous session key. Save the tool definitions and wrapped context in the session context and reuse them when patching later session.update messages.
• Wrap the catalog with APIPILOT_TOOLS_LIST_BEGIN/END markers and include it after the other context blocks. Record catalog readiness, count, timing, authentication state, and injection length; full prompt dumps remain controlled by S2S_DEBUG_PROMPT.

Fixed — Two-pass execution, authentication, and marker handling

• Strengthen the first-pass prompt so a tool selection emits exactly one CALL marker line, without a spoken preamble or trailing explanation.
• Replace exposed CALL markers in chat with a Spanish waiting message.
• Detect and execute CALL markers before sanitizing them. Sanitizing first had erased the signal required to start the second pass.
• Correct the monotonic-clock helper used by second-pass processing.
• Reject illustrative phone values from model output and fill missing values from the actual session context.
• Accept empty arguments and normalize malformed argument representations using JSON, single-quoted mappings, key/value pairs, missing braces, or empty input.
• Initialize default arguments on all discovery and exception paths.
• Normalize execute endpoints, supporting an explicit execute URL and avoiding an incorrect /tools/execute route when the backend expects /execute.
• Send empty arguments for authentication-only tools whose data is derived from the session token, avoiding conflicts with a separate phone argument.
• Reuse the discovery authorization and demo-session identity for execution.
• Add the configured API-key override for deployments using that authentication path, ahead of the service-token and session-token paths. The historical override sent X-API-Key, Authorization: Bearer, and x-demo-session together for discovery and execution, and bypassed phone login entirely. Documentation uses placeholders rather than retaining the credential from the old history.
• Add diagnostics for discovery, execution, schema validation, login-token shape, default arguments, and upstream HTTP errors.

The historical authentication priority was: configured API key, configured service token, discovery authorization, forced phone login when required, and anonymous access to tools that do not require authentication. This records the change rather than recommending a credential or deployment configuration.

Fixed — Prevent internal markers from reaching speech

• Sanitize CALL and RESULT markers in the relay, LLM output processor, and TTS handler, including truncated, inverted, and partially wrapped forms.
• Use the Spanish spoken placeholder Voy a consultar tus datos, un momento por favor. for pending calls.
• Add phone-context fallback and prompt rules preventing internal marker syntax from being spoken. Suppress the first-pass audio while preserving the second pass that contains the actual tool result.
• Remove leaked Unix, Windows, and TRAE workspace paths. Preserve ordinary text containing phone numbers when no internal marker context is present.
• The original offline sanitizer check passed 7 of 11 cases, with four artificial edge cases still failing and end-to-end voice verification pending. This is historical verification, not a claim about later changes.

2026-09-24

Added — ApiPilot reasoning in two passes

• Discover tools dynamically and present their names and descriptions to the reasoning model. A matching request produces <<<APIPILOT_CALL:NAME=tool_name,ARGUMENTS={...}>>>.
• Parse the marker, execute the selected tool, and return its text to the model in a second reasoning request. The model then produces the spoken answer; an empty second-pass answer falls back to the raw tool-result text.
• Support S2S_APIPILOT_TOOLS_EXECUTE_URL as an explicit execute-endpoint override. Normalize the configured endpoint instead of hardcoding a backend route.
• Supply known caller-context values to argument construction and use bounded catalog/context text rather than exposing authentication material to the model.

Added — Browser fetch and WebSocket tracing

• Trace fetch calls with semantic endpoint tags, method, URL, timing, response status, and bounded body/response previews. Distinguish configuration, reasoning, profile, reports, search, ApiPilot, queue, and session operations.
• Trace WebSocket connection, handshake, close, session, and tool events while limiting noisy audio-frame logs and masking secret URL parameters.
• Prefetch ApiPilot tools at every realtime connection in parallel with audio and WebSocket setup. Connection setup awaits these operations while isolating discovery failures. Cache per user, invalidate on identity changes, and refresh after successful phone login.
• Provide the browser demo-discovery path and server-side discovery logs so operators can follow both halves of a proxied request.

Fixed — Discovery refresh after login

• Refresh the tool catalog after successful phone authentication, rather than retaining the catalog fetched for the anonymous identity.
• Store client discovery state per identity and expose cache invalidation for identity changes and explicit refresh requests.

Added — Dynamic ApiPilot proxy

• Add GET /api/apipilot/tools and POST /api/apipilot/execute with a consistent {success, data, error} envelope.
• Configure discovery URL, service-token override, cache TTL, and timeout through environment variables. Cache by authorization/demo-session fingerprint and allow a logged stale-cache fallback when discovery is temporarily unavailable.
• Re-discover missing tools once before returning a not-found response, enforce requires_auth, validate required arguments, and strip unknown arguments when the schema is closed. Tools with an open schema retain flexible arguments.
• Present only the minimal tool definition to the model. Return user-facing errors in Spanish and keep error codes/status/raw details available for diagnostics.
• Expose ApiPilot availability through /api/config and record configuration availability in the startup banner without printing token values.

2026-09-23

Added — Conversation downloads

• Add the Descargar conversación action to the chat header.
• Export JSON and text entirely in the browser, including user/assistant messages, tool calls/results, timestamps, display names, and snapshot metadata.
• Preserve initials, interruption flags, and snapshot image dataUrl values. Export persistent conversation history, excluding ephemeral orb bubbles.
• Include a schema version, export timestamp, message count, and conversation entries in the JSON export. Build the text export from the same conversation entries so both formats agree.
• Download through temporary Blob URLs, revoke those URLs afterward, and show success or empty-conversation feedback. No server export endpoint is required.

Added — Static latest-summary override

• Add S2S_LATEST_SUMMARY_TEXT, with escaped newline handling, ahead of the S2S_LATEST_SUMMARY HTTP source.
• Accept the static summary without a phone number and apply it in both HTTP reasoning and realtime session context. Preserve the summary wrapper and cap.
• Include override availability in startup/context diagnostics and update the injection guide.

2026-09-07

Added — Podcast episode context

• Read episode_id from the page query and pass it to realtime and HTTP reasoning requests.
• Fetch the selected podcast episode independently of phone context, normalize dictionary/list/string responses, and wrap the serialized episode with PODCAST_EPISODE_BEGIN/END markers.
• Cap episode context at 24,000 characters and prioritize it before caller profile, scheduled reports, and latest-summary context.
• Store episode context at WebSocket connection and patch subsequent client session.update messages. Skip unavailable context without aborting the call.

Added — Session-start modes

• Keep manual orb activation as the default.
• Add S2S_START_SESSION_MODE=TALK, which waits for the microphone/audio unlock gesture and greets after session configuration.
• Add TALKFIRST, which attempts automatic activation and falls back to the unlock overlay when browser permissions require a gesture.
• Delay automatic activation by two seconds. Use the existing hola vocero greeting path rather than a separate greeting protocol.

Changed — Debug-only reports controls

• Gate the reports-sidebar toggle and suggestions behind ?debug=true.
• Apply the visibility gate in HTML, CSS, and client lifecycle handling so controls do not flash into view before configuration loads.

2026-09-04

Changed — Inline conversation and visual design

• Replace the overlay chat panel with a permanently visible inline conversation below the voice stage and above the sticky footer; remove the chat toggle.
• Introduce the historical aurora background and glass surfaces using shared violet, pink, and cyan brand tokens, including the animated voice orb.
• Make the chat surface continuous with the page, distinguish message roles, and maintain safe spacing from viewport edges and the footer.
• Put reports in a 320-pixel desktop sidebar and an off-canvas mobile panel.
• Use safe viewport units and responsive layouts to improve mobile stability.
• Keep the text-entry handler connected while allowing the input row to be hidden in voice-only layouts.

Fixed — Chat scrolling and optional controls

• Schedule repeated scrolling during token streaming and after response/tool completion to account for delayed line wrapping and layout changes.
• Disable scroll anchoring/smooth scrolling and reserve bottom space so the last assistant line remains visible above the footer.
• Make removed chat-toggle, badge, backdrop, and close-button references optional while keeping required chat/history elements strict.
• Synchronize stylesheet/script/import version query parameters and add static no-cache handling for development updates.

Added — Caller avatar and profile dialog

• Replace the header name with a circular initials avatar.
• Open a profile dialog showing contact details, account identifiers, role, organization, locale, and available creation/login timestamps.
• Accept nested profile payloads, hide empty fields, support mailto:/tel: links, and close through Escape, backdrop interaction, or the close control.

Historical context limits were 24,000 characters for an episode, 1,600 each for profile/summary, and 12,000 for a full reports list. Context behavior has evolved; consult context injection for the current configuration and ordering.