vocero-client/AGENTS.md

# Repository guidance

## Scope and working conventions

This repository is Vocero: a browser voice client, its FastAPI integration
server, a speech pipeline, and optional Node-RED flows. Use the code, commands,
configuration, and documentation within this checkout. Do not assume other
repositories or developer-specific directories are available.

All file and directory references in this guide are relative to the repository
root. HTTP routes mentioned below are URLs, not filesystem paths.

- Check the current branch, worktree, and working-tree changes before editing.
  Honor the checkout requested by the user, including direct work on `main`.
  Use an isolated worktree when requested and preserve other worktrees.
- An analysis-only or planning request authorizes inspection, not implementation.
  Return findings with source locations. Present plans in the conversation
  unless the user asks to save a file; follow any explicit approval gate.
- Write clear explanations and ask one conceptual question at a time. Carry out
  already authorized work without repeatedly asking for permission.
- Use the native patch/edit tool for manual changes. Prefer `rg`, `fd`, `jq`,
  `git`, and standard CLI tools for inspection; check availability before
  assuming a tool is missing. Do not install system-wide packages unnecessarily.
- When commits are requested, make coherent, scoped Conventional Commits as work
  progresses. Avoid excessive micro-commits and unrelated cleanup.
- Invoke `work-on-feature` only when explicitly requested. For a requested PR,
  follow the available `pr-description` skill exactly and put verification
  details in the handoff when its format excludes them. Issues are optional;
  create native Gitea issues or post comments only when requested.
- When asked to merge locally, use the requested rebase/fast-forward flow;
  preserve a recovery ref before rewriting and verify retained patches. A local
  merge does not imply a push, deployment, or restart. Report those separately.
- Treat old conversations and proposals as context. Verify claims against the
  current code and distinguish proposed authentication/deployment changes from
  implemented behavior.

## Project map

| Location                           | Responsibility                                                                              |
| ---------------------------------- | ------------------------------------------------------------------------------------------- |
| `s2s-client/`                      | The complete uv project: manifests, environment, Python tooling, tests, and FastAPI package |
| `s2s-client/s2s_client/main.py`    | Application assembly, documentation mount, and retained redirects                           |
| `s2s-client/s2s_client/api/`       | HTTP routes, route registration, and response validation helper                             |
| `s2s-client/s2s_client/models/`    | Pydantic request and response contracts                                                     |
| `s2s-client/s2s_client/core/`      | Environment parsing, logging, public URLs, and startup diagnostics                          |
| `s2s-client/s2s_client/providers/` | ApiPilot and caller/profile/report HTTP integrations                                        |
| `s2s-client/s2s_client/services/`  | Prompt/context construction, reasoning, and tool-context rendering                          |
| `s2s-client/s2s_client/realtime/`  | WebSocket relay, event handling, marker filtering, and audio forwarding                     |
| `s2s-frontend/`                    | React, TypeScript, Tailwind, and Vite browser application                                   |
| `speech_to_speech/`                | Separately run VAD → STT → LLM → TTS pipeline and realtime API                              |
| `s2s-reasoning-engine/`            | Optional Node-RED reasoning/logging service                                                 |
| `start_ui.sh`                      | Dependency sync and supervision of FastAPI plus Vite                                        |
| `docs/`                            | Maintained technical documentation, examples, and historical proposals                      |

`s2s-client/server.py` is a compatibility shim for `server:app`; new backend
implementation belongs in the package. Keep HTTP handlers focused on transport
and reuse providers/services for shared behavior.

The old static browser site, daily talk-time quotas, quota database/accounting,
and Hugging Face account UI were deliberately removed. Do not restore their
files, routes, dependencies, or compatibility layers as incidental cleanup.
Load-balancer allocation/queues and upstream caller authentication remain.

## Tooling and checks

The uv project must stay inside `s2s-client/`, including
`s2s-client/pyproject.toml`, `s2s-client/uv.lock`, `s2s-client/.python-version`,
`s2s-client/.venv/`, `s2s-client/Makefile`, and
`s2s-client/.pre-commit-config.yaml`.
Use Python 3.12 and the uv version required by `tool.uv.required-version`
(currently 0.12.5). Update dependencies through uv and commit the lockfile;
do not use pip to modify this environment or move it to the repository root.
Keep uv pins in `s2s-client/Dockerfile` and setup documentation aligned when
changing the required version.

The frontend uses npm and `s2s-frontend/package-lock.json`. Its Node requirement is
`^22.18.0 || >=24.0.0`; consult `s2s-frontend/package.json` for current scripts.
Keep application code and supported configuration files in TypeScript.

Commands below run from the repository root:

| Command                               | Purpose                                                               |
| ------------------------------------- | --------------------------------------------------------------------- |
| `make -C s2s-client install`          | Sync the locked Python environment and development tools              |
| `make -C s2s-client format`           | Apply Ruff fixes and formatting                                       |
| `make -C s2s-client check`            | Check lock freshness, formatting, Ruff lint, mypy, and pytest         |
| `make -C s2s-client test-launcher`    | Test readiness, failures, dependency sync, and process-group shutdown |
| `make -C s2s-client test-image`       | Build and smoke-test the actual backend Docker image                  |
| `npm --prefix s2s-frontend ci`        | Install locked frontend dependencies                                  |
| `npm --prefix s2s-frontend run check` | Check formatting, lint, TypeScript, Vitest, and launcher integration  |
| `npm --prefix s2s-frontend run build` | Type-check and build frontend assets                                  |

Python uses Ruff with a 120-character line limit, double quotes, import ordering,
and typed application functions checked by mypy. Frontend formatting and lint
rules live in `s2s-frontend/prettier.config.ts` and
`s2s-frontend/eslint.config.ts`; use those tools rather than introducing
competing conventions.

Run checks appropriate to the change: backend checks for Python/API work,
frontend checks and build for browser work, launcher integration for supervisor
changes, and image tests for packaging/Docker changes. Documentation-only work
normally needs link/path and diff checks, not service startup. Wait for commands
to finish before reporting success, and state skipped coverage or blockers.

Tests should use mock upstream HTTP and local WebSocket fixtures, isolating
environment configuration. Do not rely on deployed services, real credentials,
or model downloads for client tests. The client gate does not test the separate
speech pipeline, microphone hardware, or real model output. That pipeline has
no complete dependency manifest or general test gate in this checkout; inspect
the selected backend and explain the verification limits for speech changes.

## API and compatibility rules

Read [backend development and compatibility](docs/client/development.md) before
refactoring the server or changing schemas. Existing wire behavior is the
baseline, including behavior that looks unsafe, inconsistent, or buggy.
Intentional bug fixes or contract changes need their own scope and regression
evidence; do not bundle them into tooling or structural refactors.

- Preserve paths, methods, statuses, error shapes, headers, field names,
  omissions, coercion, authentication precedence, and outgoing provider calls.
  Trace both frontend consumers and provider parsing; a permissive consumer is
  not evidence that an upstream schema can safely be tightened.
- Retain existing request-model defaults and extra-field behavior. Do not make
  new fields required, forbid previously accepted extras, or normalize loosely
  parsed provider values without evidence and an intentional behavior change.
- Response models validate fields constructed by this service. Upstream-owned
  values remain permissive where the old parser was permissive: `Any`, optional
  fields, open objects, and arbitrary profile JSON are deliberate boundaries.
- `validated_json_response` in `s2s-client/s2s_client/api/responses.py` validates
  with Pydantic, then serializes the original payload. Do not substitute
  `model_dump()` or automatic response-model serialization: those can insert
  defaults, filter unknown keys,
  coerce values, or alter serialization. Keep OpenAPI declarations aligned with
  the actual validation boundary.
- Preserve environment/dotenv parsing, default flags, and import-time behavior
  during refactors. Do not introduce stricter configuration validation or change
  authentication behavior as part of a tooling change.
- `s2s-client/tests/fixtures/legacy-contracts.json` captures the server before the
  migration at `3d42df5`. `s2s-client/tests/test_compatibility.py` compares HTTP
  response bytes, statuses, selected headers, outgoing requests, request schemas,
  and realtime frames. Do not regenerate captures merely to make a refactor pass.
- Known reasoning failures, ApiPilot authentication/cache quirks, and relay
  behavior are documented in the compatibility guide. Preserve narrow lint/type
  exceptions that retain those behaviors until a deliberate fix is in scope.
- The pytest warning filter is only for Starlette TestClient's exact deprecated
  AnyIO `BlockingPortal` alias. Do not broaden warning suppression or upgrade
  runtime frameworks just to silence that warning without a compatibility review.

## Browser and realtime behavior

- React renders the UI; `s2s-frontend/src/realtime/controller.ts` owns observable
  application state, `s2s-frontend/src/realtime/client.ts` owns protocol/session
  lifecycle, and `s2s-frontend/src/audio/` owns the audio graph and worklets.
  Keep high-frequency audio/WebSocket work outside React rendering and preserve
  cleanup across unmounts and reconnects.
- Preserve the existing 16 kHz PCM16 audio path, worklet registration, interruption
  behavior, and teardown. Session startup initializes audio, waits for
  `session.created`, and sends `session.update` before audio or the greeting.
  A `session.updated` acknowledgment is optional; requiring it previously broke
  startup with this speech service.
- Preserve query parameters such as `phoneNumber` and `episode_id`, existing
  `s2s.ws.*` preference keys, session-only RAG, and TALK/TALKFIRST behavior unless
  the task explicitly changes them. Context comes from browser instructions and
  server/provider calls; do not assume a vector database or a completed magic-link
  authentication flow exists.
- Assistant Markdown is parsed with Marked and sanitized with DOMPurify in
  `s2s-frontend/src/lib/messages.ts`. User text stays literal, and exports retain
  raw message text. Keep sanitization and unsafe-link regression coverage when
  changing it.
- Reuse the existing design tokens, Tailwind integration, and
  `s2s-frontend/src/styles/legacy-theme.css`. Verify both light and dark themes,
  narrow screens, keyboard focus, dialog dismissal, and voice-state feedback for
  UI changes.
  Preserve intentional `?debug=true` controls and browser audio-permission flows.
- User-visible times use `America/Santiago` where no saved zone exists; avoid
  browser-local timezone assumptions. Keep raw timestamps in their existing API
  and export formats.

## Startup and deployment boundaries

`./start_ui.sh` sources `.env`, syncs `s2s-client/.venv/` with
`uv sync --project s2s-client --locked`, and installs frontend packages when the
manifests change.
It starts Vite on `0.0.0.0:8420` and FastAPI on `127.0.0.1:8421`, with
`UI_HOST`, `UI_PORT`, and `S2S_BACKEND_PORT` overrides. Use distinct ports for
concurrent worktrees. Preserve readiness checks, failure propagation, and
shutdown of both process groups, including Uvicorn's reload worker.

Staging intentionally retains Vite HMR and Uvicorn reload. The Vite child uses
`NODE_ENV=development` even under PM2. Do not replace this with static production
serving unless that deployment change is requested. Vite proxies `/api`,
`/v1/realtime`, and documentation routes to FastAPI; preserve public Host headers
used to construct WebSocket URLs. Remote staging needs explicit
`S2S_DEV_ALLOWED_HOSTS` hostnames and reverse-proxy WebSocket forwarding for both
`/v1/realtime` and `/__vite_hmr`.

The established PM2 process name is `s2s_ui`. When it points to this launcher's
deployed checkout, `pm2 restart s2s_ui --update-env` also performs the locked uv
sync; a separate sync is optional to prepare dependencies before restarting.
Ensure the required uv and Node/npm are on that process's PATH. Verify the live
PM2 script path and environment before making claims about a deployment.

The launcher does not start or restart `speech_to_speech/` or Node-RED. A configured
speech URL does not prove which source revision or implementation runs there.
`SPEECH_TO_SPEECH_URL` pins a direct browser connection; `S2S_PROXY_TARGET` enables
the local relay. Either disables load-balancer mode. The Node-RED flows in
`s2s-reasoning-engine/flows.json` are log-only; preserve the meaning of `log_only`
responses.

The backend Docker image serves APIs, WebSockets, and root `docs/`, not the React
UI. Build with the repository root as context and `s2s-client/Dockerfile`; the
image listens on 7860. FastAPI's root remains 404, `/docs` is the API explorer,
and project documentation is served below `/docs/` with retained legacy redirects.

## Language, documentation, and secrets

- Use English for code, identifiers, comments, logs, internal instructions, and
  developer documentation. Spanish is the default for visible UI, spoken output,
  and user-facing API messages. Preserve literal wake phrases and protocol names;
  translating an existing response or prompt can change a contract.
- Keep maintained documentation under `docs/client/`, `docs/speech/`, `docs/adr/`,
  `docs/examples/`, or `docs/proposals/`. Update existing documents first and keep
  `docs/README.md` current when adding a document. Use one canonical English file
  without language suffixes. Historical proposals do not define current behavior.
- Reserve root Markdown for established files such as `README.md` and `AGENTS.md`.
  Keep task summaries in the conversation unless a saved artifact is requested.
  Record notable changes under `Unreleased` in `s2s-client/CHANGELOG.md`.
- Keep credentials and deployment values out of commits and examples. Never copy
  the root `.env` into the frontend or expose backend secrets through `VITE_*`.
  The launcher sources `.env` before startup, so it can override shell values;
  direct Python startup uses `load_dotenv(..., override=False)`. Preserve both
  behaviors and avoid importing live configuration when a test can isolate it.

Start with [usage](docs/client/usage.md), [architecture](docs/client/architecture.md),
[development](docs/client/development.md), and [troubleshooting](docs/client/troubleshooting.md).
For speech work, read [the realtime protocol](docs/speech/realtime-api.md) and the
relevant [STT](docs/speech/stt.md), [LLM](docs/speech/llm.md), or
[TTS](docs/speech/tts.md) guide before changing pipeline behavior.