diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..aa8fe74 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,239 @@ +# Repository guidance + +## Scope and working conventions + +This repository is Vocero: a browser voice client, its FastAPI integration +server, a speech pipeline, and optional Node-RED flows. Use the code, commands, +configuration, and documentation within this checkout. Do not assume other +repositories or developer-specific directories are available. + +All file and directory references in this guide are relative to the repository +root. HTTP routes mentioned below are URLs, not filesystem paths. + +- Check the current branch, worktree, and working-tree changes before editing. + Honor the checkout requested by the user, including direct work on `main`. + Use an isolated worktree when requested and preserve other worktrees. +- An analysis-only or planning request authorizes inspection, not implementation. + Return findings with source locations. Present plans in the conversation + unless the user asks to save a file; follow any explicit approval gate. +- Write clear explanations and ask one conceptual question at a time. Carry out + already authorized work without repeatedly asking for permission. +- Use the native patch/edit tool for manual changes. Prefer `rg`, `fd`, `jq`, + `git`, and standard CLI tools for inspection; check availability before + assuming a tool is missing. Do not install system-wide packages unnecessarily. +- When commits are requested, make coherent, scoped Conventional Commits as work + progresses. Avoid excessive micro-commits and unrelated cleanup. +- Invoke `work-on-feature` only when explicitly requested. For a requested PR, + follow the available `pr-description` skill exactly and put verification + details in the handoff when its format excludes them. Issues are optional; + create native Gitea issues or post comments only when requested. +- When asked to merge locally, use the requested rebase/fast-forward flow; + preserve a recovery ref before rewriting and verify retained patches. A local + merge does not imply a push, deployment, or restart. Report those separately. +- Treat old conversations and proposals as context. Verify claims against the + current code and distinguish proposed authentication/deployment changes from + implemented behavior. + +## Project map + +| Location | Responsibility | +| ---------------------------------- | ------------------------------------------------------------------------------------------- | +| `s2s-client/` | The complete uv project: manifests, environment, Python tooling, tests, and FastAPI package | +| `s2s-client/s2s_client/main.py` | Application assembly, documentation mount, and retained redirects | +| `s2s-client/s2s_client/api/` | HTTP routes, route registration, and response validation helper | +| `s2s-client/s2s_client/models/` | Pydantic request and response contracts | +| `s2s-client/s2s_client/core/` | Environment parsing, logging, public URLs, and startup diagnostics | +| `s2s-client/s2s_client/providers/` | ApiPilot and caller/profile/report HTTP integrations | +| `s2s-client/s2s_client/services/` | Prompt/context construction, reasoning, and tool-context rendering | +| `s2s-client/s2s_client/realtime/` | WebSocket relay, event handling, marker filtering, and audio forwarding | +| `s2s-frontend/` | React, TypeScript, Tailwind, and Vite browser application | +| `speech_to_speech/` | Separately run VAD → STT → LLM → TTS pipeline and realtime API | +| `s2s-reasoning-engine/` | Optional Node-RED reasoning/logging service | +| `start_ui.sh` | Dependency sync and supervision of FastAPI plus Vite | +| `docs/` | Maintained technical documentation, examples, and historical proposals | + +`s2s-client/server.py` is a compatibility shim for `server:app`; new backend +implementation belongs in the package. Keep HTTP handlers focused on transport +and reuse providers/services for shared behavior. + +The old static browser site, daily talk-time quotas, quota database/accounting, +and Hugging Face account UI were deliberately removed. Do not restore their +files, routes, dependencies, or compatibility layers as incidental cleanup. +Load-balancer allocation/queues and upstream caller authentication remain. + +## Tooling and checks + +The uv project must stay inside `s2s-client/`, including +`s2s-client/pyproject.toml`, `s2s-client/uv.lock`, `s2s-client/.python-version`, +`s2s-client/.venv/`, `s2s-client/Makefile`, and +`s2s-client/.pre-commit-config.yaml`. +Use Python 3.12 and the uv version required by `tool.uv.required-version` +(currently 0.12.5). Update dependencies through uv and commit the lockfile; +do not use pip to modify this environment or move it to the repository root. +Keep uv pins in `s2s-client/Dockerfile` and setup documentation aligned when +changing the required version. + +The frontend uses npm and `s2s-frontend/package-lock.json`. Its Node requirement is +`^22.18.0 || >=24.0.0`; consult `s2s-frontend/package.json` for current scripts. +Keep application code and supported configuration files in TypeScript. + +Commands below run from the repository root: + +| Command | Purpose | +| ------------------------------------- | --------------------------------------------------------------------- | +| `make -C s2s-client install` | Sync the locked Python environment and development tools | +| `make -C s2s-client format` | Apply Ruff fixes and formatting | +| `make -C s2s-client check` | Check lock freshness, formatting, Ruff lint, mypy, and pytest | +| `make -C s2s-client test-launcher` | Test readiness, failures, dependency sync, and process-group shutdown | +| `make -C s2s-client test-image` | Build and smoke-test the actual backend Docker image | +| `npm --prefix s2s-frontend ci` | Install locked frontend dependencies | +| `npm --prefix s2s-frontend run check` | Check formatting, lint, TypeScript, Vitest, and launcher integration | +| `npm --prefix s2s-frontend run build` | Type-check and build frontend assets | + +Python uses Ruff with a 120-character line limit, double quotes, import ordering, +and typed application functions checked by mypy. Frontend formatting and lint +rules live in `s2s-frontend/prettier.config.ts` and +`s2s-frontend/eslint.config.ts`; use those tools rather than introducing +competing conventions. + +Run checks appropriate to the change: backend checks for Python/API work, +frontend checks and build for browser work, launcher integration for supervisor +changes, and image tests for packaging/Docker changes. Documentation-only work +normally needs link/path and diff checks, not service startup. Wait for commands +to finish before reporting success, and state skipped coverage or blockers. + +Tests should use mock upstream HTTP and local WebSocket fixtures, isolating +environment configuration. Do not rely on deployed services, real credentials, +or model downloads for client tests. The client gate does not test the separate +speech pipeline, microphone hardware, or real model output. That pipeline has +no complete dependency manifest or general test gate in this checkout; inspect +the selected backend and explain the verification limits for speech changes. + +## API and compatibility rules + +Read [backend development and compatibility](docs/client/development.md) before +refactoring the server or changing schemas. Existing wire behavior is the +baseline, including behavior that looks unsafe, inconsistent, or buggy. +Intentional bug fixes or contract changes need their own scope and regression +evidence; do not bundle them into tooling or structural refactors. + +- Preserve paths, methods, statuses, error shapes, headers, field names, + omissions, coercion, authentication precedence, and outgoing provider calls. + Trace both frontend consumers and provider parsing; a permissive consumer is + not evidence that an upstream schema can safely be tightened. +- Retain existing request-model defaults and extra-field behavior. Do not make + new fields required, forbid previously accepted extras, or normalize loosely + parsed provider values without evidence and an intentional behavior change. +- Response models validate fields constructed by this service. Upstream-owned + values remain permissive where the old parser was permissive: `Any`, optional + fields, open objects, and arbitrary profile JSON are deliberate boundaries. +- `validated_json_response` in `s2s-client/s2s_client/api/responses.py` validates + with Pydantic, then serializes the original payload. Do not substitute + `model_dump()` or automatic response-model serialization: those can insert + defaults, filter unknown keys, + coerce values, or alter serialization. Keep OpenAPI declarations aligned with + the actual validation boundary. +- Preserve environment/dotenv parsing, default flags, and import-time behavior + during refactors. Do not introduce stricter configuration validation or change + authentication behavior as part of a tooling change. +- `s2s-client/tests/fixtures/legacy-contracts.json` captures the server before the + migration at `3d42df5`. `s2s-client/tests/test_compatibility.py` compares HTTP + response bytes, statuses, selected headers, outgoing requests, request schemas, + and realtime frames. Do not regenerate captures merely to make a refactor pass. +- Known reasoning failures, ApiPilot authentication/cache quirks, and relay + behavior are documented in the compatibility guide. Preserve narrow lint/type + exceptions that retain those behaviors until a deliberate fix is in scope. +- The pytest warning filter is only for Starlette TestClient's exact deprecated + AnyIO `BlockingPortal` alias. Do not broaden warning suppression or upgrade + runtime frameworks just to silence that warning without a compatibility review. + +## Browser and realtime behavior + +- React renders the UI; `s2s-frontend/src/realtime/controller.ts` owns observable + application state, `s2s-frontend/src/realtime/client.ts` owns protocol/session + lifecycle, and `s2s-frontend/src/audio/` owns the audio graph and worklets. + Keep high-frequency audio/WebSocket work outside React rendering and preserve + cleanup across unmounts and reconnects. +- Preserve the existing 16 kHz PCM16 audio path, worklet registration, interruption + behavior, and teardown. Session startup initializes audio, waits for + `session.created`, and sends `session.update` before audio or the greeting. + A `session.updated` acknowledgment is optional; requiring it previously broke + startup with this speech service. +- Preserve query parameters such as `phoneNumber` and `episode_id`, existing + `s2s.ws.*` preference keys, session-only RAG, and TALK/TALKFIRST behavior unless + the task explicitly changes them. Context comes from browser instructions and + server/provider calls; do not assume a vector database or a completed magic-link + authentication flow exists. +- Assistant Markdown is parsed with Marked and sanitized with DOMPurify in + `s2s-frontend/src/lib/messages.ts`. User text stays literal, and exports retain + raw message text. Keep sanitization and unsafe-link regression coverage when + changing it. +- Reuse the existing design tokens, Tailwind integration, and + `s2s-frontend/src/styles/legacy-theme.css`. Verify both light and dark themes, + narrow screens, keyboard focus, dialog dismissal, and voice-state feedback for + UI changes. + Preserve intentional `?debug=true` controls and browser audio-permission flows. +- User-visible times use `America/Santiago` where no saved zone exists; avoid + browser-local timezone assumptions. Keep raw timestamps in their existing API + and export formats. + +## Startup and deployment boundaries + +`./start_ui.sh` sources `.env`, syncs `s2s-client/.venv/` with +`uv sync --project s2s-client --locked`, and installs frontend packages when the +manifests change. +It starts Vite on `0.0.0.0:8420` and FastAPI on `127.0.0.1:8421`, with +`UI_HOST`, `UI_PORT`, and `S2S_BACKEND_PORT` overrides. Use distinct ports for +concurrent worktrees. Preserve readiness checks, failure propagation, and +shutdown of both process groups, including Uvicorn's reload worker. + +Staging intentionally retains Vite HMR and Uvicorn reload. The Vite child uses +`NODE_ENV=development` even under PM2. Do not replace this with static production +serving unless that deployment change is requested. Vite proxies `/api`, +`/v1/realtime`, and documentation routes to FastAPI; preserve public Host headers +used to construct WebSocket URLs. Remote staging needs explicit +`S2S_DEV_ALLOWED_HOSTS` hostnames and reverse-proxy WebSocket forwarding for both +`/v1/realtime` and `/__vite_hmr`. + +The established PM2 process name is `s2s_ui`. When it points to this launcher's +deployed checkout, `pm2 restart s2s_ui --update-env` also performs the locked uv +sync; a separate sync is optional to prepare dependencies before restarting. +Ensure the required uv and Node/npm are on that process's PATH. Verify the live +PM2 script path and environment before making claims about a deployment. + +The launcher does not start or restart `speech_to_speech/` or Node-RED. A configured +speech URL does not prove which source revision or implementation runs there. +`SPEECH_TO_SPEECH_URL` pins a direct browser connection; `S2S_PROXY_TARGET` enables +the local relay. Either disables load-balancer mode. The Node-RED flows in +`s2s-reasoning-engine/flows.json` are log-only; preserve the meaning of `log_only` +responses. + +The backend Docker image serves APIs, WebSockets, and root `docs/`, not the React +UI. Build with the repository root as context and `s2s-client/Dockerfile`; the +image listens on 7860. FastAPI's root remains 404, `/docs` is the API explorer, +and project documentation is served below `/docs/` with retained legacy redirects. + +## Language, documentation, and secrets + +- Use English for code, identifiers, comments, logs, internal instructions, and + developer documentation. Spanish is the default for visible UI, spoken output, + and user-facing API messages. Preserve literal wake phrases and protocol names; + translating an existing response or prompt can change a contract. +- Keep maintained documentation under `docs/client/`, `docs/speech/`, `docs/adr/`, + `docs/examples/`, or `docs/proposals/`. Update existing documents first and keep + `docs/README.md` current when adding a document. Use one canonical English file + without language suffixes. Historical proposals do not define current behavior. +- Reserve root Markdown for established files such as `README.md` and `AGENTS.md`. + Keep task summaries in the conversation unless a saved artifact is requested. + Record notable changes under `Unreleased` in `s2s-client/CHANGELOG.md`. +- Keep credentials and deployment values out of commits and examples. Never copy + the root `.env` into the frontend or expose backend secrets through `VITE_*`. + The launcher sources `.env` before startup, so it can override shell values; + direct Python startup uses `load_dotenv(..., override=False)`. Preserve both + behaviors and avoid importing live configuration when a test can isolate it. + +Start with [usage](docs/client/usage.md), [architecture](docs/client/architecture.md), +[development](docs/client/development.md), and [troubleshooting](docs/client/troubleshooting.md). +For speech work, read [the realtime protocol](docs/speech/realtime-api.md) and the +relevant [STT](docs/speech/stt.md), [LLM](docs/speech/llm.md), or +[TTS](docs/speech/tts.md) guide before changing pipeline behavior.