valenti/vocero-client
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(dev): explain the combined Vite and FastAPI launcher

Browse Source
This commit is contained in:
alvaro
2026-10-06 16:53:40 -03:00
parent c8f8aff8c7
commit 99413fc06d
6 changed files with 170 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/client/architecture.md
+26 -19
View File
@@ -1,18 +1,24 @@
# Client architecture
The [browser client](../../s2s-client) combines a voice UI with a FastAPI server.
The [React browser client](../../frontend-s2s) uses the
[FastAPI backend](../../s2s-client) for HTTP APIs and a speech WebSocket relay.
`start_ui.sh` serves Vite on the existing public port `8420` and FastAPI on
loopback port `8421`. Vite forwards backend routes and provides hot reload;
Uvicorn reloads Python edits. The launcher supervises both processes.
The speech pipeline and optional Node-RED reasoning service run separately.
Runtime configuration comes from environment variables and the root `.env`.
```mermaid
flowchart LR
Browser[Browser UI]
Vite[Vite on public port 8420]
Server[Client FastAPI server]
Speech[Realtime speech backend]
Reasoning[Node-RED or HTTP reasoning provider]
Tools[ApiPilot backend]
Browser -->|HTTP configuration and proxies| Server
Browser <-->|WebSocket via local proxy| Server
Browser <-->|Frontend and HMR| Vite
Vite -->|HTTP configuration and proxies| Server
Vite <-->|Speech WebSocket relay| Server
Server <-->|Realtime events and audio| Speech
Browser <-->|Optional direct connection| Speech
Server -->|Optional transcript reasoning| Reasoning
@@ -37,15 +43,14 @@ audio. It is not an unconditional byte-for-byte tunnel.
| Source | Responsibility |
|---|---|
| [server.py](../../s2s-client/server.py) | Configuration, static serving, WebSocket relay, context loading, ApiPilot, reasoning, search, queue and usage routes |
| [main.js](../../s2s-client/main.js) | UI state, settings, transcripts, caller/report displays, custom RAG context, and external reasoning |
| [index.html](../../s2s-client/index.html) | Orb and light-theme voice controls, inline chat, dialogs, profile and reports sidebar |
| [style.css](../../s2s-client/style.css) | Dark aurora/glass theme, light theme, responsive layout and motion |
| [ui/chat.js](../../s2s-client/ui/chat.js) | Transcript history, tool results, export and autoscroll |
| [ui/account.js](../../s2s-client/ui/account.js) | Account controls, queue and usage dialogs |
| [ui/dom.js](../../s2s-client/ui/dom.js) | DOM, escaping and error-display helpers |
| [ws/s2s-ws-client.js](../../s2s-client/ws/s2s-ws-client.js) | Realtime events, microphone/playback, session updates, tool-call events and ApiPilot prompt catalog |
| [ws/codec.js](../../s2s-client/ws/codec.js) | PCM/base64 conversion and transcript extraction |
| [ws/orb-visualizer.js](../../s2s-client/ws/orb-visualizer.js) | Audio-reactive orb state |
| [start_ui.sh](../../start_ui.sh) | Dependency setup, ports, readiness, reload processes, and shutdown |
| [vite.config.ts](../../frontend-s2s/vite.config.ts) | Public development endpoint, HMR, allowed hosts, and backend routing |
| [App.tsx](../../frontend-s2s/src/App.tsx) | React layout, voice controls, dialogs, profile and reports sidebar |
| [controller.ts](../../frontend-s2s/src/realtime/controller.ts) | Application state, settings, context, queue, tools and external reasoning |
| [client.ts](../../frontend-s2s/src/realtime/client.ts) | Realtime protocol, session lifecycle, audio and transcript events |
| [components](../../frontend-s2s/src/components) | Chat, account, settings and voice components |
| [audio](../../frontend-s2s/src/audio) | TypeScript capture/playback worklets, PCM codec, and visualizer |
| [legacy-theme.css](../../frontend-s2s/src/styles/legacy-theme.css) | Retained themes and responsive visual styles alongside Tailwind |
| [auth.py](../../s2s-client/auth.py) | Hugging Face OAuth and caller identity/tier |
| [limiter.py](../../s2s-client/limiter.py) | SQLite daily talk-time reservations and reconciliation |
| [flows.json](../../s2s-reasoning-engine/flows.json) | Node-RED reasoning and assistant-response logging |
@@ -65,21 +70,23 @@ audio. It is not an unconditional byte-for-byte tunnel.
| `/api/session`, `/api/queue/*` | Load-balancer session and waiting-queue proxies |
| `/api/me`, `/api/session/heartbeat`, `/api/session/end` | Identity, reservation extension and usage reconciliation |
The root serves the UI. `/docs` remains the FastAPI API explorer, while project
documentation files are served in the `/docs/` subdirectories. Markdown remains
The public root serves the React UI. `/__vite_hmr` carries Vite hot updates;
`/v1/realtime` remains the speech connection. `/docs` remains the FastAPI API
explorer, while project documentation files are served in the `/docs/`
subdirectories. Markdown remains
source text, and directories without `index.html` are not browsable pages.
Static HTML, JS and CSS
responses receive `Cache-Control: no-store` during development.
The existing static client remains in `s2s-client` for direct FastAPI and Docker
usage; its static responses retain the existing no-cache headers.
## Audio path
1. The browser captures microphone audio with echo cancellation, noise
suppression and automatic gain-control settings.
2. [mic-capture.js](../../s2s-client/worklets/mic-capture.js) resamples to 16 kHz
2. [mic-capture.worklet.ts](../../frontend-s2s/src/audio/mic-capture.worklet.ts) resamples to 16 kHz
mono PCM16. The browser sends base64 `input_audio_buffer.append` frames.
3. The backend processes VAD → STT → LLM → TTS and emits realtime events.
4. Assistant PCM16 audio at 24 kHz is decoded and sent to
[audio-playback.js](../../s2s-client/worklets/audio-playback.js), which buffers,
4. Assistant PCM16 audio at the current 16 kHz protocol rate is decoded and sent to
[audio-playback.worklet.ts](../../frontend-s2s/src/audio/audio-playback.worklet.ts), which buffers,
resamples and fades playback edges. Input speech can clear playback for barge-in.
Audio worklets are loaded through `AudioWorklet.addModule`; keep their processor
docs/client/troubleshooting.md
+18
View File
@@ -22,6 +22,24 @@ In browser DevTools, inspect the WebSocket handshake and `session.created`,
inside `session`. A cached browser module can disagree with current server
code; inspect served asset versions and reload with cache disabled when needed.
## Launcher and hot reload
`start_ui.sh` serves Vite on `8420` and FastAPI on loopback port `8421` by
default. Check both processes in the launcher/PM2 logs. If either exits, the
launcher stops the other and returns a failure. Port conflicts are errors;
use distinct `UI_PORT` and `S2S_BACKEND_PORT` values for additional worktrees.
A Vite "Blocked request" response means the public hostname is missing from
`S2S_DEV_ALLOWED_HOSTS`. Configure it without a scheme or port and restart the
launcher. If the page loads but edits do not appear, check the browser's
`/__vite_hmr` WebSocket and the reverse proxy's upgrade handling. Speech uses
the separate `/v1/realtime` WebSocket on the same public origin.
For a startup dependency error, check Node/npm on the process manager's PATH,
Python venv support, and `setsid`/`curl`. The first start installs dependencies;
later starts reuse them until frontend manifests change. Changes to root
`.env` or launcher settings require a restart.
## The assistant promises a tool but nothing happens
Follow the actual execution mechanism:
docs/client/usage.md
+22 -7
View File
@@ -9,23 +9,35 @@ reasoning, caller context, ApiPilot tools and load-balancer requests.
From the repository root:
```bash
SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime ./start_ui.sh
S2S_PROXY_TARGET=ws://localhost:8765/v1/realtime ./start_ui.sh
```
The launcher loads the root `.env`, creates `.venv` if needed, installs the
client requirements when the dependency check fails, and runs uvicorn on port
8420. `UI_PORT` overrides the UI port. Because `.env` is sourced by the script,
values in that file can replace variables supplied on the command line.
client requirements when the dependency check fails, and installs the locked
frontend dependencies when needed. Vite serves the React frontend on port
8420 and proxies the backend routes to Uvicorn on `127.0.0.1:8421`.
`UI_PORT` overrides the public port; `S2S_BACKEND_PORT` overrides the internal
port. Both servers reload source edits, and stopping the launcher stops both.
The speech pipeline remains a separate service. Because `.env` is sourced by
the script, values in that file can replace variables supplied on the command line.
To install and start the UI manually:
Use Node.js 22.18+ on the 22.x line or Node.js 24+, npm, Python 3 with venv,
Bash 4.4+, `setsid`, and `curl`. Remote staging access also needs the exact
hostname in `S2S_DEV_ALLOWED_HOSTS`, HTTPS, and WebSocket forwarding for speech
and Vite HMR. See [frontend development setup](../../frontend-s2s/README.md).
To install and start the backend manually:
```bash
python3 -m venv .venv
.venv/bin/python -m pip install -r s2s-client/requirements.txt
SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime \
.venv/bin/python -m uvicorn --app-dir s2s-client server:app --port 8420
S2S_PROXY_TARGET=ws://localhost:8765/v1/realtime \
.venv/bin/python -m uvicorn --app-dir s2s-client server:app \
--host 127.0.0.1 --port 8421 --reload --reload-dir s2s-client
```
Then, in a second terminal, run `cd frontend-s2s`, `npm ci`, and `npm run dev`.
Open <http://localhost:8420/>, tap the orb/microphone control, grant microphone
access, and speak. Microphone access requires HTTPS or a local browser origin;
use HTTPS when accessing the UI from another machine.
@@ -72,6 +84,9 @@ server; it is not a universally available local service.
## Docker
This existing image serves the retained static client. It does not run the
React/Vite development launcher.
Use the project root as the build context:
```bash