docs(dev): explain the combined Vite and FastAPI launcher
This commit is contained in:
@@ -6,7 +6,8 @@ documentation, source identifiers, comments, and internal instructions use Engli
|
|||||||
|
|
||||||
| Component | Purpose |
|
| Component | Purpose |
|
||||||
|---|---|
|
|---|---|
|
||||||
| [s2s-client](s2s-client) | Browser UI, FastAPI server, realtime WebSocket proxy, search and ApiPilot bridges |
|
| [frontend-s2s](frontend-s2s) | React, TypeScript, and Tailwind browser UI, served by Vite with hot reload |
|
||||||
|
| [s2s-client](s2s-client) | FastAPI backend, realtime WebSocket proxy, search and ApiPilot bridges; retained static client |
|
||||||
| [speech_to_speech](speech_to_speech) | Queue-driven VAD, speech recognition, language model, and speech synthesis pipeline |
|
| [speech_to_speech](speech_to_speech) | Queue-driven VAD, speech recognition, language model, and speech synthesis pipeline |
|
||||||
| [s2s-reasoning-engine](s2s-reasoning-engine) | Optional Node-RED HTTP reasoning and conversation logging flows |
|
| [s2s-reasoning-engine](s2s-reasoning-engine) | Optional Node-RED HTTP reasoning and conversation logging flows |
|
||||||
|
|
||||||
@@ -20,16 +21,23 @@ The speech backend needs its own installed model and runtime dependencies;
|
|||||||
From the repository root, point the client at an existing realtime backend:
|
From the repository root, point the client at an existing realtime backend:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime ./start_ui.sh
|
S2S_PROXY_TARGET=ws://localhost:8765/v1/realtime ./start_ui.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
Open <http://localhost:8420/> and allow microphone access. The launcher creates
|
Open <http://localhost:8420/> and allow microphone access. The launcher creates
|
||||||
`.venv`, installs missing UI requirements, loads the root `.env` if present, and
|
`.venv`, installs missing Python requirements and locked frontend dependencies,
|
||||||
starts the server. For the local WebSocket proxy instead of a direct browser
|
and loads the root `.env` if present. Vite listens on `UI_PORT` (default `8420`)
|
||||||
connection, configure `S2S_PROXY_TARGET` and leave `SPEECH_TO_SPEECH_URL` unset.
|
and forwards HTTP/WebSocket backend routes to FastAPI on `127.0.0.1:8421`.
|
||||||
|
Both servers reload during development; stopping the launcher stops both.
|
||||||
|
Leave `SPEECH_TO_SPEECH_URL` unset to use the same-origin speech proxy.
|
||||||
|
|
||||||
To build the UI container, use the repository root as the Docker build context so
|
The launcher requires Python 3 with venv support, Node.js 22.18+ on the 22.x
|
||||||
the root documentation is included:
|
line or Node.js 24+, npm, Bash 4.4+, `setsid`, and `curl`. For staging hostnames,
|
||||||
|
set `S2S_DEV_ALLOWED_HOSTS` and forward the Vite HMR WebSocket through the existing
|
||||||
|
HTTPS reverse proxy. See [development setup](frontend-s2s/README.md).
|
||||||
|
|
||||||
|
The existing Docker image still serves the retained static client. To build it,
|
||||||
|
use the repository root as the Docker build context so documentation is included:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker build -f s2s-client/Dockerfile -t s2s-client .
|
docker build -f s2s-client/Dockerfile -t s2s-client .
|
||||||
|
|||||||
+26
-19
@@ -1,18 +1,24 @@
|
|||||||
# Client architecture
|
# Client architecture
|
||||||
|
|
||||||
The [browser client](../../s2s-client) combines a voice UI with a FastAPI server.
|
The [React browser client](../../frontend-s2s) uses the
|
||||||
|
[FastAPI backend](../../s2s-client) for HTTP APIs and a speech WebSocket relay.
|
||||||
|
`start_ui.sh` serves Vite on the existing public port `8420` and FastAPI on
|
||||||
|
loopback port `8421`. Vite forwards backend routes and provides hot reload;
|
||||||
|
Uvicorn reloads Python edits. The launcher supervises both processes.
|
||||||
The speech pipeline and optional Node-RED reasoning service run separately.
|
The speech pipeline and optional Node-RED reasoning service run separately.
|
||||||
Runtime configuration comes from environment variables and the root `.env`.
|
Runtime configuration comes from environment variables and the root `.env`.
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
flowchart LR
|
flowchart LR
|
||||||
Browser[Browser UI]
|
Browser[Browser UI]
|
||||||
|
Vite[Vite on public port 8420]
|
||||||
Server[Client FastAPI server]
|
Server[Client FastAPI server]
|
||||||
Speech[Realtime speech backend]
|
Speech[Realtime speech backend]
|
||||||
Reasoning[Node-RED or HTTP reasoning provider]
|
Reasoning[Node-RED or HTTP reasoning provider]
|
||||||
Tools[ApiPilot backend]
|
Tools[ApiPilot backend]
|
||||||
Browser -->|HTTP configuration and proxies| Server
|
Browser <-->|Frontend and HMR| Vite
|
||||||
Browser <-->|WebSocket via local proxy| Server
|
Vite -->|HTTP configuration and proxies| Server
|
||||||
|
Vite <-->|Speech WebSocket relay| Server
|
||||||
Server <-->|Realtime events and audio| Speech
|
Server <-->|Realtime events and audio| Speech
|
||||||
Browser <-->|Optional direct connection| Speech
|
Browser <-->|Optional direct connection| Speech
|
||||||
Server -->|Optional transcript reasoning| Reasoning
|
Server -->|Optional transcript reasoning| Reasoning
|
||||||
@@ -37,15 +43,14 @@ audio. It is not an unconditional byte-for-byte tunnel.
|
|||||||
| Source | Responsibility |
|
| Source | Responsibility |
|
||||||
|---|---|
|
|---|---|
|
||||||
| [server.py](../../s2s-client/server.py) | Configuration, static serving, WebSocket relay, context loading, ApiPilot, reasoning, search, queue and usage routes |
|
| [server.py](../../s2s-client/server.py) | Configuration, static serving, WebSocket relay, context loading, ApiPilot, reasoning, search, queue and usage routes |
|
||||||
| [main.js](../../s2s-client/main.js) | UI state, settings, transcripts, caller/report displays, custom RAG context, and external reasoning |
|
| [start_ui.sh](../../start_ui.sh) | Dependency setup, ports, readiness, reload processes, and shutdown |
|
||||||
| [index.html](../../s2s-client/index.html) | Orb and light-theme voice controls, inline chat, dialogs, profile and reports sidebar |
|
| [vite.config.ts](../../frontend-s2s/vite.config.ts) | Public development endpoint, HMR, allowed hosts, and backend routing |
|
||||||
| [style.css](../../s2s-client/style.css) | Dark aurora/glass theme, light theme, responsive layout and motion |
|
| [App.tsx](../../frontend-s2s/src/App.tsx) | React layout, voice controls, dialogs, profile and reports sidebar |
|
||||||
| [ui/chat.js](../../s2s-client/ui/chat.js) | Transcript history, tool results, export and autoscroll |
|
| [controller.ts](../../frontend-s2s/src/realtime/controller.ts) | Application state, settings, context, queue, tools and external reasoning |
|
||||||
| [ui/account.js](../../s2s-client/ui/account.js) | Account controls, queue and usage dialogs |
|
| [client.ts](../../frontend-s2s/src/realtime/client.ts) | Realtime protocol, session lifecycle, audio and transcript events |
|
||||||
| [ui/dom.js](../../s2s-client/ui/dom.js) | DOM, escaping and error-display helpers |
|
| [components](../../frontend-s2s/src/components) | Chat, account, settings and voice components |
|
||||||
| [ws/s2s-ws-client.js](../../s2s-client/ws/s2s-ws-client.js) | Realtime events, microphone/playback, session updates, tool-call events and ApiPilot prompt catalog |
|
| [audio](../../frontend-s2s/src/audio) | TypeScript capture/playback worklets, PCM codec, and visualizer |
|
||||||
| [ws/codec.js](../../s2s-client/ws/codec.js) | PCM/base64 conversion and transcript extraction |
|
| [legacy-theme.css](../../frontend-s2s/src/styles/legacy-theme.css) | Retained themes and responsive visual styles alongside Tailwind |
|
||||||
| [ws/orb-visualizer.js](../../s2s-client/ws/orb-visualizer.js) | Audio-reactive orb state |
|
|
||||||
| [auth.py](../../s2s-client/auth.py) | Hugging Face OAuth and caller identity/tier |
|
| [auth.py](../../s2s-client/auth.py) | Hugging Face OAuth and caller identity/tier |
|
||||||
| [limiter.py](../../s2s-client/limiter.py) | SQLite daily talk-time reservations and reconciliation |
|
| [limiter.py](../../s2s-client/limiter.py) | SQLite daily talk-time reservations and reconciliation |
|
||||||
| [flows.json](../../s2s-reasoning-engine/flows.json) | Node-RED reasoning and assistant-response logging |
|
| [flows.json](../../s2s-reasoning-engine/flows.json) | Node-RED reasoning and assistant-response logging |
|
||||||
@@ -65,21 +70,23 @@ audio. It is not an unconditional byte-for-byte tunnel.
|
|||||||
| `/api/session`, `/api/queue/*` | Load-balancer session and waiting-queue proxies |
|
| `/api/session`, `/api/queue/*` | Load-balancer session and waiting-queue proxies |
|
||||||
| `/api/me`, `/api/session/heartbeat`, `/api/session/end` | Identity, reservation extension and usage reconciliation |
|
| `/api/me`, `/api/session/heartbeat`, `/api/session/end` | Identity, reservation extension and usage reconciliation |
|
||||||
|
|
||||||
The root serves the UI. `/docs` remains the FastAPI API explorer, while project
|
The public root serves the React UI. `/__vite_hmr` carries Vite hot updates;
|
||||||
documentation files are served in the `/docs/` subdirectories. Markdown remains
|
`/v1/realtime` remains the speech connection. `/docs` remains the FastAPI API
|
||||||
|
explorer, while project documentation files are served in the `/docs/`
|
||||||
|
subdirectories. Markdown remains
|
||||||
source text, and directories without `index.html` are not browsable pages.
|
source text, and directories without `index.html` are not browsable pages.
|
||||||
Static HTML, JS and CSS
|
The existing static client remains in `s2s-client` for direct FastAPI and Docker
|
||||||
responses receive `Cache-Control: no-store` during development.
|
usage; its static responses retain the existing no-cache headers.
|
||||||
|
|
||||||
## Audio path
|
## Audio path
|
||||||
|
|
||||||
1. The browser captures microphone audio with echo cancellation, noise
|
1. The browser captures microphone audio with echo cancellation, noise
|
||||||
suppression and automatic gain-control settings.
|
suppression and automatic gain-control settings.
|
||||||
2. [mic-capture.js](../../s2s-client/worklets/mic-capture.js) resamples to 16 kHz
|
2. [mic-capture.worklet.ts](../../frontend-s2s/src/audio/mic-capture.worklet.ts) resamples to 16 kHz
|
||||||
mono PCM16. The browser sends base64 `input_audio_buffer.append` frames.
|
mono PCM16. The browser sends base64 `input_audio_buffer.append` frames.
|
||||||
3. The backend processes VAD → STT → LLM → TTS and emits realtime events.
|
3. The backend processes VAD → STT → LLM → TTS and emits realtime events.
|
||||||
4. Assistant PCM16 audio at 24 kHz is decoded and sent to
|
4. Assistant PCM16 audio at the current 16 kHz protocol rate is decoded and sent to
|
||||||
[audio-playback.js](../../s2s-client/worklets/audio-playback.js), which buffers,
|
[audio-playback.worklet.ts](../../frontend-s2s/src/audio/audio-playback.worklet.ts), which buffers,
|
||||||
resamples and fades playback edges. Input speech can clear playback for barge-in.
|
resamples and fades playback edges. Input speech can clear playback for barge-in.
|
||||||
|
|
||||||
Audio worklets are loaded through `AudioWorklet.addModule`; keep their processor
|
Audio worklets are loaded through `AudioWorklet.addModule`; keep their processor
|
||||||
|
|||||||
@@ -22,6 +22,24 @@ In browser DevTools, inspect the WebSocket handshake and `session.created`,
|
|||||||
inside `session`. A cached browser module can disagree with current server
|
inside `session`. A cached browser module can disagree with current server
|
||||||
code; inspect served asset versions and reload with cache disabled when needed.
|
code; inspect served asset versions and reload with cache disabled when needed.
|
||||||
|
|
||||||
|
## Launcher and hot reload
|
||||||
|
|
||||||
|
`start_ui.sh` serves Vite on `8420` and FastAPI on loopback port `8421` by
|
||||||
|
default. Check both processes in the launcher/PM2 logs. If either exits, the
|
||||||
|
launcher stops the other and returns a failure. Port conflicts are errors;
|
||||||
|
use distinct `UI_PORT` and `S2S_BACKEND_PORT` values for additional worktrees.
|
||||||
|
|
||||||
|
A Vite "Blocked request" response means the public hostname is missing from
|
||||||
|
`S2S_DEV_ALLOWED_HOSTS`. Configure it without a scheme or port and restart the
|
||||||
|
launcher. If the page loads but edits do not appear, check the browser's
|
||||||
|
`/__vite_hmr` WebSocket and the reverse proxy's upgrade handling. Speech uses
|
||||||
|
the separate `/v1/realtime` WebSocket on the same public origin.
|
||||||
|
|
||||||
|
For a startup dependency error, check Node/npm on the process manager's PATH,
|
||||||
|
Python venv support, and `setsid`/`curl`. The first start installs dependencies;
|
||||||
|
later starts reuse them until frontend manifests change. Changes to root
|
||||||
|
`.env` or launcher settings require a restart.
|
||||||
|
|
||||||
## The assistant promises a tool but nothing happens
|
## The assistant promises a tool but nothing happens
|
||||||
|
|
||||||
Follow the actual execution mechanism:
|
Follow the actual execution mechanism:
|
||||||
|
|||||||
+22
-7
@@ -9,23 +9,35 @@ reasoning, caller context, ApiPilot tools and load-balancer requests.
|
|||||||
From the repository root:
|
From the repository root:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime ./start_ui.sh
|
S2S_PROXY_TARGET=ws://localhost:8765/v1/realtime ./start_ui.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
The launcher loads the root `.env`, creates `.venv` if needed, installs the
|
The launcher loads the root `.env`, creates `.venv` if needed, installs the
|
||||||
client requirements when the dependency check fails, and runs uvicorn on port
|
client requirements when the dependency check fails, and installs the locked
|
||||||
8420. `UI_PORT` overrides the UI port. Because `.env` is sourced by the script,
|
frontend dependencies when needed. Vite serves the React frontend on port
|
||||||
values in that file can replace variables supplied on the command line.
|
8420 and proxies the backend routes to Uvicorn on `127.0.0.1:8421`.
|
||||||
|
`UI_PORT` overrides the public port; `S2S_BACKEND_PORT` overrides the internal
|
||||||
|
port. Both servers reload source edits, and stopping the launcher stops both.
|
||||||
|
The speech pipeline remains a separate service. Because `.env` is sourced by
|
||||||
|
the script, values in that file can replace variables supplied on the command line.
|
||||||
|
|
||||||
To install and start the UI manually:
|
Use Node.js 22.18+ on the 22.x line or Node.js 24+, npm, Python 3 with venv,
|
||||||
|
Bash 4.4+, `setsid`, and `curl`. Remote staging access also needs the exact
|
||||||
|
hostname in `S2S_DEV_ALLOWED_HOSTS`, HTTPS, and WebSocket forwarding for speech
|
||||||
|
and Vite HMR. See [frontend development setup](../../frontend-s2s/README.md).
|
||||||
|
|
||||||
|
To install and start the backend manually:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python3 -m venv .venv
|
python3 -m venv .venv
|
||||||
.venv/bin/python -m pip install -r s2s-client/requirements.txt
|
.venv/bin/python -m pip install -r s2s-client/requirements.txt
|
||||||
SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime \
|
S2S_PROXY_TARGET=ws://localhost:8765/v1/realtime \
|
||||||
.venv/bin/python -m uvicorn --app-dir s2s-client server:app --port 8420
|
.venv/bin/python -m uvicorn --app-dir s2s-client server:app \
|
||||||
|
--host 127.0.0.1 --port 8421 --reload --reload-dir s2s-client
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Then, in a second terminal, run `cd frontend-s2s`, `npm ci`, and `npm run dev`.
|
||||||
|
|
||||||
Open <http://localhost:8420/>, tap the orb/microphone control, grant microphone
|
Open <http://localhost:8420/>, tap the orb/microphone control, grant microphone
|
||||||
access, and speak. Microphone access requires HTTPS or a local browser origin;
|
access, and speak. Microphone access requires HTTPS or a local browser origin;
|
||||||
use HTTPS when accessing the UI from another machine.
|
use HTTPS when accessing the UI from another machine.
|
||||||
@@ -72,6 +84,9 @@ server; it is not a universally available local service.
|
|||||||
|
|
||||||
## Docker
|
## Docker
|
||||||
|
|
||||||
|
This existing image serves the retained static client. It does not run the
|
||||||
|
React/Vite development launcher.
|
||||||
|
|
||||||
Use the project root as the build context:
|
Use the project root as the build context:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
+80
-41
@@ -6,48 +6,84 @@ rendering and state, including the realtime WebSocket and audio integration.
|
|||||||
|
|
||||||
## Run during development
|
## Run during development
|
||||||
|
|
||||||
Use Node.js 22.18 or later on the 22.x line, or Node.js 24+. Install dependencies
|
From the repository root, run `./start_ui.sh`. Open `http://localhost:8420`.
|
||||||
with npm; the lockfile is committed.
|
Add `?debug=true` to access settings, tools, and custom RAG context. Existing
|
||||||
|
`phoneNumber` and `episode_id` query parameters are supported.
|
||||||
|
|
||||||
1. Start the existing backend using `./start_ui.sh` from your configured S2S
|
The launcher requires Python 3 with venv support, Node.js 22.18 or later on the
|
||||||
checkout, or reuse an already-running instance. It normally listens on port
|
22.x line or Node.js 24+, npm, Bash 4.4+, `setsid` (util-linux), and `curl`.
|
||||||
`8420` and continues serving the original static UI there. Its Python
|
It creates `.venv` and installs missing Python dependencies. It runs
|
||||||
environment, `.env`, speech service, and upstream services must be configured
|
`npm ci --include=dev` on first start and when either frontend package manifest
|
||||||
as before.
|
changes. Subsequent restarts reuse the installed dependencies.
|
||||||
2. In a second terminal, start this frontend:
|
|
||||||
|
|
||||||
```sh
|
| Server | Default address | Purpose |
|
||||||
cd frontend-s2s
|
| ------- | ---------------- | ---------------------------------------------- |
|
||||||
npm ci
|
| Vite | `0.0.0.0:8420` | Public React UI, backend proxy, and hot reload |
|
||||||
npm run dev
|
| FastAPI | `127.0.0.1:8421` | Internal HTTP APIs and speech WebSocket relay |
|
||||||
```
|
|
||||||
|
|
||||||
3. Open `http://127.0.0.1:5173`. Add `?debug=true` to access settings, tools,
|
`UI_PORT` preserves the existing public port override. `UI_HOST` defaults to
|
||||||
and custom RAG context. Existing `phoneNumber` and `episode_id` query
|
`0.0.0.0`; `S2S_BACKEND_PORT` changes the loopback backend port. The two ports
|
||||||
parameters are supported.
|
must differ. Occupied ports fail startup rather than silently selecting a new
|
||||||
|
endpoint. The launcher waits for FastAPI before starting Vite, stops both
|
||||||
|
process groups on Ctrl+C/SIGTERM, and exits with a failure if either server exits.
|
||||||
|
Uvicorn watches only `s2s-client`, avoiding frontend and dependency directories.
|
||||||
|
The Vite child explicitly uses `NODE_ENV=development`, including when a process
|
||||||
|
manager sets `NODE_ENV=production`, so React Fast Refresh remains enabled.
|
||||||
|
|
||||||
Vite serves React and Tailwind with hot updates. It proxies `/api`,
|
Vite serves React and Tailwind with hot updates. It proxies `/api`,
|
||||||
`/v1/realtime` (including WebSockets), OAuth, and backend documentation routes
|
`/v1/realtime` (including WebSockets), OAuth, and backend documentation routes
|
||||||
to FastAPI. The browser uses one origin on port `5173`, so there is no separate
|
to FastAPI. The browser uses one origin on port `8420`, so there is no separate
|
||||||
API origin or development CORS setup. The proxy preserves the incoming Host
|
API origin or development CORS setup. The proxy preserves the incoming Host
|
||||||
header because FastAPI uses it to construct the speech WebSocket URL.
|
header because FastAPI uses it to construct the speech WebSocket URL.
|
||||||
|
|
||||||
For a different backend, copy `.env.example` to `.env.local` and set
|
Runtime UI flags, voices, reasoning mode, prompts, and the speech URL still come from FastAPI's
|
||||||
`S2S_BACKEND_URL`. This is a **Vite server** setting. Runtime UI flags, voices,
|
|
||||||
reasoning mode, prompts, and the speech URL still come from FastAPI's
|
|
||||||
`/api/config`. A backend explicitly configured with `SPEECH_TO_SPEECH_URL`
|
`/api/config`. A backend explicitly configured with `SPEECH_TO_SPEECH_URL`
|
||||||
continues to return that direct speech endpoint.
|
continues to return that direct speech endpoint.
|
||||||
|
|
||||||
The Vite configuration reads only this project's environment files. Do not
|
The launcher sources the root `.env`, retaining its precedence over shell
|
||||||
copy the repository's backend secrets into this directory or into `VITE_*`
|
variables. The Vite configuration reads this project's environment files and
|
||||||
|
the inherited `S2S_*` settings. Do not copy the repository's backend secrets
|
||||||
|
into this directory or into `VITE_*`
|
||||||
variables, which are exposed to browser code.
|
variables, which are exposed to browser code.
|
||||||
|
|
||||||
For access through a development reverse proxy, set the exact hostname in
|
## Staging with hot reload
|
||||||
`S2S_DEV_ALLOWED_HOSTS`. Use `npm run dev -- --host 0.0.0.0` only when Vite must
|
|
||||||
listen beyond localhost. The browser needs HTTPS for microphone access outside
|
PM2 can continue running the root `start_ui.sh` as its existing `s2s_ui` process.
|
||||||
localhost, and that proxy must forward WebSocket upgrades for both speech and
|
Ensure the Node/npm versions above are on PM2's PATH and deploy `frontend-s2s`
|
||||||
Vite HMR. Wiring the staging entrypoint and launcher is a separate step:
|
alongside `s2s-client` and `docs`. The reverse proxy still targets port `8420`.
|
||||||
**this port does not change `start_ui.sh`.**
|
|
||||||
|
Set the exact public hostname in the root `.env`, without scheme or port:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
S2S_DEV_ALLOWED_HOSTS=voice-staging.example.com
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace the example with the existing staging hostname; use commas for multiple
|
||||||
|
hosts. IP addresses and localhost are accepted without this setting.
|
||||||
|
The HTTPS reverse proxy must preserve `Host`, `X-Forwarded-Host` and
|
||||||
|
`X-Forwarded-Proto`, and forward WebSocket upgrades for `/v1/realtime` and
|
||||||
|
`/__vite_hmr`. HMR uses the browser's public origin, including HTTPS/WSS, so no
|
||||||
|
second public port is needed. The browser needs HTTPS for microphone access
|
||||||
|
outside localhost. Restart `s2s_ui` after changing launcher or environment
|
||||||
|
settings; source edits use the reloaders. The speech service remains separately
|
||||||
|
managed.
|
||||||
|
|
||||||
|
## Run the servers separately
|
||||||
|
|
||||||
|
Start FastAPI on `127.0.0.1:8421` as described in the
|
||||||
|
[client usage guide](../docs/client/usage.md), then run:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd frontend-s2s
|
||||||
|
npm ci
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
Standalone Vite binds `127.0.0.1:8420`; add `-- --host 0.0.0.0` for remote
|
||||||
|
access. For a different backend, copy `.env.example` to `.env.local` and set
|
||||||
|
`S2S_BACKEND_URL`. The combined launcher overrides that setting to point at the
|
||||||
|
backend it starts. Do not start standalone Vite while the combined launcher
|
||||||
|
already owns the public port.
|
||||||
|
|
||||||
Component and CSS changes use Fast Refresh. Editing the session controller or
|
Component and CSS changes use Fast Refresh. Editing the session controller or
|
||||||
audio modules can require restarting a conversation or a page reload. Teardown
|
audio modules can require restarting a conversation or a page reload. Teardown
|
||||||
@@ -60,26 +96,29 @@ The scripts and code style follow `frontend-admin` conventions, with Vite and
|
|||||||
React plugins replacing Next.js-specific tooling. Executable configuration
|
React plugins replacing Next.js-specific tooling. Executable configuration
|
||||||
files use TypeScript; TypeScript's own project settings use JSON.
|
files use TypeScript; TypeScript's own project settings use JSON.
|
||||||
|
|
||||||
| Command | Purpose |
|
| Command | Purpose |
|
||||||
| -------------------- | ----------------------------------------------------------- |
|
| ----------------------- | ----------------------------------------------------------- |
|
||||||
| `npm run dev` | Vite development server on port 5173, with a fixed port |
|
| `npm run dev` | Vite development server on port 8420, with a fixed port |
|
||||||
| `npm run check` | Formatting, strict typed ESLint, type checking, and tests |
|
| `npm run check` | Formatting, strict typed ESLint, type checking, and tests |
|
||||||
| `npm run lint:fix` | Apply ESLint fixes |
|
| `npm run lint:fix` | Apply ESLint fixes |
|
||||||
| `npm run format` | Format source and sort Tailwind classes |
|
| `npm run format` | Format source and sort Tailwind classes |
|
||||||
| `npm run typecheck` | Check application, tests, audio worklets, and configuration |
|
| `npm run typecheck` | Check application, tests, audio worklets, and configuration |
|
||||||
| `npm test` | Run Vitest and React Testing Library regression tests |
|
| `npm test` | Run Vitest and React Testing Library regression tests |
|
||||||
| `npm run test:watch` | Watch tests while editing |
|
| `npm run test:watch` | Watch tests while editing |
|
||||||
| `npm run build` | Typecheck and compile the app and worklets into `dist/` |
|
| `npm run test:launcher` | Test launcher supervision and signal handling on Linux |
|
||||||
| `npm run preview` | Inspect the compiled output locally |
|
| `npm run build` | Typecheck and compile the app and worklets into `dist/` |
|
||||||
|
| `npm run preview` | Inspect the compiled output locally |
|
||||||
|
|
||||||
No build step is needed before `npm run dev`. `dist/` is generated and ignored
|
No build step is needed before `npm run dev`. `dist/` is generated and ignored
|
||||||
by Git. Build output is available for later serving integration; this change
|
by Git. The launcher serves source through Vite; it does not use `dist/` or
|
||||||
does not mount it in FastAPI or change the staging launch process.
|
`vite preview`.
|
||||||
|
|
||||||
Tests cover session initialization, response sequencing, cumulative transcripts,
|
Tests cover session initialization, response sequencing, cumulative transcripts,
|
||||||
interruptions, queue cancellation, microphone-permission cancellation, tool
|
interruptions, queue cancellation, microphone-permission cancellation, tool
|
||||||
results, external reasoning and `LOG_ONLY`, late-response cleanup, the idle
|
results, external reasoning and `LOG_ONLY`, late-response cleanup, the idle
|
||||||
timeout, settings, RAG, profiles, and sanitized Markdown/export behavior.
|
timeout, settings, RAG, profiles, and sanitized Markdown/export behavior.
|
||||||
|
Launcher tests start temporary local children to check readiness, environment
|
||||||
|
forwarding, failures, and shutdown of complete process groups.
|
||||||
Vitest handles Node 26's experimental web storage behavior internally.
|
Vitest handles Node 26's experimental web storage behavior internally.
|
||||||
|
|
||||||
## Source layout
|
## Source layout
|
||||||
|
|||||||
@@ -10,19 +10,24 @@ short_description: Voice chat over WebSocket against a HF speech-to-speech
|
|||||||
hf_oauth: true
|
hf_oauth: true
|
||||||
---
|
---
|
||||||
|
|
||||||
# Realtime voice client
|
# Realtime voice backend and retained static client
|
||||||
|
|
||||||
Browser voice-chat UI and FastAPI server for an OpenAI Realtime-compatible
|
FastAPI server and retained static voice-chat UI for an OpenAI Realtime-compatible
|
||||||
speech-to-speech backend. The browser streams microphone audio over WebSocket,
|
speech-to-speech backend. The browser streams microphone audio over WebSocket,
|
||||||
plays assistant audio, displays transcripts, and offers Spanish controls.
|
plays assistant audio, displays transcripts, and offers Spanish controls.
|
||||||
|
|
||||||
|
The root `start_ui.sh` now serves the [React frontend](../frontend-s2s) with
|
||||||
|
Vite on public port `8420`, proxying backend requests here on loopback port
|
||||||
|
`8421`. This directory's static UI remains available when FastAPI is run
|
||||||
|
directly or through the existing Docker image.
|
||||||
|
|
||||||
The complete [usage guide](../docs/client/usage.md) covers local startup,
|
The complete [usage guide](../docs/client/usage.md) covers local startup,
|
||||||
connection modes, Docker, settings, and usage limits. See the
|
connection modes, Docker, settings, and usage limits. See the
|
||||||
[documentation index](../docs/README.md), [architecture](../docs/client/architecture.md),
|
[documentation index](../docs/README.md), [architecture](../docs/client/architecture.md),
|
||||||
and [change history](CHANGELOG.md) for implementation details and history.
|
and [change history](CHANGELOG.md) for implementation details and history.
|
||||||
|
|
||||||
Assistant replies render Markdown with Marked and DOMPurify. Their pinned,
|
The retained static client's assistant replies render Markdown with Marked and
|
||||||
checked-in browser modules are served from `vendor/`, including upstream
|
DOMPurify. Their pinned, checked-in browser modules are served from `vendor/`, including upstream
|
||||||
licenses, so the UI does not require a CDN or a Node.js runtime. To refresh the
|
licenses, so the UI does not require a CDN or a Node.js runtime. To refresh the
|
||||||
browser dependencies with Node.js 20 or newer after updating `package.json`:
|
browser dependencies with Node.js 20 or newer after updating `package.json`:
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user