docs: consolidate English documentation under root docs
This commit is contained in:
@@ -0,0 +1,153 @@
|
||||
# Client usage and configuration
|
||||
|
||||
The browser voice client connects to an existing OpenAI Realtime-compatible
|
||||
speech backend. The UI server can also proxy WebSocket traffic, search,
|
||||
reasoning, caller context, ApiPilot tools and load-balancer requests.
|
||||
|
||||
## Local startup
|
||||
|
||||
From the repository root:
|
||||
|
||||
```bash
|
||||
SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime ./start_ui.sh
|
||||
```
|
||||
|
||||
The launcher loads the root `.env`, creates `.venv` if needed, installs the
|
||||
client requirements when the dependency check fails, and runs uvicorn on port
|
||||
8420. `UI_PORT` overrides the UI port. Because `.env` is sourced by the script,
|
||||
values in that file can replace variables supplied on the command line.
|
||||
|
||||
To install and start the UI manually:
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
.venv/bin/python -m pip install -r s2s-client/requirements.txt
|
||||
SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime \
|
||||
.venv/bin/python -m uvicorn --app-dir s2s-client server:app --port 8420
|
||||
```
|
||||
|
||||
Open <http://localhost:8420/>, tap the orb/microphone control, grant microphone
|
||||
access, and speak. Microphone access requires HTTPS or a local browser origin;
|
||||
use HTTPS when accessing the UI from another machine.
|
||||
|
||||
The speech pipeline has separate model/runtime dependencies and no complete
|
||||
dependency manifest in this checkout. In an already prepared speech
|
||||
environment, a local realtime server can be started with:
|
||||
|
||||
```bash
|
||||
python -m speech_to_speech.s2s_pipeline \
|
||||
--mode realtime \
|
||||
--stt parakeet-tdt \
|
||||
--llm_backend transformers \
|
||||
--tts kokoro \
|
||||
--model_name Qwen/Qwen3-4B-Instruct-2507 \
|
||||
--llm_device cuda \
|
||||
--llm_torch_dtype float16 \
|
||||
--enable_live_transcription
|
||||
```
|
||||
|
||||
Select backends and devices supported by the host; the example model is not a
|
||||
fixed deployment requirement. The default realtime endpoint is
|
||||
`ws://localhost:8765/v1/realtime`, configured through `--ws_host` and `--ws_port`.
|
||||
`--num_pipelines` controls the number of isolated concurrent realtime sessions.
|
||||
See [LLM](../speech/llm.md), [STT](../speech/stt.md), and [TTS](../speech/tts.md).
|
||||
|
||||
## Connection modes
|
||||
|
||||
| Configuration | Browser connection | Settings URL | Load-balancer features |
|
||||
|---|---|---|---|
|
||||
| `SPEECH_TO_SPEECH_URL` set | Direct to pinned URL | Locked | Disabled |
|
||||
| `S2S_PROXY_TARGET` set, no direct URL | Same-origin `/v1/realtime` proxy | Locked | Disabled |
|
||||
| `LOAD_BALANCER_URL` set, neither above set | Per-session compute URL from `/api/session` | Hidden | Queue enabled; metering requires `SPACE_ID` too |
|
||||
| None set | User-provided realtime URL | Editable | Disabled |
|
||||
|
||||
Direct URLs may be full `ws(s)://.../v1/realtime` addresses or host values that
|
||||
the browser normalizes. A direct URL must be reachable **from the browser**.
|
||||
`S2S_PROXY_TARGET` must be reachable **from the UI server**. HTTPS deployments
|
||||
need a secure browser WebSocket URL.
|
||||
|
||||
The launcher supplies `ws://ai-server:12345/v1/realtime` as a proxy-target
|
||||
default when no value is configured. That hostname must resolve from the UI
|
||||
server; it is not a universally available local service.
|
||||
|
||||
## Docker
|
||||
|
||||
Use the project root as the build context:
|
||||
|
||||
```bash
|
||||
docker build -f s2s-client/Dockerfile -t s2s-client .
|
||||
docker run --rm -p 7860:7860 \
|
||||
-e SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime s2s-client
|
||||
```
|
||||
|
||||
For the proxy path, replace the direct setting with a server-reachable
|
||||
`S2S_PROXY_TARGET`. If the upstream is another container, attach the UI to the
|
||||
appropriate existing network and use that service's hostname. An upstream
|
||||
bound only to host loopback is not reachable via an arbitrary container name.
|
||||
|
||||
The image includes `s2s-client` and root `docs`. `/docs` is still the FastAPI API
|
||||
explorer; documentation lives under `/docs/client/`, `/docs/speech/`,
|
||||
`/docs/adr/`, `/docs/examples/`, and `/docs/proposals/`. Legacy `/project/` and
|
||||
`/public/docs.html` redirect to the historical proposal. Markdown is served as a
|
||||
source file rather than rendered through a separate documentation site. Request
|
||||
file paths such as `/docs/README.md`; directories without `index.html` are not
|
||||
browsable pages.
|
||||
|
||||
## Settings and tools
|
||||
|
||||
Voice, editable system instructions and connection settings are kept in browser
|
||||
`localStorage` under `s2s.ws.*` keys. A non-empty `S2S_PRIMARY_PROMPT` pins the
|
||||
instruction field. Restart applies connection/voice changes to a new session.
|
||||
Model voice names are runtime identifiers and stay unchanged across UI
|
||||
translations.
|
||||
|
||||
Web search needs `SERPER_API_KEY` on the server or a user-supplied key in the
|
||||
tools panel. The server proxies requests through `/api/search`; its own key is
|
||||
not returned to the browser. Dynamic ApiPilot tools are configured through
|
||||
`S2S_APIPILOT_TOOLS_URL`; see [tool calling](tool-calling.md).
|
||||
|
||||
Operator RAG text is applied through the RAG dialog and updates live
|
||||
instructions. Caller links may include `phoneNumber=%2B...`; episode links may
|
||||
include `episode_id=...`. See [context injection](context-injection.md) for
|
||||
enabled data sources and limits.
|
||||
|
||||
## Theme and session startup
|
||||
|
||||
`S2S_THEME=white` selects the light interface. `dark` or an empty value selects
|
||||
the dark aurora theme. Theme-specific sections remain hidden until configuration
|
||||
resolves, with a fallback to avoid an invisible page after config failure.
|
||||
|
||||
`S2S_START_SESSION_MODE` controls initial microphone/session startup:
|
||||
|
||||
| Value | Behavior |
|
||||
|---|---|
|
||||
| Empty/default | User taps to start |
|
||||
| `TALK` | After a 2-second warmup, show an audio unlocker; a user gesture starts the session and greeting |
|
||||
| `TALKFIRST` | After a 2-second warmup, attempt automatic start and inject the Spanish wake phrase; microphone failure falls back to the unlocker |
|
||||
|
||||
The warmup is a timer, not proof that all upstream context requests have
|
||||
finished. Browser permission and audio policies can still require a gesture.
|
||||
The greeting uses the same `hola vocero` input as wake-word handling.
|
||||
|
||||
## Queue and daily usage
|
||||
|
||||
Load-balancer mode proxies session acquisition and queue polling. Waiting does
|
||||
not reserve conversation time; a daily budget is reserved when a compute slot
|
||||
is granted. The server enables metering only with both `LOAD_BALANCER_URL` and
|
||||
`SPACE_ID` and with neither pinned direct/proxy mode active.
|
||||
|
||||
| Environment setting | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `LIMIT_ANON_SEC` | 300 | Anonymous seconds per UTC day |
|
||||
| `LIMIT_FREE_SEC` | 600 | Signed-in non-PRO seconds per UTC day |
|
||||
| `UNLIMITED_ORGS` | Adds to built-in organizations | Additional unlimited organization members |
|
||||
| `USAGE_HASH_SECRET` | Random if unset | Identity-key hashing and anonymous cookie signing |
|
||||
|
||||
PRO members are unlimited. Built-in unlimited organizations are `cerebras`,
|
||||
`HuggingFaceM4`, `smolagents`, and `pollen-robotics`; organization matching is
|
||||
case-insensitive. See [auth.py](../../s2s-client/auth.py) and
|
||||
[limiter.py](../../s2s-client/limiter.py) for the current enforcement behavior.
|
||||
|
||||
For external transcript processing, configure the selected
|
||||
[reasoning mode](reasoning.md). For connection, context and audio diagnostics,
|
||||
use [troubleshooting](troubleshooting.md).
|
||||
Reference in New Issue
Block a user