Compare commits
14
Commits
0965077cac
...
3d42df5749
| Author | SHA1 | Date | |
|---|---|---|---|
 alvaro
|
3d42df5749
|
||
|
alvaro
|
c375581077
|
||
|
alvaro
|
179f5ac7db
|
||
|
alvaro
|
f91001169c
|
||
|
alvaro
|
e4b3833770
|
||
|
alvaro
|
6c4ee3a5e3
|
||
|
alvaro
|
99413fc06d
|
||
|
alvaro
|
c8f8aff8c7
|
||
|
alvaro
|
65c965ae6b
|
||
|
alvaro
|
cb09e75bbd
|
||
|
alvaro
|
7285d22e6c
|
||
|
alvaro
|
36efd5e13c
|
||
|
alvaro
|
833788b8d8
|
||
|
alvaro
|
36e2bee885
|
+3
-1
@@ -38,6 +38,8 @@ node_modules/
|
|||||||
.npm/
|
.npm/
|
||||||
package-lock.json
|
package-lock.json
|
||||||
!speech_to_speech/package-lock.json
|
!speech_to_speech/package-lock.json
|
||||||
|
!s2s-frontend/package-lock.json
|
||||||
|
s2s-frontend/.eslintcache
|
||||||
yarn-error.log
|
yarn-error.log
|
||||||
npm-debug.log*
|
npm-debug.log*
|
||||||
yarn-debug.log*
|
yarn-debug.log*
|
||||||
@@ -129,7 +131,7 @@ s2s-reasoning-engine/lib/flows_cred.json
|
|||||||
s2s-reasoning-engine/.config.*
|
s2s-reasoning-engine/.config.*
|
||||||
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
# ─────────────────────────────────────────────────────────────
|
||||||
# s2s-client static artifacts generated at runtime by browser / proxy logs
|
# Client backend and speech-service runtime artifacts
|
||||||
# ─────────────────────────────────────────────────────────────
|
# ─────────────────────────────────────────────────────────────
|
||||||
s2s-client/.cache/
|
s2s-client/.cache/
|
||||||
s2s-client/uploads/
|
s2s-client/uploads/
|
||||||
|
|||||||
@@ -6,7 +6,8 @@ documentation, source identifiers, comments, and internal instructions use Engli
|
|||||||
|
|
||||||
| Component | Purpose |
|
| Component | Purpose |
|
||||||
|---|---|
|
|---|---|
|
||||||
| [s2s-client](s2s-client) | Browser UI, FastAPI server, realtime WebSocket proxy, search and ApiPilot bridges |
|
| [s2s-frontend](s2s-frontend) | React, TypeScript, and Tailwind browser UI, served by Vite with hot reload |
|
||||||
|
| [s2s-client](s2s-client) | FastAPI backend, realtime WebSocket proxy, search and ApiPilot bridges |
|
||||||
| [speech_to_speech](speech_to_speech) | Queue-driven VAD, speech recognition, language model, and speech synthesis pipeline |
|
| [speech_to_speech](speech_to_speech) | Queue-driven VAD, speech recognition, language model, and speech synthesis pipeline |
|
||||||
| [s2s-reasoning-engine](s2s-reasoning-engine) | Optional Node-RED HTTP reasoning and conversation logging flows |
|
| [s2s-reasoning-engine](s2s-reasoning-engine) | Optional Node-RED HTTP reasoning and conversation logging flows |
|
||||||
|
|
||||||
@@ -20,16 +21,24 @@ The speech backend needs its own installed model and runtime dependencies;
|
|||||||
From the repository root, point the client at an existing realtime backend:
|
From the repository root, point the client at an existing realtime backend:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime ./start_ui.sh
|
S2S_PROXY_TARGET=ws://localhost:8765/v1/realtime ./start_ui.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
Open <http://localhost:8420/> and allow microphone access. The launcher creates
|
Open <http://localhost:8420/> and allow microphone access. The launcher creates
|
||||||
`.venv`, installs missing UI requirements, loads the root `.env` if present, and
|
`.venv`, installs missing Python requirements and locked frontend dependencies,
|
||||||
starts the server. For the local WebSocket proxy instead of a direct browser
|
and loads the root `.env` if present. Vite listens on `UI_PORT` (default `8420`)
|
||||||
connection, configure `S2S_PROXY_TARGET` and leave `SPEECH_TO_SPEECH_URL` unset.
|
and forwards HTTP/WebSocket backend routes to FastAPI on `127.0.0.1:8421`.
|
||||||
|
Both servers reload during development; stopping the launcher stops both.
|
||||||
|
Leave `SPEECH_TO_SPEECH_URL` unset to use the same-origin speech proxy.
|
||||||
|
|
||||||
To build the UI container, use the repository root as the Docker build context so
|
The launcher requires Python 3 with venv support, Node.js 22.18+ on the 22.x
|
||||||
the root documentation is included:
|
line or Node.js 24+, npm, Bash 4.4+, `setsid`, and `curl`. For staging hostnames,
|
||||||
|
set `S2S_DEV_ALLOWED_HOSTS` and forward the Vite HMR WebSocket through the existing
|
||||||
|
HTTPS reverse proxy. See [development setup](s2s-frontend/README.md).
|
||||||
|
|
||||||
|
The Docker image serves backend routes and shared documentation. The React
|
||||||
|
frontend runs separately through Vite. To build the backend image, use the
|
||||||
|
repository root as the Docker build context so documentation is included:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker build -f s2s-client/Dockerfile -t s2s-client .
|
docker build -f s2s-client/Dockerfile -t s2s-client .
|
||||||
|
|||||||
+27
-23
@@ -1,18 +1,24 @@
|
|||||||
# Client architecture
|
# Client architecture
|
||||||
|
|
||||||
The [browser client](../../s2s-client) combines a voice UI with a FastAPI server.
|
The [React browser client](../../s2s-frontend) uses the
|
||||||
|
[FastAPI backend](../../s2s-client) for HTTP APIs and a speech WebSocket relay.
|
||||||
|
`start_ui.sh` serves Vite on the existing public port `8420` and FastAPI on
|
||||||
|
loopback port `8421`. Vite forwards backend routes and provides hot reload;
|
||||||
|
Uvicorn reloads Python edits. The launcher supervises both processes.
|
||||||
The speech pipeline and optional Node-RED reasoning service run separately.
|
The speech pipeline and optional Node-RED reasoning service run separately.
|
||||||
Runtime configuration comes from environment variables and the root `.env`.
|
Runtime configuration comes from environment variables and the root `.env`.
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
flowchart LR
|
flowchart LR
|
||||||
Browser[Browser UI]
|
Browser[Browser UI]
|
||||||
|
Vite[Vite on public port 8420]
|
||||||
Server[Client FastAPI server]
|
Server[Client FastAPI server]
|
||||||
Speech[Realtime speech backend]
|
Speech[Realtime speech backend]
|
||||||
Reasoning[Node-RED or HTTP reasoning provider]
|
Reasoning[Node-RED or HTTP reasoning provider]
|
||||||
Tools[ApiPilot backend]
|
Tools[ApiPilot backend]
|
||||||
Browser -->|HTTP configuration and proxies| Server
|
Browser <-->|Frontend and HMR| Vite
|
||||||
Browser <-->|WebSocket via local proxy| Server
|
Vite -->|HTTP configuration and proxies| Server
|
||||||
|
Vite <-->|Speech WebSocket relay| Server
|
||||||
Server <-->|Realtime events and audio| Speech
|
Server <-->|Realtime events and audio| Speech
|
||||||
Browser <-->|Optional direct connection| Speech
|
Browser <-->|Optional direct connection| Speech
|
||||||
Server -->|Optional transcript reasoning| Reasoning
|
Server -->|Optional transcript reasoning| Reasoning
|
||||||
@@ -36,18 +42,15 @@ audio. It is not an unconditional byte-for-byte tunnel.
|
|||||||
|
|
||||||
| Source | Responsibility |
|
| Source | Responsibility |
|
||||||
|---|---|
|
|---|---|
|
||||||
| [server.py](../../s2s-client/server.py) | Configuration, static serving, WebSocket relay, context loading, ApiPilot, reasoning, search, queue and usage routes |
|
| [server.py](../../s2s-client/server.py) | Configuration, documentation serving, WebSocket relay, context loading, ApiPilot, reasoning, search and queue routes |
|
||||||
| [main.js](../../s2s-client/main.js) | UI state, settings, transcripts, caller/report displays, custom RAG context, and external reasoning |
|
| [start_ui.sh](../../start_ui.sh) | Dependency setup, ports, readiness, reload processes, and shutdown |
|
||||||
| [index.html](../../s2s-client/index.html) | Orb and light-theme voice controls, inline chat, dialogs, profile and reports sidebar |
|
| [vite.config.ts](../../s2s-frontend/vite.config.ts) | Public development endpoint, HMR, allowed hosts, and backend routing |
|
||||||
| [style.css](../../s2s-client/style.css) | Dark aurora/glass theme, light theme, responsive layout and motion |
|
| [App.tsx](../../s2s-frontend/src/App.tsx) | React layout, voice controls, dialogs, profile and reports sidebar |
|
||||||
| [ui/chat.js](../../s2s-client/ui/chat.js) | Transcript history, tool results, export and autoscroll |
|
| [controller.ts](../../s2s-frontend/src/realtime/controller.ts) | Application state, settings, context, queue, tools and external reasoning |
|
||||||
| [ui/account.js](../../s2s-client/ui/account.js) | Account controls, queue and usage dialogs |
|
| [client.ts](../../s2s-frontend/src/realtime/client.ts) | Realtime protocol, session lifecycle, audio and transcript events |
|
||||||
| [ui/dom.js](../../s2s-client/ui/dom.js) | DOM, escaping and error-display helpers |
|
| [components](../../s2s-frontend/src/components) | Chat, caller profile, settings and voice components |
|
||||||
| [ws/s2s-ws-client.js](../../s2s-client/ws/s2s-ws-client.js) | Realtime events, microphone/playback, session updates, tool-call events and ApiPilot prompt catalog |
|
| [audio](../../s2s-frontend/src/audio) | TypeScript capture/playback worklets, PCM codec, and visualizer |
|
||||||
| [ws/codec.js](../../s2s-client/ws/codec.js) | PCM/base64 conversion and transcript extraction |
|
| [legacy-theme.css](../../s2s-frontend/src/styles/legacy-theme.css) | Retained themes and responsive visual styles alongside Tailwind |
|
||||||
| [ws/orb-visualizer.js](../../s2s-client/ws/orb-visualizer.js) | Audio-reactive orb state |
|
|
||||||
| [auth.py](../../s2s-client/auth.py) | Hugging Face OAuth and caller identity/tier |
|
|
||||||
| [limiter.py](../../s2s-client/limiter.py) | SQLite daily talk-time reservations and reconciliation |
|
|
||||||
| [flows.json](../../s2s-reasoning-engine/flows.json) | Node-RED reasoning and assistant-response logging |
|
| [flows.json](../../s2s-reasoning-engine/flows.json) | Node-RED reasoning and assistant-response logging |
|
||||||
|
|
||||||
## Server routes
|
## Server routes
|
||||||
@@ -63,23 +66,24 @@ audio. It is not an unconditional byte-for-byte tunnel.
|
|||||||
| `POST /api/apipilot/execute` | Validated tool execution returning a `success/data/error` envelope |
|
| `POST /api/apipilot/execute` | Validated tool execution returning a `success/data/error` envelope |
|
||||||
| `POST /api/search` | Serper web search proxy |
|
| `POST /api/search` | Serper web search proxy |
|
||||||
| `/api/session`, `/api/queue/*` | Load-balancer session and waiting-queue proxies |
|
| `/api/session`, `/api/queue/*` | Load-balancer session and waiting-queue proxies |
|
||||||
| `/api/me`, `/api/session/heartbeat`, `/api/session/end` | Identity, reservation extension and usage reconciliation |
|
|
||||||
|
|
||||||
The root serves the UI. `/docs` remains the FastAPI API explorer, while project
|
The public root serves the React UI. `/__vite_hmr` carries Vite hot updates;
|
||||||
documentation files are served in the `/docs/` subdirectories. Markdown remains
|
`/v1/realtime` remains the speech connection. `/docs` remains the FastAPI API
|
||||||
|
explorer, while project documentation files are served in the `/docs/`
|
||||||
|
subdirectories. Markdown remains
|
||||||
source text, and directories without `index.html` are not browsable pages.
|
source text, and directories without `index.html` are not browsable pages.
|
||||||
Static HTML, JS and CSS
|
FastAPI serves backend and documentation routes only. The frontend source and
|
||||||
responses receive `Cache-Control: no-store` during development.
|
assets live in `s2s-frontend` and are served by Vite.
|
||||||
|
|
||||||
## Audio path
|
## Audio path
|
||||||
|
|
||||||
1. The browser captures microphone audio with echo cancellation, noise
|
1. The browser captures microphone audio with echo cancellation, noise
|
||||||
suppression and automatic gain-control settings.
|
suppression and automatic gain-control settings.
|
||||||
2. [mic-capture.js](../../s2s-client/worklets/mic-capture.js) resamples to 16 kHz
|
2. [mic-capture.worklet.ts](../../s2s-frontend/src/audio/mic-capture.worklet.ts) resamples to 16 kHz
|
||||||
mono PCM16. The browser sends base64 `input_audio_buffer.append` frames.
|
mono PCM16. The browser sends base64 `input_audio_buffer.append` frames.
|
||||||
3. The backend processes VAD → STT → LLM → TTS and emits realtime events.
|
3. The backend processes VAD → STT → LLM → TTS and emits realtime events.
|
||||||
4. Assistant PCM16 audio at 24 kHz is decoded and sent to
|
4. Assistant PCM16 audio at the current 16 kHz protocol rate is decoded and sent to
|
||||||
[audio-playback.js](../../s2s-client/worklets/audio-playback.js), which buffers,
|
[audio-playback.worklet.ts](../../s2s-frontend/src/audio/audio-playback.worklet.ts), which buffers,
|
||||||
resamples and fades playback edges. Input speech can clear playback for barge-in.
|
resamples and fades playback edges. Input speech can clear playback for barge-in.
|
||||||
|
|
||||||
Audio worklets are loaded through `AudioWorklet.addModule`; keep their processor
|
Audio worklets are loaded through `AudioWorklet.addModule`; keep their processor
|
||||||
|
|||||||
@@ -3,8 +3,8 @@
|
|||||||
Context reaches the assistant through browser-built instructions, server-patched
|
Context reaches the assistant through browser-built instructions, server-patched
|
||||||
realtime session instructions, and HTTP reasoning context. The implementation is
|
realtime session instructions, and HTTP reasoning context. The implementation is
|
||||||
in [server.py](../../s2s-client/server.py),
|
in [server.py](../../s2s-client/server.py),
|
||||||
[main.js](../../s2s-client/main.js), and the
|
[context.ts](../../s2s-frontend/src/lib/context.ts), and the
|
||||||
[WebSocket client](../../s2s-client/ws/s2s-ws-client.js).
|
[WebSocket client](../../s2s-frontend/src/realtime/client.ts).
|
||||||
|
|
||||||
## Independent controls
|
## Independent controls
|
||||||
|
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
# Design language
|
# Design language
|
||||||
|
|
||||||
This guide describes the current browser UI. Use
|
This guide describes the current browser UI. Use
|
||||||
[style.css](../../s2s-client/style.css) as the token/selector reference and
|
[legacy-theme.css](../../s2s-frontend/src/styles/legacy-theme.css) as the token/selector
|
||||||
[index.html](../../s2s-client/index.html) for the theme-specific structure.
|
reference and [App.tsx](../../s2s-frontend/src/App.tsx) for the theme-specific structure.
|
||||||
Earlier documentation described a monochrome-only dark UI; the current design
|
Earlier documentation described a monochrome-only dark UI; the current design
|
||||||
also uses aurora gradients, glass surfaces, and a separate light theme.
|
also uses aurora gradients, glass surfaces, and a separate light theme.
|
||||||
|
|
||||||
@@ -36,7 +36,7 @@ selector overrides rather than a second stylesheet.
|
|||||||
| Listening/user speaking | Cyan (`--listening`) |
|
| Listening/user speaking | Cyan (`--listening`) |
|
||||||
| Assistant speaking | Violet (`--speaking`) |
|
| Assistant speaking | Violet (`--speaking`) |
|
||||||
| Processing | Amber (`--processing`) |
|
| Processing | Amber (`--processing`) |
|
||||||
| Ready/authenticated | Green (`--success`) |
|
| Ready/success | Green (`--success`) |
|
||||||
| Error | Red (`--error`) |
|
| Error | Red (`--error`) |
|
||||||
|
|
||||||
`--voice-user`, `--voice-assistant` and `--voice-tool` connect transcript roles
|
`--voice-user`, `--voice-assistant` and `--voice-tool` connect transcript roles
|
||||||
|
|||||||
@@ -34,7 +34,7 @@ names shown in a deployment are configuration examples, not definitions.
|
|||||||
| Queue | Users waiting for an available compute slot; waiting is not yet an active conversation |
|
| Queue | Users waiting for an available compute slot; waiting is not yet an active conversation |
|
||||||
| Ticket | A held queue position returned by the load balancer |
|
| Ticket | A held queue position returned by the load balancer |
|
||||||
| Position | The displayed place in the queue; distinct from a time estimate |
|
| Position | The displayed place in the queue; distinct from a time estimate |
|
||||||
| Claim | The transition from a waiting ticket to an allocated realtime session and usage reservation |
|
| Claim | The transition from a waiting ticket to an allocated realtime session |
|
||||||
| At capacity | A full waiting queue that cannot accept another ticket |
|
| At capacity | A full waiting queue that cannot accept another ticket |
|
||||||
| Barge-in | User speech interrupts an active assistant response, coordinating playback clearing and backend cancellation |
|
| Barge-in | User speech interrupts an active assistant response, coordinating playback clearing and backend cancellation |
|
||||||
| Log-only reasoning | External transcript processing that observes/logs the turn without replacing native speech output |
|
| Log-only reasoning | External transcript processing that observes/logs the turn without replacing native speech output |
|
||||||
|
|||||||
+10
-11
@@ -15,15 +15,15 @@ The mode is normalized case-insensitively. `NODERED` and `NODE-RED` resolve to
|
|||||||
|
|
||||||
## Shared instruction assembly
|
## Shared instruction assembly
|
||||||
|
|
||||||
In [main.js](../../s2s-client/main.js), `effectiveReasoningInstructions()` starts
|
In [context.ts](../../s2s-frontend/src/lib/context.ts), `buildInstructions()` starts
|
||||||
with `pinnedPrompt || settings.instructions`. `/api/config` pins
|
with `config.primaryPrompt || settings.instructions`. `/api/config` pins
|
||||||
`S2S_PRIMARY_PROMPT` when non-empty; otherwise the settings prompt or
|
`S2S_PRIMARY_PROMPT` when non-empty; otherwise the settings prompt or
|
||||||
`DEFAULT_INSTRUCTIONS` is used. The current server does not fetch a remote agent
|
`DEFAULT_INSTRUCTIONS` is used. The current server does not fetch a remote agent
|
||||||
profile to choose this base prompt.
|
profile to choose this base prompt.
|
||||||
|
|
||||||
The browser adds caller context when `injectProfile` is enabled, custom RAG text
|
The browser adds caller context when `injectProfile` is enabled, custom RAG text
|
||||||
when the operator has applied it, and an available ApiPilot catalog block.
|
when the operator has applied it, and an available ApiPilot catalog block.
|
||||||
`effectiveInstructions()` adds `OPENAI_RENDER_HINT` only in `OPENAI` mode.
|
`speechInstructions()` adds the speech-rendering hint only in `OPENAI` mode.
|
||||||
The WebSocket client packages these instructions in `session.update.session`;
|
The WebSocket client packages these instructions in `session.update.session`;
|
||||||
the local proxy may then add server-resolved context. Details belong in
|
the local proxy may then add server-resolved context. Details belong in
|
||||||
[context injection](context-injection.md).
|
[context injection](context-injection.md).
|
||||||
@@ -49,8 +49,8 @@ S2S_REASONING_TYPE=NODE_RED
|
|||||||
S2S_REASONING_NODE_RED_URL=http://localhost:1881
|
S2S_REASONING_NODE_RED_URL=http://localhost:1881
|
||||||
```
|
```
|
||||||
|
|
||||||
`maybeRunOpenAiReasoning()` handles both external modes despite its historical
|
`VoiceController.reason()` in [controller.ts](../../s2s-frontend/src/realtime/controller.ts)
|
||||||
function name. It runs after a complete user transcript and posts a body with
|
handles both external modes. It runs after a complete user transcript and posts a body with
|
||||||
`transcript`, `instructions`, `language`, `user_context`, `phoneNumber`, and
|
`transcript`, `instructions`, `language`, `user_context`, `phoneNumber`, and
|
||||||
`episode_id`. The server builds context and forwards the reasoning fields to
|
`episode_id`. The server builds context and forwards the reasoning fields to
|
||||||
Node-RED with a 30-second timeout.
|
Node-RED with a 30-second timeout.
|
||||||
@@ -84,10 +84,9 @@ browser's native `session.tools` catalog as OpenAI Chat Completions `tools`.
|
|||||||
|
|
||||||
For a non-log-only answer, the browser cancels/suppresses the active native
|
For a non-log-only answer, the browser cancels/suppresses the active native
|
||||||
response and appends the external assistant text. It uses browser
|
response and appends the external assistant text. It uses browser
|
||||||
`SpeechSynthesis` when available. Its compatibility fallback sends
|
`SpeechSynthesis` when available. The React client adds the answer to the realtime
|
||||||
`[VOCERO_RENDER_REPLY]` plus the answer to the realtime backend and requests a
|
conversation as an assistant item; without browser speech synthesis it remains
|
||||||
response. The render hint asks the model to read only the answer after that
|
visible as text.
|
||||||
marker. This instruction is model behavior, not a guarantee of exact playback.
|
|
||||||
|
|
||||||
External reasoning adds an HTTP operation to each completed transcript and has
|
External reasoning adds an HTTP operation to each completed transcript and has
|
||||||
separate provider token usage. It is not the speech pipeline's `responses-api`
|
separate provider token usage. It is not the speech pipeline's `responses-api`
|
||||||
@@ -99,8 +98,8 @@ realtime backend instead.
|
|||||||
| Goal | Source |
|
| Goal | Source |
|
||||||
|---|---|
|
|---|---|
|
||||||
| Change the base agent prompt | `S2S_PRIMARY_PROMPT` or browser settings when unpinned |
|
| Change the base agent prompt | `S2S_PRIMARY_PROMPT` or browser settings when unpinned |
|
||||||
| Change browser context assembly | `effectiveReasoningInstructions()` in `main.js` |
|
| Change browser context assembly | `buildInstructions()` in `s2s-frontend/src/lib/context.ts` |
|
||||||
| Change HTTP provider messages | `reasoning()` in `server.py` |
|
| Change HTTP provider messages | `reasoning()` in `server.py` |
|
||||||
| Change external-answer rendering | `maybeRunOpenAiReasoning()` in `main.js` |
|
| Change external-answer rendering | `VoiceController.reason()` and `RealtimeClient.renderExternalAnswer()` |
|
||||||
| Change Node-RED processing | `s2s-reasoning-engine/flows.json` |
|
| Change Node-RED processing | `s2s-reasoning-engine/flows.json` |
|
||||||
| Change native model/tool generation | `speech_to_speech/LLM` and realtime handlers |
|
| Change native model/tool generation | `speech_to_speech/LLM` and realtime handlers |
|
||||||
|
|||||||
@@ -48,8 +48,9 @@ For a native call, the backend emits:
|
|||||||
```
|
```
|
||||||
|
|
||||||
`arguments` is a JSON-encoded string. The
|
`arguments` is a JSON-encoded string. The
|
||||||
[WebSocket client](../../s2s-client/ws/s2s-ws-client.js) parses it and dispatches
|
[WebSocket client](../../s2s-frontend/src/realtime/client.ts) validates the event and
|
||||||
`toolcall`; [main.js](../../s2s-client/main.js) runs the supported executor.
|
dispatches `tool`; [controller.ts](../../s2s-frontend/src/realtime/controller.ts)
|
||||||
|
parses the arguments and runs the supported executor.
|
||||||
Web search uses `POST /api/search`, keeping the deployment's Serper key on the
|
Web search uses `POST /api/search`, keeping the deployment's Serper key on the
|
||||||
server. Tool output returns to the model as:
|
server. Tool output returns to the model as:
|
||||||
|
|
||||||
|
|||||||
@@ -22,6 +22,24 @@ In browser DevTools, inspect the WebSocket handshake and `session.created`,
|
|||||||
inside `session`. A cached browser module can disagree with current server
|
inside `session`. A cached browser module can disagree with current server
|
||||||
code; inspect served asset versions and reload with cache disabled when needed.
|
code; inspect served asset versions and reload with cache disabled when needed.
|
||||||
|
|
||||||
|
## Launcher and hot reload
|
||||||
|
|
||||||
|
`start_ui.sh` serves Vite on `8420` and FastAPI on loopback port `8421` by
|
||||||
|
default. Check both processes in the launcher/PM2 logs. If either exits, the
|
||||||
|
launcher stops the other and returns a failure. Port conflicts are errors;
|
||||||
|
use distinct `UI_PORT` and `S2S_BACKEND_PORT` values for additional worktrees.
|
||||||
|
|
||||||
|
A Vite "Blocked request" response means the public hostname is missing from
|
||||||
|
`S2S_DEV_ALLOWED_HOSTS`. Configure it without a scheme or port and restart the
|
||||||
|
launcher. If the page loads but edits do not appear, check the browser's
|
||||||
|
`/__vite_hmr` WebSocket and the reverse proxy's upgrade handling. Speech uses
|
||||||
|
the separate `/v1/realtime` WebSocket on the same public origin.
|
||||||
|
|
||||||
|
For a startup dependency error, check Node/npm on the process manager's PATH,
|
||||||
|
Python venv support, and `setsid`/`curl`. The first start installs dependencies;
|
||||||
|
later starts reuse them until frontend manifests change. Changes to root
|
||||||
|
`.env` or launcher settings require a restart.
|
||||||
|
|
||||||
## The assistant promises a tool but nothing happens
|
## The assistant promises a tool but nothing happens
|
||||||
|
|
||||||
Follow the actual execution mechanism:
|
Follow the actual execution mechanism:
|
||||||
@@ -60,7 +78,7 @@ code before claiming a root cause is resolved.
|
|||||||
`S2S_DEBUG_PROMPT=true` enables complete prompt logs, including caller information
|
`S2S_DEBUG_PROMPT=true` enables complete prompt logs, including caller information
|
||||||
and conversation text. Use summaries when full payloads are unnecessary. The
|
and conversation text. Use summaries when full payloads are unnecessary. The
|
||||||
browser's application functions are module-scoped; do not assume names such as
|
browser's application functions are module-scoped; do not assume names such as
|
||||||
`effectiveInstructions()` are available directly on `window` in DevTools.
|
`buildInstructions()` are available directly on `window` in DevTools.
|
||||||
|
|
||||||
## Short gaps or clipped words around tool results
|
## Short gaps or clipped words around tool results
|
||||||
|
|
||||||
@@ -101,15 +119,17 @@ microphone/audio permissions. If automatic startup fails, use the unlocker
|
|||||||
gesture and inspect the permission error rather than increasing the timer
|
gesture and inspect the permission error rather than increasing the timer
|
||||||
without evidence.
|
without evidence.
|
||||||
|
|
||||||
## Read-only source checks
|
## Local validation
|
||||||
|
|
||||||
From the repository root, useful syntax checks are:
|
From the repository root, run the frontend checks and local backend fixtures:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
node --check s2s-client/main.js
|
npm --prefix s2s-frontend run check
|
||||||
node --check s2s-client/ws/s2s-ws-client.js
|
npm --prefix s2s-frontend run build
|
||||||
python -m py_compile s2s-client/server.py
|
.venv/bin/python -m unittest discover -s s2s-client/tests -v
|
||||||
```
|
```
|
||||||
|
|
||||||
Syntax checks do not validate microphone access, network services, model output,
|
These checks cover frontend contracts and local session/queue/WebSocket
|
||||||
or audio quality. Collect a fresh browser/backend reproduction for those paths.
|
fixtures. They do not validate deployed services, real microphone hardware,
|
||||||
|
model output, or audio quality. Use a local browser/backend reproduction for
|
||||||
|
those paths before deployment.
|
||||||
|
|||||||
+37
-23
@@ -9,23 +9,35 @@ reasoning, caller context, ApiPilot tools and load-balancer requests.
|
|||||||
From the repository root:
|
From the repository root:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime ./start_ui.sh
|
S2S_PROXY_TARGET=ws://localhost:8765/v1/realtime ./start_ui.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
The launcher loads the root `.env`, creates `.venv` if needed, installs the
|
The launcher loads the root `.env`, creates `.venv` if needed, installs the
|
||||||
client requirements when the dependency check fails, and runs uvicorn on port
|
client requirements when the dependency check fails, and installs the locked
|
||||||
8420. `UI_PORT` overrides the UI port. Because `.env` is sourced by the script,
|
frontend dependencies when needed. Vite serves the React frontend on port
|
||||||
values in that file can replace variables supplied on the command line.
|
8420 and proxies the backend routes to Uvicorn on `127.0.0.1:8421`.
|
||||||
|
`UI_PORT` overrides the public port; `S2S_BACKEND_PORT` overrides the internal
|
||||||
|
port. Both servers reload source edits, and stopping the launcher stops both.
|
||||||
|
The speech pipeline remains a separate service. Because `.env` is sourced by
|
||||||
|
the script, values in that file can replace variables supplied on the command line.
|
||||||
|
|
||||||
To install and start the UI manually:
|
Use Node.js 22.18+ on the 22.x line or Node.js 24+, npm, Python 3 with venv,
|
||||||
|
Bash 4.4+, `setsid`, and `curl`. Remote staging access also needs the exact
|
||||||
|
hostname in `S2S_DEV_ALLOWED_HOSTS`, HTTPS, and WebSocket forwarding for speech
|
||||||
|
and Vite HMR. See [frontend development setup](../../s2s-frontend/README.md).
|
||||||
|
|
||||||
|
To install and start the backend manually:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python3 -m venv .venv
|
python3 -m venv .venv
|
||||||
.venv/bin/python -m pip install -r s2s-client/requirements.txt
|
.venv/bin/python -m pip install -r s2s-client/requirements.txt
|
||||||
SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime \
|
S2S_PROXY_TARGET=ws://localhost:8765/v1/realtime \
|
||||||
.venv/bin/python -m uvicorn --app-dir s2s-client server:app --port 8420
|
.venv/bin/python -m uvicorn --app-dir s2s-client server:app \
|
||||||
|
--host 127.0.0.1 --port 8421 --reload --reload-dir s2s-client
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Then, in a second terminal, run `cd s2s-frontend`, `npm ci`, and `npm run dev`.
|
||||||
|
|
||||||
Open <http://localhost:8420/>, tap the orb/microphone control, grant microphone
|
Open <http://localhost:8420/>, tap the orb/microphone control, grant microphone
|
||||||
access, and speak. Microphone access requires HTTPS or a local browser origin;
|
access, and speak. Microphone access requires HTTPS or a local browser origin;
|
||||||
use HTTPS when accessing the UI from another machine.
|
use HTTPS when accessing the UI from another machine.
|
||||||
@@ -58,7 +70,7 @@ See [LLM](../speech/llm.md), [STT](../speech/stt.md), and [TTS](../speech/tts.md
|
|||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| `SPEECH_TO_SPEECH_URL` set | Direct to pinned URL | Locked | Disabled |
|
| `SPEECH_TO_SPEECH_URL` set | Direct to pinned URL | Locked | Disabled |
|
||||||
| `S2S_PROXY_TARGET` set, no direct URL | Same-origin `/v1/realtime` proxy | Locked | Disabled |
|
| `S2S_PROXY_TARGET` set, no direct URL | Same-origin `/v1/realtime` proxy | Locked | Disabled |
|
||||||
| `LOAD_BALANCER_URL` set, neither above set | Per-session compute URL from `/api/session` | Hidden | Queue enabled; metering requires `SPACE_ID` too |
|
| `LOAD_BALANCER_URL` set, neither above set | Per-session compute URL from `/api/session` | Hidden | Allocation, queue polling and cancellation |
|
||||||
| None set | User-provided realtime URL | Editable | Disabled |
|
| None set | User-provided realtime URL | Editable | Disabled |
|
||||||
|
|
||||||
Direct URLs may be full `ws(s)://.../v1/realtime` addresses or host values that
|
Direct URLs may be full `ws(s)://.../v1/realtime` addresses or host values that
|
||||||
@@ -72,6 +84,10 @@ server; it is not a universally available local service.
|
|||||||
|
|
||||||
## Docker
|
## Docker
|
||||||
|
|
||||||
|
This image serves FastAPI backend routes and shared documentation. Run the
|
||||||
|
React frontend separately with Vite and point `S2S_BACKEND_URL` at the exposed
|
||||||
|
backend address. The image does not run the development launcher or serve a UI.
|
||||||
|
|
||||||
Use the project root as the build context:
|
Use the project root as the build context:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -129,24 +145,22 @@ The warmup is a timer, not proof that all upstream context requests have
|
|||||||
finished. Browser permission and audio policies can still require a gesture.
|
finished. Browser permission and audio policies can still require a gesture.
|
||||||
The greeting uses the same `hola vocero` input as wake-word handling.
|
The greeting uses the same `hola vocero` input as wake-word handling.
|
||||||
|
|
||||||
## Queue and daily usage
|
## Queue and session lifecycle
|
||||||
|
|
||||||
Load-balancer mode proxies session acquisition and queue polling. Waiting does
|
Load-balancer mode proxies `POST /api/session`. A free compute slot returns a
|
||||||
not reserve conversation time; a daily budget is reserved when a compute slot
|
grant containing `session_id`, `connect_url`, and `pending_timeout_s`. Otherwise
|
||||||
is granted. The server enables metering only with both `LOAD_BALANCER_URL` and
|
the browser receives a queue ticket and polls `GET /api/queue/{queue_id}` until
|
||||||
`SPACE_ID` and with neither pinned direct/proxy mode active.
|
a slot is granted. The browser offers a timed join action after waiting.
|
||||||
|
|
||||||
| Environment setting | Default | Purpose |
|
`DELETE /api/queue/{queue_id}` cancels a waiting ticket. On session stop or page
|
||||||
|---|---|---|
|
exit, the browser uses `POST /api/queue/end` through `sendBeacon`, falling back
|
||||||
| `LIMIT_ANON_SEC` | 300 | Anonymous seconds per UTC day |
|
to a keepalive fetch. Expired tickets and missed join deadlines require a new
|
||||||
| `LIMIT_FREE_SEC` | 600 | Signed-in non-PRO seconds per UTC day |
|
attempt. Full queues return `503` with `state: "at_capacity"`.
|
||||||
| `UNLIMITED_ORGS` | Adds to built-in organizations | Additional unlimited organization members |
|
|
||||||
| `USAGE_HASH_SECRET` | Random if unset | Identity-key hashing and anonymous cookie signing |
|
|
||||||
|
|
||||||
PRO members are unlimited. Built-in unlimited organizations are `cerebras`,
|
Once connected, session teardown closes the microphone, audio graph and
|
||||||
`HuggingFaceM4`, `smolagents`, and `pollen-robotics`; organization matching is
|
WebSocket. The 30-second inactivity timeout remains active when the assistant
|
||||||
case-insensitive. See [auth.py](../../s2s-client/auth.py) and
|
is not speaking. Caller-profile lookup and upstream authorization operate
|
||||||
[limiter.py](../../s2s-client/limiter.py) for the current enforcement behavior.
|
independently of allocation and use their existing configured endpoints.
|
||||||
|
|
||||||
For external transcript processing, configure the selected
|
For external transcript processing, configure the selected
|
||||||
[reasoning mode](reasoning.md). For connection, context and audio diagnostics,
|
[reasoning mode](reasoning.md). For connection, context and audio diagnostics,
|
||||||
|
|||||||
@@ -6,7 +6,7 @@
|
|||||||
.idea/
|
.idea/
|
||||||
*.swp
|
*.swp
|
||||||
|
|
||||||
# Static HTTP server logs (we don't ship them)
|
# Server logs
|
||||||
*.log
|
*.log
|
||||||
|
|
||||||
# Python
|
# Python
|
||||||
@@ -14,8 +14,6 @@ __pycache__/
|
|||||||
*.pyc
|
*.pyc
|
||||||
.venv/
|
.venv/
|
||||||
venv/
|
venv/
|
||||||
node_modules/
|
|
||||||
!package-lock.json
|
|
||||||
|
|
||||||
# Local-only replica deploy script (never tracked)
|
# Local-only replica deploy script (never tracked)
|
||||||
deploy_replica.py
|
deploy_replica.py
|
||||||
|
|||||||
@@ -9,6 +9,11 @@ the implementation at the recorded date; current operating instructions live in
|
|||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
|
||||||
|
- Removed the legacy browser site in favor of the React application in
|
||||||
|
`s2s-frontend`. FastAPI now serves only backend and documentation routes.
|
||||||
|
- Retired daily talk-time accounting and the associated Hugging Face account
|
||||||
|
UI. Load-balancer allocation, queue lifecycle, caller profiles, upstream
|
||||||
|
authentication, and voice-session cleanup remain available.
|
||||||
- Consolidated technical documentation under the root `docs/` directory and
|
- Consolidated technical documentation under the root `docs/` directory and
|
||||||
replaced translated changelog copies with this English history.
|
replaced translated changelog copies with this English history.
|
||||||
- Moved the agent-profile and reference-audio examples into `docs/examples/`.
|
- Moved the agent-profile and reference-audio examples into `docs/examples/`.
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
# Build from the project root: docker build -f s2s-client/Dockerfile -t s2s-client .
|
# Build from the project root: docker build -f s2s-client/Dockerfile -t s2s-client .
|
||||||
# Serve the client, same-origin API proxies, and shared project documentation.
|
# Serve backend APIs, the WebSocket proxy, and shared project documentation.
|
||||||
|
# Run the React/Vite frontend separately; this image contains no browser UI.
|
||||||
# HF Spaces (sdk: docker) routes traffic to $PORT, default 7860.
|
# HF Spaces (sdk: docker) routes traffic to $PORT, default 7860.
|
||||||
FROM python:3.11-slim
|
FROM python:3.11-slim
|
||||||
|
|
||||||
|
|||||||
+21
-20
@@ -7,40 +7,41 @@ sdk: docker
|
|||||||
app_port: 7860
|
app_port: 7860
|
||||||
pinned: false
|
pinned: false
|
||||||
short_description: Voice chat over WebSocket against a HF speech-to-speech
|
short_description: Voice chat over WebSocket against a HF speech-to-speech
|
||||||
hf_oauth: true
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Realtime voice client
|
# Realtime voice backend
|
||||||
|
|
||||||
Browser voice-chat UI and FastAPI server for an OpenAI Realtime-compatible
|
FastAPI APIs and WebSocket relay for the [React voice frontend](../s2s-frontend).
|
||||||
speech-to-speech backend. The browser streams microphone audio over WebSocket,
|
This service handles configuration, caller context, upstream authentication,
|
||||||
plays assistant audio, displays transcripts, and offers Spanish controls.
|
search, reasoning, ApiPilot tools, and optional load-balancer allocation.
|
||||||
|
|
||||||
|
The root `start_ui.sh` now serves the [React frontend](../s2s-frontend) with
|
||||||
|
Vite on public port `8420`, proxying backend requests here on loopback port
|
||||||
|
`8421`. The browser UI lives entirely in `s2s-frontend`; running FastAPI directly
|
||||||
|
or through Docker exposes only backend routes and shared documentation.
|
||||||
|
|
||||||
The complete [usage guide](../docs/client/usage.md) covers local startup,
|
The complete [usage guide](../docs/client/usage.md) covers local startup,
|
||||||
connection modes, Docker, settings, and usage limits. See the
|
connection modes, Docker, settings, and queue behavior. See the
|
||||||
[documentation index](../docs/README.md), [architecture](../docs/client/architecture.md),
|
[documentation index](../docs/README.md), [architecture](../docs/client/architecture.md),
|
||||||
and [change history](CHANGELOG.md) for implementation details and history.
|
and [change history](CHANGELOG.md) for implementation details and history.
|
||||||
|
|
||||||
Assistant replies render Markdown with Marked and DOMPurify. Their pinned,
|
|
||||||
checked-in browser modules are served from `vendor/`, including upstream
|
|
||||||
licenses, so the UI does not require a CDN or a Node.js runtime. To refresh the
|
|
||||||
browser dependencies with Node.js 20 or newer after updating `package.json`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd s2s-client
|
|
||||||
npm install --ignore-scripts
|
|
||||||
npm run vendor
|
|
||||||
```
|
|
||||||
|
|
||||||
Commit `package-lock.json` and the generated `vendor/` files together. Existing
|
|
||||||
locked versions can be restored with `npm ci --ignore-scripts`.
|
|
||||||
|
|
||||||
Build from the repository root so the container includes the documentation:
|
Build from the repository root so the container includes the documentation:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker build -f s2s-client/Dockerfile -t s2s-client .
|
docker build -f s2s-client/Dockerfile -t s2s-client .
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Run the backend regression tests from the repository root after installing
|
||||||
|
`requirements.txt` in `.venv`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/bin/python -m unittest discover -s s2s-client/tests -v
|
||||||
|
```
|
||||||
|
|
||||||
|
Tests isolate environment configuration, use HTTP fixtures for allocation and
|
||||||
|
caller authentication, and start a loopback WebSocket service to verify audio
|
||||||
|
frames, keepalive, and teardown. No deployed services or database are needed.
|
||||||
|
|
||||||
FastAPI's API explorer remains at `/docs`. Project documents are served as files below
|
FastAPI's API explorer remains at `/docs`. Project documents are served as files below
|
||||||
`/docs/client/`, `/docs/speech/`, `/docs/adr/`, `/docs/examples/`, and
|
`/docs/client/`, `/docs/speech/`, `/docs/adr/`, `/docs/examples/`, and
|
||||||
`/docs/proposals/`. The historical `/project/` and `/public/docs.html` URLs
|
`/docs/proposals/`. The historical `/project/` and `/public/docs.html` URLs
|
||||||
|
|||||||
@@ -1,229 +0,0 @@
|
|||||||
"""
|
|
||||||
HF OAuth + per-request identity for the duration limiter.
|
|
||||||
|
|
||||||
Login uses Hugging Face's native Spaces OAuth via `huggingface_hub`
|
|
||||||
(`attach_huggingface_oauth` / `parse_huggingface_oauth`). The OAuth env
|
|
||||||
(`OAUTH_CLIENT_ID`, ...) is injected by the platform when the Space README sets
|
|
||||||
`hf_oauth: true`, so this only activates on a deployed Space — locally and in
|
|
||||||
direct mode there's no OAuth and the limiter treats everyone as anonymous.
|
|
||||||
|
|
||||||
Identity:
|
|
||||||
- signed in -> tier 'pro' | 'free', keyed by hashed HF `sub`
|
|
||||||
- anonymous -> tier 'anon', keyed by BOTH hashed client IP and a hashed
|
|
||||||
signed-cookie id (OR-matched in the limiter)
|
|
||||||
"""
|
|
||||||
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
import secrets
|
|
||||||
|
|
||||||
import limiter
|
|
||||||
|
|
||||||
logger = logging.getLogger("s2s.auth")
|
|
||||||
|
|
||||||
# huggingface_hub adds these routes when OAuth is attached. Centralised so a
|
|
||||||
# version change is a one-line fix; the paths are handed to the client via
|
|
||||||
# /api/me rather than hardcoded there.
|
|
||||||
OAUTH_LOGIN_PATH = "/oauth/huggingface/login"
|
|
||||||
OAUTH_LOGOUT_PATH = "/oauth/huggingface/logout"
|
|
||||||
|
|
||||||
ANON_COOKIE = "s2s_anon"
|
|
||||||
_COOKIE_MAX_AGE = 60 * 60 * 24 * 30 # 30 days
|
|
||||||
|
|
||||||
|
|
||||||
# Members of these orgs get unlimited usage (like PRO) out of the box. The
|
|
||||||
# UNLIMITED_ORGS env adds to this set; it doesn't replace it.
|
|
||||||
_DEFAULT_UNLIMITED_ORGS = {"cerebras", "huggingfacem4", "smolagents", "pollen-robotics"}
|
|
||||||
|
|
||||||
|
|
||||||
def _unlimited_orgs() -> "set[str]":
|
|
||||||
"""Org usernames whose members get unlimited usage (like PRO).
|
|
||||||
|
|
||||||
Defaults to {cerebras, HuggingFaceM4, smolagents}; the UNLIMITED_ORGS env
|
|
||||||
(comma/space-separated, e.g. `UNLIMITED_ORGS=my-team`) adds more. Matched
|
|
||||||
case-insensitively against the signed-in user's organisations."""
|
|
||||||
raw = os.environ.get("UNLIMITED_ORGS", "")
|
|
||||||
extra = {o.strip().lower() for o in raw.replace(",", " ").split() if o.strip()}
|
|
||||||
return _DEFAULT_UNLIMITED_ORGS | extra
|
|
||||||
|
|
||||||
try:
|
|
||||||
from huggingface_hub import attach_huggingface_oauth, parse_huggingface_oauth
|
|
||||||
_OAUTH_IMPORTABLE = True
|
|
||||||
except Exception as exc: # pragma: no cover - import guard
|
|
||||||
logger.info("huggingface_hub OAuth unavailable (%s); sign-in disabled.", exc)
|
|
||||||
_OAUTH_IMPORTABLE = False
|
|
||||||
|
|
||||||
# Set by attach(): True once OAuth is actually wired (importable + env present).
|
|
||||||
oauth_enabled = False
|
|
||||||
|
|
||||||
|
|
||||||
def attach(app) -> bool:
|
|
||||||
"""Wire HF OAuth onto the app if it's importable and configured. Returns
|
|
||||||
whether sign-in is available."""
|
|
||||||
global oauth_enabled
|
|
||||||
if not _OAUTH_IMPORTABLE or not os.environ.get("OAUTH_CLIENT_ID"):
|
|
||||||
return False
|
|
||||||
try:
|
|
||||||
attach_huggingface_oauth(app)
|
|
||||||
oauth_enabled = True
|
|
||||||
logger.info("HF OAuth attached (sign-in enabled).")
|
|
||||||
except Exception as exc: # pragma: no cover - defensive
|
|
||||||
logger.warning("Failed to attach HF OAuth: %r", exc)
|
|
||||||
oauth_enabled = False
|
|
||||||
return oauth_enabled
|
|
||||||
|
|
||||||
|
|
||||||
def _field(obj, name, default=None):
|
|
||||||
"""Read a field whether the user-info is an object or a dict."""
|
|
||||||
if obj is None:
|
|
||||||
return default
|
|
||||||
if isinstance(obj, dict):
|
|
||||||
return obj.get(name, default)
|
|
||||||
return getattr(obj, name, default)
|
|
||||||
|
|
||||||
|
|
||||||
# Surface what we detect (orgs, tier) in logs and on /api/me when set. Handy for
|
|
||||||
# verifying org gating on the live Space without guessing.
|
|
||||||
AUTH_DEBUG = bool(os.environ.get("AUTH_DEBUG"))
|
|
||||||
|
|
||||||
# whoami-v2 org lookups are cached for the process lifetime, keyed by token, so
|
|
||||||
# /api/me + /api/session don't each hit the Hub.
|
|
||||||
_orgs_cache: "dict[str, set[str]]" = {}
|
|
||||||
|
|
||||||
|
|
||||||
def current_oauth(request):
|
|
||||||
"""The parsed HF OAuth info (user_info + access_token), or None."""
|
|
||||||
if not oauth_enabled:
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
return parse_huggingface_oauth(request)
|
|
||||||
except Exception:
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def current_user(request):
|
|
||||||
"""The signed-in HF user-info, or None."""
|
|
||||||
return _field(current_oauth(request), "user_info")
|
|
||||||
|
|
||||||
|
|
||||||
def _user_org_names(user) -> "set[str]":
|
|
||||||
"""The user's organisations from the OAuth userinfo, by username/name/id."""
|
|
||||||
names = set()
|
|
||||||
for org in _field(user, "orgs", []) or []:
|
|
||||||
for key in ("preferred_username", "name", "sub"):
|
|
||||||
val = _field(org, key)
|
|
||||||
if val:
|
|
||||||
names.add(str(val).lower())
|
|
||||||
return names
|
|
||||||
|
|
||||||
|
|
||||||
def _orgs_via_token(token: str) -> "set[str]":
|
|
||||||
"""Fallback org lookup via the Hub `whoami-v2` API, using the user's OAuth
|
|
||||||
access token. Covers the case where the userinfo claim omits `orgs`."""
|
|
||||||
if not token:
|
|
||||||
return set()
|
|
||||||
if token in _orgs_cache:
|
|
||||||
return _orgs_cache[token]
|
|
||||||
names: "set[str]" = set()
|
|
||||||
try:
|
|
||||||
import httpx
|
|
||||||
|
|
||||||
resp = httpx.get(
|
|
||||||
"https://huggingface.co/api/whoami-v2",
|
|
||||||
headers={"Authorization": f"Bearer {token}"},
|
|
||||||
timeout=5.0,
|
|
||||||
)
|
|
||||||
resp.raise_for_status()
|
|
||||||
for org in resp.json().get("orgs", []) or []:
|
|
||||||
for key in ("name", "fullname"):
|
|
||||||
val = org.get(key)
|
|
||||||
if val:
|
|
||||||
names.add(str(val).lower())
|
|
||||||
except Exception as exc: # pragma: no cover - network/permission dependent
|
|
||||||
logger.info("whoami-v2 org lookup failed: %r", exc)
|
|
||||||
_orgs_cache[token] = names
|
|
||||||
return names
|
|
||||||
|
|
||||||
|
|
||||||
def _org_names(user, token=None, allow=None) -> "set[str]":
|
|
||||||
"""The user's org usernames from the OAuth userinfo claim. If that doesn't
|
|
||||||
already satisfy `allow`, fall back to the Hub `whoami-v2` API (the claim is
|
|
||||||
often empty or partial), so membership is resolved either way."""
|
|
||||||
names = _user_org_names(user)
|
|
||||||
if token and (allow is None or not (allow & names)):
|
|
||||||
names = names | _orgs_via_token(token)
|
|
||||||
return names
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_tier(user, token=None) -> str:
|
|
||||||
"""Tier for a signed-in user: 'pro' (paying), 'org' (allow-listed org
|
|
||||||
member, unlimited), or 'free'. PRO wins over org if both apply."""
|
|
||||||
if bool(_field(user, "is_pro", False)):
|
|
||||||
return "pro"
|
|
||||||
allow = _unlimited_orgs()
|
|
||||||
names = _org_names(user, token, allow)
|
|
||||||
tier = "org" if (allow & names) else "free"
|
|
||||||
if AUTH_DEBUG:
|
|
||||||
logger.info("tier=%s orgs=%s allow=%s", tier, sorted(names), sorted(allow))
|
|
||||||
return tier
|
|
||||||
|
|
||||||
|
|
||||||
def user_view(request) -> dict:
|
|
||||||
"""Public profile for /api/me."""
|
|
||||||
info = current_oauth(request)
|
|
||||||
user = _field(info, "user_info")
|
|
||||||
if not user:
|
|
||||||
return {"loggedIn": False, "tier": "anon"}
|
|
||||||
token = _field(info, "access_token")
|
|
||||||
out = {
|
|
||||||
"loggedIn": True,
|
|
||||||
"username": _field(user, "preferred_username") or _field(user, "name") or "tú",
|
|
||||||
"avatar": _field(user, "picture"),
|
|
||||||
"tier": resolve_tier(user, token),
|
|
||||||
}
|
|
||||||
if AUTH_DEBUG:
|
|
||||||
out["orgs"] = sorted(_org_names(user, token))
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _client_ip(request) -> str:
|
|
||||||
"""Real client IP. On HF the app sits behind a proxy, so the user's address
|
|
||||||
is the first hop in X-Forwarded-For, not request.client.host."""
|
|
||||||
xff = request.headers.get("x-forwarded-for", "")
|
|
||||||
if xff:
|
|
||||||
return xff.split(",")[0].strip()
|
|
||||||
return request.client.host if request.client else "unknown"
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_identity(request):
|
|
||||||
"""Resolve (tier, keys, set_cookie) for this request.
|
|
||||||
|
|
||||||
`keys` are the limiter usage_daily keys to debit (one for signed-in, two for
|
|
||||||
anonymous). `set_cookie` is a signed value to Set-Cookie when we minted a new
|
|
||||||
anonymous id, else None.
|
|
||||||
"""
|
|
||||||
info = current_oauth(request)
|
|
||||||
user = _field(info, "user_info")
|
|
||||||
if user:
|
|
||||||
sub = _field(user, "sub") or _field(user, "preferred_username")
|
|
||||||
token = _field(info, "access_token")
|
|
||||||
return resolve_tier(user, token), [limiter.hash_key(f"sub:{sub}")], None
|
|
||||||
|
|
||||||
# Anonymous: key by IP and a signed cookie id, minting the cookie if absent.
|
|
||||||
ip = _client_ip(request)
|
|
||||||
cookie_id = limiter.verify_cookie(request.cookies.get(ANON_COOKIE, ""))
|
|
||||||
set_cookie = None
|
|
||||||
if not cookie_id:
|
|
||||||
cookie_id = secrets.token_urlsafe(18)
|
|
||||||
set_cookie = limiter.sign_cookie(cookie_id)
|
|
||||||
keys = [limiter.hash_key(f"ip:{ip}"), limiter.hash_key(f"cookie:{cookie_id}")]
|
|
||||||
return "anon", keys, set_cookie
|
|
||||||
|
|
||||||
|
|
||||||
def set_anon_cookie(response, signed: str) -> None:
|
|
||||||
# The Space runs inside an iframe on huggingface.co, so the cookie lives in a
|
|
||||||
# cross-site context — it must be SameSite=None; Secure or the browser drops it.
|
|
||||||
response.set_cookie(
|
|
||||||
ANON_COOKIE, signed,
|
|
||||||
max_age=_COOKIE_MAX_AGE, httponly=True, samesite="none", secure=True,
|
|
||||||
)
|
|
||||||
@@ -1,700 +0,0 @@
|
|||||||
<!doctype html>
|
|
||||||
<html lang="es">
|
|
||||||
<head>
|
|
||||||
<meta charset="UTF-8" />
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
|
|
||||||
<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate, max-age=0" />
|
|
||||||
<meta http-equiv="Pragma" content="no-cache" />
|
|
||||||
<meta http-equiv="Expires" content="0" />
|
|
||||||
<title>Vocero · Conversación de voz</title>
|
|
||||||
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
|
||||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
|
|
||||||
<link
|
|
||||||
href="https://fonts.googleapis.com/css2?family=Geist+Mono:wght@400;500;600&family=Inter:wght@400;500;600;700;800&display=swap"
|
|
||||||
rel="stylesheet"
|
|
||||||
/>
|
|
||||||
<!-- ═══════════════════════════════════════════════════════════════════
|
|
||||||
ANTI-FOUC VEIL (S2S_THEME flash fix) — MUST stay INLINE and
|
|
||||||
BEFORE the <link rel="stylesheet" href="style.css"> tag below.
|
|
||||||
|
|
||||||
Problem reported by user 2026-09-10:
|
|
||||||
"When I refresh, the previous purple theme briefly appears."
|
|
||||||
|
|
||||||
The earlier guard `html:not([data-theme]) [data-theme-only]` only
|
|
||||||
hid the two parallel widget branches, but the UNTHEMED DEFAULTS in
|
|
||||||
style.css (:root purple palette, body background #07080d, topbar
|
|
||||||
glass, #app deep-space colours, purple orb border gradients) were
|
|
||||||
still rendering for the ~1-2 frames between body paint commit and
|
|
||||||
fetchConfig() returning api/config.theme → setAttribute("data-theme").
|
|
||||||
The user saw a dark-purple flash before the white Vocero theme
|
|
||||||
applied — exactly the symptom described.
|
|
||||||
|
|
||||||
FIX (3 layers, fail-safe):
|
|
||||||
1. While html has NO data-theme attribute (pre-fetchConfig),
|
|
||||||
render <body> COMPLETELY INVISIBLE + background:white +
|
|
||||||
border:none (the two most visible paint leaks fixed first).
|
|
||||||
Display:none would block CSS transitions from being recorded;
|
|
||||||
visibility:hidden keeps layout but forbids painting entirely,
|
|
||||||
which is exactly what we want.
|
|
||||||
2. Safety timeout: if fetchConfig() fails to resolve (e.g.
|
|
||||||
static hosting / no server / offline) the veil auto-lifts
|
|
||||||
after 2500 ms so the user is NOT stuck on a blank page.
|
|
||||||
3. Immediately after root.setAttribute("data-theme", …) in
|
|
||||||
fetchConfig() we also add class="theme-ready" which the veil
|
|
||||||
additionally checks so the user sees a 1-frame lift, not a
|
|
||||||
delayed paint. The visibility guard is a double-OR so it
|
|
||||||
expires on EITHER signal.
|
|
||||||
═══════════════════════════════════════════════════════════════════ -->
|
|
||||||
<style id="s2s-anti-fouc-veil">
|
|
||||||
html:not([data-theme]) body,
|
|
||||||
html:not(.theme-ready) body {
|
|
||||||
visibility: hidden !important;
|
|
||||||
background: #ffffff !important;
|
|
||||||
color: #0f172a !important;
|
|
||||||
opacity: 0 !important;
|
|
||||||
transition: none !important;
|
|
||||||
animation: none !important;
|
|
||||||
}
|
|
||||||
html:not([data-theme]),
|
|
||||||
html:not(.theme-ready) {
|
|
||||||
background: #ffffff !important;
|
|
||||||
}
|
|
||||||
/* Also kill the purple orb + any glass chrome during the veil window,
|
|
||||||
because even when <body> is visibility:hidden some browsers do a
|
|
||||||
first compositing pass that leaks a few pixels of background. */
|
|
||||||
html:not([data-theme]) #main-circle,
|
|
||||||
html:not([data-theme]) .orb-wrap,
|
|
||||||
html:not([data-theme]) .stage,
|
|
||||||
html:not([data-theme]) .topbar,
|
|
||||||
html:not([data-theme]) #app,
|
|
||||||
html:not(.theme-ready) #main-circle,
|
|
||||||
html:not(.theme-ready) .orb-wrap,
|
|
||||||
html:not(.theme-ready) .stage,
|
|
||||||
html:not(.theme-ready) .topbar,
|
|
||||||
html:not(.theme-ready) #app {
|
|
||||||
background: transparent !important;
|
|
||||||
background-image: none !important;
|
|
||||||
backdrop-filter: none !important;
|
|
||||||
-webkit-backdrop-filter: none !important;
|
|
||||||
box-shadow: none !important;
|
|
||||||
border: none !important;
|
|
||||||
color: transparent !important;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
<script>
|
|
||||||
// Fail-safe veil lift: if fetchConfig() fails to reply (static hosting,
|
|
||||||
// offline, api/config 500) the user never sees <body> stuck invisible.
|
|
||||||
// setTimeout (not rAF) because we want a real wall-clock timeout.
|
|
||||||
window.__s2sVeilLiftTimeout = window.setTimeout(function () {
|
|
||||||
var r = document.documentElement;
|
|
||||||
if (!r.hasAttribute("data-theme")) r.setAttribute("data-theme", "dark");
|
|
||||||
r.classList.add("theme-ready");
|
|
||||||
}, 2500);
|
|
||||||
// Also try to clear the fail-safe as soon as DOMContentLoaded fires so
|
|
||||||
// we don't accidentally fire the lift AFTER a normal-config path.
|
|
||||||
document.addEventListener("DOMContentLoaded", function () {
|
|
||||||
var r = document.documentElement;
|
|
||||||
if (r.hasAttribute("data-theme") && !r.classList.contains("theme-ready")) {
|
|
||||||
r.classList.add("theme-ready");
|
|
||||||
if (window.__s2sVeilLiftTimeout) window.clearTimeout(window.__s2sVeilLiftTimeout);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
</script>
|
|
||||||
<link rel="stylesheet" href="style.css?v=202609291430" />
|
|
||||||
<script>
|
|
||||||
document.addEventListener("DOMContentLoaded", () => {
|
|
||||||
const l = document.querySelector('link[rel="stylesheet"]');
|
|
||||||
if (l && !l.href.includes("v=202609291430")) {
|
|
||||||
l.href = l.href.replace(/v=\d+/, "v=202609291430");
|
|
||||||
}
|
|
||||||
});
|
|
||||||
</script>
|
|
||||||
</head>
|
|
||||||
<!--
|
|
||||||
`booting` disables transitions/animations until the first paint commits.
|
|
||||||
Without this the orb visibly fades/scales in on the very first frame
|
|
||||||
because `state-idle` and the generic `.ind` defaults differ. The
|
|
||||||
bootstrap script at the end of main.js strips the class after one rAF.
|
|
||||||
-->
|
|
||||||
<body class="booting">
|
|
||||||
<div id="talk-unlocker" class="talk-unlocker" hidden data-debug-only-never>
|
|
||||||
<div class="talk-unlocker-hint">
|
|
||||||
<div class="talk-unlocker-spinner" aria-hidden="true"></div>
|
|
||||||
<p class="talk-unlocker-title">Preparando Vocero…</p>
|
|
||||||
<p class="talk-unlocker-sub">Toca en cualquier lugar para empezar a hablar</p>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div id="app">
|
|
||||||
<header class="topbar">
|
|
||||||
<div class="brand">
|
|
||||||
<div class="ident">
|
|
||||||
<div class="ident-head">
|
|
||||||
<a class="ident-title">Vocero</a>
|
|
||||||
<!-- Subtitle shown below Vocero title (theme=white only) -->
|
|
||||||
<p class="ident-subtitle" hidden data-theme-only="white">Agente conversacional</p>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<!-- User avatar circle (theme=white only): grid item next to the brand, not
|
|
||||||
absolute-positioned → eliminates overlap with title/subtitle entirely.
|
|
||||||
Same element id so main.js syncVoceroHeaderUser() still fills the text. -->
|
|
||||||
<button id="ident-user-name" class="ident-user-name" type="button" hidden disabled data-theme-only="white" aria-label="Ver datos del usuario" aria-haspopup="dialog" aria-controls="user-profile-dialog"></button>
|
|
||||||
<!-- Left sidebar (reports) toggle: always visible, pinned left side of the
|
|
||||||
topbar control cluster. Clicking slides in the offcanvas sidebar on
|
|
||||||
mobile; it's a no-op hover hint on desktop where sidebar is always
|
|
||||||
shown. -->
|
|
||||||
<button id="reports-sidebar-toggle" class="icon-btn" title="Reportes programados" aria-label="Abrir panel de reportes" aria-expanded="false" aria-controls="reports-sidebar" hidden data-reports-sidebar-toggle data-debug-only="true" data-theme-only="dark">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M3 3h18v18H3z" fill="none"/><line x1="9" y1="3" x2="9" y2="21"/></svg>
|
|
||||||
<span id="reports-sidebar-badge" class="chat-badge" aria-hidden="true"></span>
|
|
||||||
</button>
|
|
||||||
<div class="topbar-right" hidden data-debug-only="true">
|
|
||||||
<!-- HF login chip / sign-in pill. Populated by ui/account.js; only
|
|
||||||
shown when the deploy runs behind a load balancer with OAuth. -->
|
|
||||||
<div id="account" class="account" hidden></div>
|
|
||||||
<button id="about-btn-m" class="icon-btn about-btn-mobile" title="Acerca de este espacio" aria-label="Acerca de este espacio">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"/><line x1="12" y1="16" x2="12" y2="12"/><line x1="12" y1="8" x2="12.01" y2="8"/></svg>
|
|
||||||
</button>
|
|
||||||
<button id="tools-btn" class="icon-btn" title="Herramientas" aria-label="Herramientas">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/></svg>
|
|
||||||
</button>
|
|
||||||
<button id="rag-btn" class="icon-btn" title="Contexto RAG personalizado" aria-label="Contexto RAG personalizado">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M4 19.5A2.5 2.5 0 0 1 6.5 17H20"/><path d="M6.5 2H20v20H6.5A2.5 2.5 0 0 1 4 19.5v-15A2.5 2.5 0 0 1 6.5 2z"/></svg>
|
|
||||||
</button>
|
|
||||||
<button id="settings-btn" class="icon-btn" title="Configuración" aria-label="Configuración">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="3"/><path d="M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 1 1-2.83 2.83l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 1 1-4 0v-.09A1.65 1.65 0 0 0 9 19.4a1.65 1.65 0 0 0-1.82.33l-.06.06a2 2 0 1 1-2.83-2.83l.06-.06a1.65 1.65 0 0 0 .33-1.82 1.65 1.65 0 0 0-1.51-1H3a2 2 0 1 1 0-4h.09A1.65 1.65 0 0 0 4.6 9a1.65 1.65 0 0 0-.33-1.82l-.06-.06a2 2 0 1 1 2.83-2.83l.06.06a1.65 1.65 0 0 0 1.82.33H9a1.65 1.65 0 0 0 1-1.51V3a2 2 0 1 1 4 0v.09a1.65 1.65 0 0 0 1 1.51 1.65 1.65 0 0 0 1.82-.33l.06-.06a2 2 0 1 1 2.83 2.83l-.06.06a1.65 1.65 0 0 0-.33 1.82V9c0 .66.39 1.25 1 1.51H21a2 2 0 1 1 0 4h-.09a1.65 1.65 0 0 0-1.51 1z"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<a class="feedback-link" href="https://tally.so/r/pb8LWq" target="_blank" rel="noopener noreferrer">Enviar comentarios</a>
|
|
||||||
|
|
||||||
<!-- Theme=white only: Status card between topbar and stage
|
|
||||||
Ready to converse / Reports, analysis, data and more.
|
|
||||||
with light-blue chart icon and chevron-right arrow. -->
|
|
||||||
<div class="status-card" hidden data-theme-only="white" aria-label="Estado de la sesión">
|
|
||||||
<div class="status-card-icon" aria-hidden="true">
|
|
||||||
<svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.4" stroke-linecap="round" stroke-linejoin="round">
|
|
||||||
<line x1="6" y1="20" x2="6" y2="12"/>
|
|
||||||
<line x1="11" y1="20" x2="11" y2="6"/>
|
|
||||||
<line x1="16" y1="20" x2="16" y2="15"/>
|
|
||||||
<line x1="21" y1="20" x2="21" y2="10"/>
|
|
||||||
</svg>
|
|
||||||
</div>
|
|
||||||
<div class="status-card-text">
|
|
||||||
<h2 class="status-card-title">Listo para conversar</h2>
|
|
||||||
<p class="status-card-sub">Informes, análisis, datos y mucho más.</p>
|
|
||||||
</div>
|
|
||||||
<button class="status-card-arrow icon-btn" type="button" aria-label="Ver más detalles" title="Detalles">
|
|
||||||
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.4" stroke-linecap="round" stroke-linejoin="round">
|
|
||||||
<polyline points="9 18 15 12 9 6"></polyline>
|
|
||||||
</svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<main class="stage">
|
|
||||||
<!-- DARK THEME orb (purple gradient + purple icons): shown only on native dark theme, hidden in white -->
|
|
||||||
<div class="orb-wrap" data-theme-only="dark">
|
|
||||||
<div id="mic-gate" class="mic-gate">
|
|
||||||
<svg id="mic-gate-arc" class="mic-gate-arc" viewBox="0 0 100 100" aria-hidden="true">
|
|
||||||
<path id="mga-track" class="mga-track" fill="none" />
|
|
||||||
<path id="mga-fill" class="mga-fill" fill="none" />
|
|
||||||
<path id="mga-hit" class="mga-hit" fill="none" />
|
|
||||||
<circle id="mga-handle" class="mga-handle" r="3" />
|
|
||||||
</svg>
|
|
||||||
<button id="mic-btn" class="side-btn" type="button" aria-label="Silenciar micrófono" title="Silenciar micrófono" aria-hidden="true">
|
|
||||||
<svg class="mic-on" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="2" width="6" height="12" rx="3"/><path d="M5 10a7 7 0 0 0 14 0"/><line x1="12" y1="19" x2="12" y2="22"/></svg>
|
|
||||||
<svg class="mic-off" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><line x1="2" y1="2" x2="22" y2="22"/><path d="M9 5a3 3 0 0 1 6 0v4"/><path d="M9 10v1a3 3 0 0 0 5.1 2.1"/><path d="M19 10a7 7 0 0 1-1.24 3.97"/><path d="M5 10a7 7 0 0 0 11 5.67"/><line x1="12" y1="19" x2="12" y2="22"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<button
|
|
||||||
id="main-circle"
|
|
||||||
class="circle state-idle"
|
|
||||||
type="button"
|
|
||||||
aria-label="Iniciar conversación de voz"
|
|
||||||
>
|
|
||||||
<span class="circle-glow" aria-hidden="true"></span>
|
|
||||||
<span class="circle-ring" aria-hidden="true"></span>
|
|
||||||
<span class="circle-ring-outer" aria-hidden="true"></span>
|
|
||||||
<span class="circle-core">
|
|
||||||
<span class="circle-indicator" aria-hidden="true">
|
|
||||||
<svg class="ind ind-mic" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<rect x="9" y="2" width="6" height="12" rx="3" fill="currentColor" stroke="none"/>
|
|
||||||
<path d="M5 10a7 7 0 0 0 14 0"/>
|
|
||||||
<line x1="12" y1="19" x2="12" y2="22"/>
|
|
||||||
<line x1="8" y1="22" x2="16" y2="22"/>
|
|
||||||
</svg>
|
|
||||||
<svg class="ind ind-error" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="9"/><line x1="12" y1="8" x2="12" y2="13"/><line x1="12" y1="16" x2="12" y2="16"/></svg>
|
|
||||||
<span class="ind ind-spinner"></span>
|
|
||||||
<span class="ind ind-thinking">
|
|
||||||
<span class="dot"></span>
|
|
||||||
<span class="dot"></span>
|
|
||||||
<span class="dot"></span>
|
|
||||||
</span>
|
|
||||||
<span class="ind ind-bars">
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
</span>
|
|
||||||
<svg class="ind ind-voice" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<path d="M3 10v4a1 1 0 0 0 1 1h3l5 4V5L7 9H4a1 1 0 0 0-1 1z" fill="currentColor" stroke="none"/>
|
|
||||||
<path class="wave wave-1" d="M16 8a5 5 0 0 1 0 8"/>
|
|
||||||
<path class="wave wave-2" d="M19 5a9 9 0 0 1 0 14"/>
|
|
||||||
</svg>
|
|
||||||
</span>
|
|
||||||
</span>
|
|
||||||
</button>
|
|
||||||
|
|
||||||
<button id="stop-btn" class="side-btn" type="button" aria-label="Finalizar" title="Finalizar" aria-hidden="true">
|
|
||||||
<svg viewBox="0 0 24 24" fill="currentColor"><rect x="6" y="6" width="12" height="12" rx="2"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- WHITE THEME mic stage (replaces the purple orb)
|
|
||||||
- Blue mic circular button (116 px, deep blue, white mic)
|
|
||||||
- Two waveform banks left + right (audio-reactive CSS bars)
|
|
||||||
- Listening status text
|
|
||||||
- Mute and stop controls below.
|
|
||||||
Mirrors the screenshot center UI exactly. -->
|
|
||||||
<div class="white-mic-stage" hidden data-theme-only="white" aria-label="Control de voz">
|
|
||||||
<!-- Waveform left side: 7 bars mirror right side, heights driven by JS vars -->
|
|
||||||
<div class="white-waveform white-waveform-left" aria-hidden="true">
|
|
||||||
<span class="w-bar wb-1" style="--bar-h: 0.4"></span>
|
|
||||||
<span class="w-bar wb-2" style="--bar-h: 0.65"></span>
|
|
||||||
<span class="w-bar wb-3" style="--bar-h: 0.9"></span>
|
|
||||||
<span class="w-bar wb-4" style="--bar-h: 1"></span>
|
|
||||||
<span class="w-bar wb-5" style="--bar-h: 0.7"></span>
|
|
||||||
<span class="w-bar wb-6" style="--bar-h: 0.55"></span>
|
|
||||||
<span class="w-bar wb-7" style="--bar-h: 0.3"></span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Starts a voice session; becomes an indicator while connected. -->
|
|
||||||
<button id="white-mic-circle"
|
|
||||||
class="white-mic-circle state-idle"
|
|
||||||
type="button"
|
|
||||||
aria-label="Iniciar conversación de voz">
|
|
||||||
<span class="wmc-halo" aria-hidden="true"></span>
|
|
||||||
<span class="wmc-ring" aria-hidden="true"></span>
|
|
||||||
<span class="wmc-core" aria-hidden="true"></span>
|
|
||||||
<svg class="wmc-icon wmc-mic" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<rect x="9" y="2" width="6" height="12" rx="3" fill="currentColor" stroke="none"/>
|
|
||||||
<path d="M5 10a7 7 0 0 0 14 0"/>
|
|
||||||
<line x1="12" y1="19" x2="12" y2="22"/>
|
|
||||||
<line x1="8" y1="22" x2="16" y2="22"/>
|
|
||||||
</svg>
|
|
||||||
<span class="wmc-icon wmc-spinner" aria-hidden="true"></span>
|
|
||||||
</button>
|
|
||||||
|
|
||||||
<!-- Waveform right side: 7 bars mirroring left -->
|
|
||||||
<div class="white-waveform white-waveform-right" aria-hidden="true">
|
|
||||||
<span class="w-bar wb-1" style="--bar-h: 0.3"></span>
|
|
||||||
<span class="w-bar wb-2" style="--bar-h: 0.55"></span>
|
|
||||||
<span class="w-bar wb-3" style="--bar-h: 0.7"></span>
|
|
||||||
<span class="w-bar wb-4" style="--bar-h: 1"></span>
|
|
||||||
<span class="w-bar wb-5" style="--bar-h: 0.9"></span>
|
|
||||||
<span class="w-bar wb-6" style="--bar-h: 0.65"></span>
|
|
||||||
<span class="w-bar wb-7" style="--bar-h: 0.4"></span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Status caption below mic circle: listening, thinking, etc. -->
|
|
||||||
<p id="white-mic-caption" class="white-mic-caption" role="status">Toca para empezar</p>
|
|
||||||
<p id="white-mic-subcaption" class="white-mic-subcaption" hidden></p>
|
|
||||||
|
|
||||||
<div id="white-session-controls" class="white-session-controls" role="group" aria-label="Controles de la conversación" hidden>
|
|
||||||
<button id="white-mute-btn" class="white-mute-btn" type="button" hidden disabled aria-label="Silenciar micrófono" title="Silenciar micrófono">
|
|
||||||
<svg class="mic-on" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<rect x="9" y="2" width="6" height="12" rx="3"/><path d="M5 10a7 7 0 0 0 14 0"/><line x1="12" y1="19" x2="12" y2="22"/>
|
|
||||||
</svg>
|
|
||||||
<svg class="mic-off" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<line x1="2" y1="2" x2="22" y2="22"/><path d="M9 5a3 3 0 0 1 6 0v4"/><path d="M9 10v1a3 3 0 0 0 5.1 2.1"/><path d="M19 10a7 7 0 0 1-1.24 3.97"/><path d="M5 10a7 7 0 0 0 11 5.67"/><line x1="12" y1="19" x2="12" y2="22"/>
|
|
||||||
</svg>
|
|
||||||
<span id="white-mute-label">Silenciar micrófono</span>
|
|
||||||
</button>
|
|
||||||
<button id="white-stop-btn"
|
|
||||||
class="white-stop-btn"
|
|
||||||
type="button"
|
|
||||||
hidden
|
|
||||||
aria-label="Detener"
|
|
||||||
title="Detener sesión de voz">
|
|
||||||
<svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true"><rect x="6" y="6" width="12" height="12" rx="2"/></svg>
|
|
||||||
<span>Detener</span>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<p id="circle-caption" class="circle-caption" role="status">Toca para empezar</p>
|
|
||||||
<p id="circle-subcaption" class="circle-subcaption" hidden></p>
|
|
||||||
|
|
||||||
<div id="queue-actions" class="queue-actions" hidden>
|
|
||||||
<button id="join-queue-btn" class="join-queue-btn" type="button" hidden>
|
|
||||||
Entrar ahora
|
|
||||||
</button>
|
|
||||||
<button id="leave-queue-btn" class="leave-queue-btn" type="button" hidden>
|
|
||||||
Salir de la cola
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</main>
|
|
||||||
|
|
||||||
<!-- Theme=white only: summary, data lookup and draft-response quick
|
|
||||||
actions below the mic area, each with an icon. -->
|
|
||||||
<div class="quick-actions" hidden data-theme-only="white" aria-label="Acciones rápidas">
|
|
||||||
<button class="quick-action-btn" type="button" data-action="summary">
|
|
||||||
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/>
|
|
||||||
<polyline points="14 2 14 8 20 8"/>
|
|
||||||
<line x1="8" y1="13" x2="16" y2="13"/>
|
|
||||||
<line x1="8" y1="17" x2="14" y2="17"/>
|
|
||||||
</svg>
|
|
||||||
<span>Resumen</span>
|
|
||||||
</button>
|
|
||||||
<button class="quick-action-btn" type="button" data-action="query-data">
|
|
||||||
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<line x1="6" y1="20" x2="6" y2="12"/>
|
|
||||||
<line x1="11" y1="20" x2="11" y2="6"/>
|
|
||||||
<line x1="16" y1="20" x2="16" y2="15"/>
|
|
||||||
<line x1="21" y1="20" x2="21" y2="10"/>
|
|
||||||
</svg>
|
|
||||||
<span>Consultar datos</span>
|
|
||||||
</button>
|
|
||||||
<button class="quick-action-btn" type="button" data-action="draft-response">
|
|
||||||
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<line x1="18" y1="2" x2="22" y2="6"/>
|
|
||||||
<path d="M7.5 20.5 19 9l-4-4L3.5 16.5 2 22z"/>
|
|
||||||
</svg>
|
|
||||||
<span>Crear respuesta</span>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Text chat ALWAYS VISIBLE, fixed between the stage and footer.
|
|
||||||
The panel is now a direct child of the flex-column #app, so it
|
|
||||||
no longer needs a topbar toggle for a right-side overlay. -->
|
|
||||||
<div id="chat-panel" class="chat-panel chat-panel-inline" role="region" aria-label="Transcripción de la conversación">
|
|
||||||
<div class="chat-panel-inner">
|
|
||||||
<header class="chat-panel-header">
|
|
||||||
<h3>Conversación</h3>
|
|
||||||
<div class="chat-header-actions" role="group" aria-label="Acciones de la conversación">
|
|
||||||
<span id="chat-status" class="chat-status-pill" hidden data-theme-only="white" data-status="idle" role="status" aria-live="polite">
|
|
||||||
<span class="chat-status-dot" aria-hidden="true"></span>
|
|
||||||
<span id="chat-status-label">Sin sesión</span>
|
|
||||||
</span>
|
|
||||||
<button id="chat-export-btn" class="chat-export-btn icon-btn"
|
|
||||||
type="button"
|
|
||||||
aria-label="Descargar conversación"
|
|
||||||
title="Descargar conversación (JSON + TXT)">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.9" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/>
|
|
||||||
<polyline points="7 10 12 15 17 10"/>
|
|
||||||
<line x1="12" y1="15" x2="12" y2="3"/>
|
|
||||||
</svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</header>
|
|
||||||
<div id="chat-history" class="chat-history" role="log" aria-live="polite" aria-atomic="false"></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Ephemeral chat bubbles anchored to the top-right -->
|
|
||||||
<div id="bubble-stack" class="bubble-stack" aria-live="polite" aria-atomic="false"></div>
|
|
||||||
|
|
||||||
<!-- Webcam self-view, shown bottom-left while the camera tool is enabled.
|
|
||||||
HIDDEN BY DEFAULT TODAY: remove hidden and restore the main.js listener to reactivate. -->
|
|
||||||
<div id="cam-pip" class="cam-pip" aria-hidden="true" hidden>
|
|
||||||
<video id="cam-video" class="cam-video" autoplay playsinline muted></video>
|
|
||||||
<span class="cam-flash" aria-hidden="true"></span>
|
|
||||||
<span class="cam-label">cámara</span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Left sidebar: scheduled reports.
|
|
||||||
Desktop: fixed left, 320px, always visible (below topbar + above sticky footer).
|
|
||||||
Mobile : off-canvas left overlay, toggled by #reports-sidebar-toggle button in topbar. -->
|
|
||||||
<aside id="reports-sidebar" class="reports-sidebar" aria-label="Reportes programados" hidden data-theme-only="dark">
|
|
||||||
<div id="reports-sidebar-backdrop" class="reports-sidebar-backdrop" data-close-reports-sidebar></div>
|
|
||||||
<div class="reports-sidebar-inner">
|
|
||||||
<header class="reports-sidebar-header">
|
|
||||||
<h3>Reportes</h3>
|
|
||||||
<button id="reports-sidebar-close" class="icon-btn" aria-label="Cerrar panel de reportes" data-close-reports-sidebar>
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
<!-- MOVED from inside .ident in the topbar — same #id so main.js syncVoceroHeaderUser()
|
|
||||||
continues to work with zero JS changes to reports rendering logic. -->
|
|
||||||
<div id="reports-hint" class="reports-hint" hidden data-theme-only="dark"></div>
|
|
||||||
<!-- Debug-only: Spanish suggestions panel (gated behind ?debug=true).
|
|
||||||
Shown as the bottom block inside the left sidebar. -->
|
|
||||||
<div id="reports-suggestions" class="reports-suggestions" hidden data-debug-only="true"></div>
|
|
||||||
</div>
|
|
||||||
</aside>
|
|
||||||
|
|
||||||
<dialog id="about-modal" class="modal about-modal">
|
|
||||||
<div class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Acerca de</h2>
|
|
||||||
<button id="about-close" class="icon-btn" aria-label="Cerrar">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<!-- General intro to the speech-to-speech project -->
|
|
||||||
<div class="about-intro">
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- The backend, shown as the path a turn actually travels -->
|
|
||||||
<div class="about-pipeline">
|
|
||||||
<p class="pipeline-title">Cómo funciona</p>
|
|
||||||
<ol class="pipeline">
|
|
||||||
<li class="pipe-endpoint">Tú hablas</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">VAD</span>
|
|
||||||
<span class="pipe-job">detecta el habla</span>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">STT</span>
|
|
||||||
<span class="pipe-job">la transcribe</span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/nvidia/parakeet-tdt-1.1b" target="_blank" rel="noopener">nvidia/parakeet-tdt-1.1b<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">VLM</span>
|
|
||||||
<span class="pipe-job">analiza y redacta la respuesta <span class="pipe-note">· mediante <a class="cerebras-credit" href="https://cerebras.ai" target="_blank" rel="noopener"><svg class="cerebras-mark" viewBox="0 0 50 50" aria-hidden="true"><path fill-rule="evenodd" clip-rule="evenodd" d="M29.4186 5.62713C24.2806 5.62713 19.353 7.6682 15.7199 11.3013C12.0868 14.9345 10.0457 19.862 10.0457 25C10.0457 30.1381 12.0868 35.0656 15.7199 38.6988C19.353 42.3319 24.2806 44.373 29.4186 44.373V47.2917C17.1061 47.2917 7.12695 37.3105 7.12695 24.998C7.12695 12.6855 17.104 2.7063 29.4165 2.7063V5.62505L29.4186 5.62713ZM39.3186 13.2875C37.7794 11.9809 35.9971 10.9915 34.0741 10.3761C32.1512 9.76067 30.1256 9.53147 28.1137 9.70166C26.1019 9.87185 24.1435 10.4381 22.3513 11.3677C20.559 12.2974 18.9683 13.5722 17.6704 15.1189C16.3726 16.6655 15.3933 18.4534 14.7888 20.3798C14.1844 22.3062 13.9667 24.3332 14.1483 26.344C14.33 28.3548 14.9073 30.3099 15.8472 32.0968C16.787 33.8838 18.0709 35.4673 19.6249 36.7563L17.7478 38.9938C15.9126 37.4545 14.3987 35.5687 13.2924 33.4442C12.1862 31.3197 11.5092 28.9981 11.3003 26.6119C11.0914 24.2258 11.3545 21.8219 12.0747 19.5374C12.7949 17.253 13.9581 15.1328 15.4978 13.298C17.0373 11.4628 18.9233 9.94877 21.0479 8.84247C23.1726 7.73618 25.4944 7.05923 27.8807 6.8503C30.267 6.64136 32.6711 6.90453 34.9557 7.62477C37.2403 8.34501 39.3607 9.50822 41.1957 11.048L39.3186 13.2875ZM34.6207 15.0459C31.9818 13.6829 28.9113 13.4172 26.0776 14.3068C23.2438 15.1964 20.876 17.1691 19.4895 19.7958C18.1029 22.4224 17.8099 25.4903 18.6741 28.332C19.5384 31.1736 21.4899 33.5589 24.104 34.9688L22.7374 37.5521C19.4721 35.7617 17.0418 32.7592 15.9707 29.1927C14.8996 25.6261 15.2737 21.7815 17.0123 18.4883C18.7509 15.1952 21.7146 12.7176 25.2637 11.5903C28.8129 10.463 32.663 10.7763 35.9832 12.4625L34.6207 15.0459ZM29.4165 17.7896C27.5042 17.7896 25.6702 18.5493 24.318 19.9015C22.9658 21.2537 22.2061 23.0877 22.2061 25C22.2061 26.9124 22.9658 28.7464 24.318 30.0986C25.6702 31.4508 27.5042 32.2105 29.4165 32.2105V35.1313C26.7296 35.1313 24.1526 34.0639 22.2527 32.1639C20.3527 30.2639 19.2853 27.687 19.2853 25C19.2853 22.3131 20.3527 19.7362 22.2527 17.8362C24.1526 15.9362 26.7296 14.8688 29.4165 14.8688V17.7896Z" fill="#F15A29"/><path d="M32.0981 22.5749C31.7875 22.2404 31.4295 21.9534 31.0356 21.7228C30.6925 21.519 30.3014 21.4097 29.9023 21.4062C29.371 21.4062 28.896 21.5041 28.4773 21.6978C28.0725 21.8802 27.7089 22.1425 27.4081 22.469C27.1074 22.7955 26.8758 23.1795 26.7273 23.5978C26.5731 24.0207 26.4981 24.4645 26.4981 24.9124C26.4981 25.3666 26.5731 25.8082 26.7273 26.227C26.877 26.6449 27.1091 27.0286 27.4096 27.3553C27.7102 27.682 28.0733 27.9451 28.4773 28.1291C28.8939 28.3228 29.3731 28.4207 29.9023 28.4207C30.3523 28.4207 30.771 28.3249 31.1564 28.1395C31.5481 27.952 31.8856 27.6707 32.146 27.3228L34.0794 29.4187C33.7877 29.7103 33.4544 29.9624 33.0752 30.1749C32.374 30.566 31.6108 30.8338 30.8189 30.9666C30.4648 31.0207 30.1585 31.0499 29.9023 31.0499C29.0617 31.0552 28.2271 30.9069 27.4398 30.6124C26.6954 30.3371 26.0144 29.914 25.4377 29.3687C24.8643 28.8208 24.4078 28.1624 24.096 27.4332C23.7572 26.6366 23.5883 25.778 23.6002 24.9124C23.6002 23.9874 23.7669 23.1478 24.096 22.3916C24.4085 21.6624 24.8627 21.0041 25.4356 20.4562C26.0148 19.9124 26.696 19.4895 27.4398 19.2145C28.2271 18.92 29.0617 18.7717 29.9023 18.777C30.6419 18.777 31.3856 18.9187 32.1356 19.202C32.8877 19.4895 33.5627 19.952 34.1023 20.5541L32.0981 22.5749Z" fill="#fff"/></svg>Cerebras</a></span></span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/google/gemma-4-31B-it" target="_blank" rel="noopener">google/gemma-4-31B-it<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">TTS</span>
|
|
||||||
<span class="pipe-job">la convierte en voz</span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice" target="_blank" rel="noopener">Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-endpoint">Vocero responde</li>
|
|
||||||
</ol>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<dialog id="settings-modal" class="modal">
|
|
||||||
<form method="dialog" class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Configuración</h2>
|
|
||||||
<button class="icon-btn" value="close" aria-label="Cerrar">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<div class="tab-panels">
|
|
||||||
<div class="tab-panel active" role="tabpanel">
|
|
||||||
<label class="field" id="conn-field">
|
|
||||||
<span id="conn-label">URL del servidor de voz</span>
|
|
||||||
<input id="lb-url" type="text" autocomplete="off" spellcheck="false" placeholder="http://localhost:8080" />
|
|
||||||
<small id="conn-hint">
|
|
||||||
URL de tu servidor de voz, por ejemplo
|
|
||||||
<code>http://localhost:8080</code> (la aplicación añade <code>/v1/realtime</code>).
|
|
||||||
</small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<div class="field-row">
|
|
||||||
<label class="field">
|
|
||||||
<span>Voz</span>
|
|
||||||
<select id="voice">
|
|
||||||
<option value="Aiden" selected>Aiden</option>
|
|
||||||
<option value="Ryan">Ryan</option>
|
|
||||||
<option value="Dylan">Dylan</option>
|
|
||||||
<option value="Eric">Eric</option>
|
|
||||||
<option value="Ono_Anna">Ono_Anna</option>
|
|
||||||
<option value="Serena">Serena</option>
|
|
||||||
<option value="Sohee">Sohee</option>
|
|
||||||
<option value="Uncle_Fu">Uncle_Fu</option>
|
|
||||||
<option value="Vivian">Vivian</option>
|
|
||||||
</select>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="field">
|
|
||||||
<span class="field-head">
|
|
||||||
Puerta de ruido
|
|
||||||
<span id="gate-value" class="field-value">Desactivada</span>
|
|
||||||
</span>
|
|
||||||
<div class="gate">
|
|
||||||
<div class="gate-track" aria-hidden="true">
|
|
||||||
<div id="gate-meter-fill" class="gate-meter-fill"></div>
|
|
||||||
<input
|
|
||||||
id="noise-gate"
|
|
||||||
type="range"
|
|
||||||
min="-66"
|
|
||||||
max="-3"
|
|
||||||
step="1"
|
|
||||||
value="-50"
|
|
||||||
aria-label="Umbral de la puerta de ruido"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
<div class="gate-ends">
|
|
||||||
<span>Desactivada</span>
|
|
||||||
<span>−3 dB</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<small>
|
|
||||||
Silencia el micrófono por debajo del umbral para evitar enviar ruido.
|
|
||||||
Desliza hasta la izquierda para desactivarla. La barra muestra tu audio durante la llamada.
|
|
||||||
</small>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<label class="field">
|
|
||||||
<span>Instrucciones</span>
|
|
||||||
<textarea id="instructions" rows="5" placeholder="Introduce las instrucciones de Vocero para esta conversación…"></textarea>
|
|
||||||
<small id="primary-prompt-meta" hidden></small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<div class="field">
|
|
||||||
<button id="restart-conversation" type="button" class="btn primary wide" disabled>
|
|
||||||
Reiniciar conversación con esta configuración
|
|
||||||
</button>
|
|
||||||
<small id="restart-hint">Conéctate primero y vuelve aquí para aplicar cambios durante la conversación.</small>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<footer class="modal-footer">
|
|
||||||
<button id="settings-save" type="submit" class="btn primary" value="save">Guardar</button>
|
|
||||||
</footer>
|
|
||||||
</form>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<dialog id="tools-modal" class="modal">
|
|
||||||
<div class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Herramientas</h2>
|
|
||||||
<button id="tools-close" class="icon-btn" aria-label="Cerrar">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<p class="tools-intro">Permite que el asistente actúe durante la conversación. Los cambios se aplican de inmediato.</p>
|
|
||||||
|
|
||||||
<div class="tool-list">
|
|
||||||
<div class="tool-row" id="tool-web-row">
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Búsqueda web</span>
|
|
||||||
<span class="tool-desc">Busca información en Google mediante Serper.</span>
|
|
||||||
</div>
|
|
||||||
<label class="switch">
|
|
||||||
<input id="tool-web" type="checkbox" />
|
|
||||||
<span class="switch-track" aria-hidden="true"></span>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
<label class="field tools-key">
|
|
||||||
<span>Clave de la API de búsqueda</span>
|
|
||||||
<input id="search-key" type="password" autocomplete="off" spellcheck="false" />
|
|
||||||
<small id="tool-web-hint"></small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<!-- Camera tool row — HIDDEN BY DEFAULT TODAY.
|
|
||||||
To reactivate later, remove hidden from the div and small,
|
|
||||||
then restore the toolCamSwitch listener and enableCamera functions in main.js. -->
|
|
||||||
<div class="tool-row tool-row-sep" id="tool-cam-row" hidden>
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Cámara</span>
|
|
||||||
<span class="tool-desc">Permite que el asistente vea a través de tu cámara.</span>
|
|
||||||
</div>
|
|
||||||
<label class="switch">
|
|
||||||
<input id="tool-cam" type="checkbox" />
|
|
||||||
<span class="switch-track" aria-hidden="true"></span>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
<small id="tool-cam-hint" class="tool-hint" hidden>La vista previa aparece abajo a la izquierda mientras la cámara está activa.</small>
|
|
||||||
|
|
||||||
<div class="tool-row tool-row-sep">
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Herramientas MCP cargadas</span>
|
|
||||||
<span class="tool-desc">Herramientas publicadas por el perfil remoto del agente.</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<small id="agent-tools-hint" class="tool-hint"></small>
|
|
||||||
<div id="agent-tools-list" class="agent-tools-list"></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<dialog id="rag-modal" class="modal">
|
|
||||||
<div class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Contexto RAG personalizado</h2>
|
|
||||||
<button id="rag-close" class="icon-btn" aria-label="Cerrar">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<p class="tools-intro">
|
|
||||||
Pega aquí texto, documentos o fragmentos RAG que quieras tener disponibles
|
|
||||||
durante la sesión de voz actual. El texto se añade a las
|
|
||||||
instrucciones del modelo y al campo <code>user_context</code> de la solicitud a
|
|
||||||
<code>/api/reasoning</code>.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<label class="field">
|
|
||||||
<span>Contexto RAG (texto libre)</span>
|
|
||||||
<textarea
|
|
||||||
id="rag-textarea"
|
|
||||||
rows="12"
|
|
||||||
spellcheck="false"
|
|
||||||
placeholder="Pega o escribe el contexto RAG que el modelo debe usar al responder. Por ejemplo: datos del cliente, políticas o fragmentos de documentos."
|
|
||||||
></textarea>
|
|
||||||
<small id="rag-hint" class="tool-hint">
|
|
||||||
No hay contexto RAG cargado. Pulsa "Enviar a la sesión" para aplicar los cambios.
|
|
||||||
</small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<footer class="modal-footer">
|
|
||||||
<button id="rag-clear" class="btn secondary" type="button" value="clear">Vaciar</button>
|
|
||||||
<div style="flex: 1"></div>
|
|
||||||
<button id="rag-submit" class="btn primary" type="button" value="submit">Enviar a la sesión</button>
|
|
||||||
</footer>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<!-- Shown when the per-day conversation budget is spent (mid-call or at
|
|
||||||
start). Title / message / CTA / note are filled per tier by
|
|
||||||
ui/account.js. The hero is the HF mark — a friendly smiling face. -->
|
|
||||||
<dialog id="limit-modal" class="modal limit-modal">
|
|
||||||
<div class="modal-content limit-card">
|
|
||||||
<button id="limit-close" class="icon-btn limit-close" aria-label="Cerrar">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
<div class="limit-badge" aria-hidden="true">
|
|
||||||
<svg class="hf-logo" viewBox="0 0 95 88" fill="none" aria-hidden="true"><path fill="#FFD21E" d="M47.21 76.5a34.75 34.75 0 1 0 0-69.5 34.75 34.75 0 0 0 0 69.5Z"/><path fill="#FF9D0B" d="M81.96 41.75a34.75 34.75 0 1 0-69.5 0 34.75 34.75 0 0 0 69.5 0Zm-73.5 0a38.75 38.75 0 1 1 77.5 0 38.75 38.75 0 0 1-77.5 0Z"/><path fill="#3A3B45" d="M58.5 32.3c1.28.44 1.78 3.06 3.07 2.38a5 5 0 1 0-6.76-2.07c.61 1.15 2.55-.72 3.7-.32ZM34.95 32.3c-1.28.44-1.79 3.06-3.07 2.38a5 5 0 1 1 6.76-2.07c-.61 1.15-2.56-.72-3.7-.32Z"/><path fill="#FF323D" d="M46.96 56.29c9.83 0 13-8.76 13-13.26 0-2.34-1.57-1.6-4.09-.36-2.33 1.15-5.46 2.74-8.9 2.74-7.19 0-13-6.88-13-2.38s3.16 13.26 13 13.26Z"/><path fill="#3A3B45" fill-rule="evenodd" d="M39.43 54a8.7 8.7 0 0 1 5.3-4.49c.4-.12.81.57 1.24 1.28.4.68.82 1.37 1.24 1.37.45 0 .9-.68 1.33-1.35.45-.7.89-1.38 1.32-1.25a8.61 8.61 0 0 1 5 4.17c3.73-2.94 5.1-7.74 5.1-10.7 0-2.34-1.57-1.6-4.09-.36l-.14.07c-2.31 1.15-5.39 2.67-8.77 2.67s-6.45-1.52-8.77-2.67c-2.6-1.29-4.23-2.1-4.23.29 0 3.05 1.46 8.06 5.47 10.97Z" clip-rule="evenodd"/><path fill="#FF9D0B" d="M70.71 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM24.21 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM17.52 48c-1.62 0-3.06.66-4.07 1.87a5.97 5.97 0 0 0-1.33 3.76 7.1 7.1 0 0 0-1.94-.3c-1.55 0-2.95.59-3.94 1.66a5.8 5.8 0 0 0-.8 7 5.3 5.3 0 0 0-1.79 2.82c-.24.9-.48 2.8.8 4.74a5.22 5.22 0 0 0-.37 5.02c1.02 2.32 3.57 4.14 8.52 6.1 3.07 1.22 5.89 2 5.91 2.01a44.33 44.33 0 0 0 10.93 1.6c5.86 0 10.05-1.8 12.46-5.34 3.88-5.69 3.33-10.9-1.7-15.92-2.77-2.78-4.62-6.87-5-7.77-.78-2.66-2.84-5.62-6.25-5.62a5.7 5.7 0 0 0-4.6 2.46c-1-1.26-1.98-2.25-2.86-2.82A7.4 7.4 0 0 0 17.52 48Zm0 4c.51 0 1.14.22 1.82.65 2.14 1.36 6.25 8.43 7.76 11.18.5.92 1.37 1.31 2.14 1.31 1.55 0 2.75-1.53.15-3.48-3.92-2.93-2.55-7.72-.68-8.01.08-.02.17-.02.24-.02 1.7 0 2.45 2.93 2.45 2.93s2.2 5.52 5.98 9.3c3.77 3.77 3.97 6.8 1.22 10.83-1.88 2.75-5.47 3.58-9.16 3.58-3.81 0-7.73-.9-9.92-1.46-.11-.03-13.45-3.8-11.76-7 .28-.54.75-.76 1.34-.76 2.38 0 6.7 3.54 8.57 3.54.41 0 .7-.17.83-.6.79-2.85-12.06-4.05-10.98-8.17.2-.73.71-1.02 1.44-1.02 3.14 0 10.2 5.53 11.68 5.53.11 0 .2-.03.24-.1.74-1.2.33-2.04-4.9-5.2-5.21-3.16-8.88-5.06-6.8-7.33.24-.26.58-.38 1-.38 3.17 0 10.66 6.82 10.66 6.82s2.02 2.1 3.25 2.1c.28 0 .52-.1.68-.38.86-1.46-8.06-8.22-8.56-11.01-.34-1.9.24-2.85 1.31-2.85Z"/><path fill="#FFD21E" d="M38.6 76.69c2.75-4.04 2.55-7.07-1.22-10.84-3.78-3.77-5.98-9.3-5.98-9.3s-.82-3.2-2.69-2.9c-1.87.3-3.24 5.08.68 8.01 3.91 2.93-.78 4.92-2.29 2.17-1.5-2.75-5.62-9.82-7.76-11.18-2.13-1.35-3.63-.6-3.13 2.2.5 2.79 9.43 9.55 8.56 11-.87 1.47-3.93-1.71-3.93-1.71s-9.57-8.71-11.66-6.44c-2.08 2.27 1.59 4.17 6.8 7.33 5.23 3.16 5.64 4 4.9 5.2-.75 1.2-12.28-8.53-13.36-4.4-1.08 4.11 11.77 5.3 10.98 8.15-.8 2.85-9.06-5.38-10.74-2.18-1.7 3.21 11.65 6.98 11.76 7.01 4.3 1.12 15.25 3.49 19.08-2.12Z"/><path fill="#FF9D0B" d="M77.4 48c1.62 0 3.07.66 4.07 1.87a5.97 5.97 0 0 1 1.33 3.76 7.1 7.1 0 0 1 1.95-.3c1.55 0 2.95.59 3.94 1.66a5.8 5.8 0 0 1 .8 7 5.3 5.3 0 0 1 1.78 2.82c.24.9.48 2.8-.8 4.74a5.22 5.22 0 0 1 .37 5.02c-1.02 2.32-3.57 4.14-8.51 6.1-3.08 1.22-5.9 2-5.92 2.01a44.33 44.33 0 0 1-10.93 1.6c-5.86 0-10.05-1.8-12.46-5.34-3.88-5.69-3.33-10.9 1.7-15.92 2.78-2.78 4.63-6.87 5.01-7.77.78-2.66 2.83-5.62 6.24-5.62a5.7 5.7 0 0 1 4.6 2.46c1-1.26 1.98-2.25 2.87-2.82A7.4 7.4 0 0 1 77.4 48Zm0 4c-.51 0-1.13.22-1.82.65-2.13 1.36-6.25 8.43-7.76 11.18a2.43 2.43 0 0 1-2.14 1.31c-1.54 0-2.75-1.53-.14-3.48 3.91-2.93 2.54-7.72.67-8.01a1.54 1.54 0 0 0-.24-.02c-1.7 0-2.45 2.93-2.45 2.93s-2.2 5.52-5.97 9.3c-3.78 3.77-3.98 6.8-1.22 10.83 1.87 2.75 5.47 3.58 9.15 3.58 3.82 0 7.73-.9 9.93-1.46.1-.03 13.45-3.8 11.76-7-.29-.54-.75-.76-1.34-.76-2.38 0-6.71 3.54-8.57 3.54-.42 0-.71-.17-.83-.6-.8-2.85 12.05-4.05 10.97-8.17-.19-.73-.7-1.02-1.44-1.02-3.14 0-10.2 5.53-11.68 5.53-.1 0-.19-.03-.23-.1-.74-1.2-.34-2.04 4.88-5.2 5.23-3.16 8.9-5.06 6.8-7.33-.23-.26-.57-.38-.98-.38-3.18 0-10.67 6.82-10.67 6.82s-2.02 2.1-3.24 2.1a.74.74 0 0 1-.68-.38c-.87-1.46 8.05-8.22 8.55-11.01.34-1.9-.24-2.85-1.31-2.85Z"/><path fill="#FFD21E" d="M56.33 76.69c-2.75-4.04-2.56-7.07 1.22-10.84 3.77-3.77 5.97-9.3 5.97-9.3s.82-3.2 2.7-2.9c1.86.3 3.23 5.08-.68 8.01-3.92 2.93.78 4.92 2.28 2.17 1.51-2.75 5.63-9.82 7.76-11.18 2.13-1.35 3.64-.6 3.13 2.2-.5 2.79-9.42 9.55-8.55 11 .86 1.47 3.92-1.71 3.92-1.71s9.58-8.71 11.66-6.44c2.08 2.27-1.58 4.17-6.8 7.33-5.23 3.16-5.63 4-4.9 5.2.75 1.2 12.28-8.53 13.36-4.4 1.08 4.11-11.76 5.3-10.97 8.15.8 2.85 9.05-5.38 10.74-2.18 1.69 3.21-11.65 6.98-11.76 7.01-4.31 1.12-15.26 3.49-19.08-2.12Z"/></svg>
|
|
||||||
</div>
|
|
||||||
<h2 id="limit-title" class="limit-title">Por ahora hemos terminado</h2>
|
|
||||||
<p id="limit-msg" class="limit-msg"></p>
|
|
||||||
<a id="limit-cta" class="btn primary wide limit-cta" href="#"></a>
|
|
||||||
<p id="limit-note" class="limit-note"></p>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<!-- User profile dialog — opened by clicking the circular avatar badge in
|
|
||||||
the brand header. Styling: large glass hero with initials + rows of
|
|
||||||
user metadata populated via main.js populateUserProfileDialog(). -->
|
|
||||||
<dialog id="user-profile-dialog" class="modal user-profile-modal" aria-labelledby="user-profile-title">
|
|
||||||
<div class="modal-content user-profile-card">
|
|
||||||
<button id="user-profile-close" class="icon-btn user-profile-close" aria-label="Cerrar perfil de usuario">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
|
|
||||||
<!-- Hero: big glass avatar circle + title + subtitle (email fallback). -->
|
|
||||||
<div class="user-profile-hero">
|
|
||||||
<div id="user-profile-avatar" class="user-profile-avatar" aria-hidden="true">AV</div>
|
|
||||||
<h2 id="user-profile-title" class="user-profile-title"></h2>
|
|
||||||
<p id="user-profile-subtitle" class="user-profile-subtitle"></p>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Metadata: a 1-col definition-list-like grid. Empty rows are hidden
|
|
||||||
automatically via JS when the field has no value, so we never show
|
|
||||||
blank labels. -->
|
|
||||||
<div id="user-profile-metadata" class="user-profile-metadata" aria-label="Información del usuario"></div>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<footer class="app-footer" aria-label="Versión de la aplicación">
|
|
||||||
<span class="app-footer-text">Azurian Vocero en tiempo real ver. Beta 2.1</span>
|
|
||||||
</footer>
|
|
||||||
|
|
||||||
<script type="module" src="main.js?v=202609102207"></script>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
@@ -1,276 +0,0 @@
|
|||||||
"""
|
|
||||||
Per-day talk-time budget for the speech-to-speech demo.
|
|
||||||
|
|
||||||
Our server isn't in the audio path (the browser dials the compute WebSocket
|
|
||||||
directly), so it can't cut a live stream. What it *can* do is meter time with a
|
|
||||||
server-clock, chunked reservation:
|
|
||||||
|
|
||||||
- At grant we reserve the first chunk (CHUNK_SEC) and debit it from the day's
|
|
||||||
budget. A parallel grant therefore sees the budget already spent.
|
|
||||||
- The client heartbeats; each heartbeat extends the reservation one chunk at a
|
|
||||||
time until the daily budget runs out, then we report `expired` so the client
|
|
||||||
tears down.
|
|
||||||
- On a clean end (sendBeacon) we reconcile to the real elapsed time and refund
|
|
||||||
the unused chunk. A crash (no end, no heartbeats) is reaped by a sweep and
|
|
||||||
forfeits at most one chunk.
|
|
||||||
|
|
||||||
All time is the server's clock. Budgets are per UTC day; a new day is simply a
|
|
||||||
new row (no explicit reset). Logged-in users are keyed by a hashed HF `sub`;
|
|
||||||
anonymous users by BOTH a hashed IP and a hashed signed-cookie id, OR-matched
|
|
||||||
(spent = max of the two) so clearing one identifier doesn't reset the budget.
|
|
||||||
|
|
||||||
Storage is SQLite at $USAGE_DB_PATH, else /data (persistent Spaces storage),
|
|
||||||
else a /tmp fallback. On /tmp the budget is only per-uptime — flagged in logs.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import hashlib
|
|
||||||
import hmac
|
|
||||||
import logging
|
|
||||||
import math
|
|
||||||
import os
|
|
||||||
import sqlite3
|
|
||||||
import tempfile
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
logger = logging.getLogger("s2s.limiter")
|
|
||||||
|
|
||||||
# ── Tunables (env-overridable) ───────────────────────────────────────────────
|
|
||||||
ANON_SEC = int(os.environ.get("LIMIT_ANON_SEC", "300")) # 5 min/day, not signed in
|
|
||||||
FREE_SEC = int(os.environ.get("LIMIT_FREE_SEC", "600")) # 10 min/day, signed in, no PRO
|
|
||||||
CHUNK_SEC = int(os.environ.get("RESERVE_CHUNK_SEC", "10")) # reservation granularity
|
|
||||||
HEARTBEAT_SEC = int(os.environ.get("HEARTBEAT_SEC", "5")) # advertised client cadence
|
|
||||||
REAP_AFTER_SEC = int(os.environ.get("SESSION_REAP_SEC", "15")) # silence before sweep
|
|
||||||
|
|
||||||
# Stable across restarts or the hashed keys (and signed cookies) rotate and the
|
|
||||||
# budget effectively resets. Set it as a Space secret. Falls back to a per-boot
|
|
||||||
# random value (keys then only hold within one uptime).
|
|
||||||
_HASH_SECRET = (os.environ.get("USAGE_HASH_SECRET", "").strip() or os.urandom(32).hex()).encode()
|
|
||||||
|
|
||||||
_lock = threading.Lock()
|
|
||||||
_db_path: "Path | None" = None
|
|
||||||
|
|
||||||
|
|
||||||
def budget_for(tier: str) -> "int | None":
|
|
||||||
"""Daily second-budget for a tier, or None for unlimited.
|
|
||||||
|
|
||||||
Unlimited tiers: 'pro' (paying PRO members) and 'org' (members of an
|
|
||||||
allow-listed organisation, see UNLIMITED_ORGS in auth.py)."""
|
|
||||||
if tier in ("pro", "org"):
|
|
||||||
return None
|
|
||||||
if tier == "free":
|
|
||||||
return FREE_SEC
|
|
||||||
return ANON_SEC
|
|
||||||
|
|
||||||
|
|
||||||
def hash_key(raw: str) -> str:
|
|
||||||
"""HMAC a raw identifier (sub / ip / cookie id) into an opaque storage key."""
|
|
||||||
digest = hmac.new(_HASH_SECRET, raw.encode("utf-8"), hashlib.sha256).hexdigest()
|
|
||||||
return f"k_{digest}"
|
|
||||||
|
|
||||||
|
|
||||||
def sign_cookie(value: str) -> str:
|
|
||||||
"""`<id>.<sig>` so a forged anon-cookie id is rejected on read."""
|
|
||||||
sig = hmac.new(_HASH_SECRET, value.encode("utf-8"), hashlib.sha256).hexdigest()[:32]
|
|
||||||
return f"{value}.{sig}"
|
|
||||||
|
|
||||||
|
|
||||||
def verify_cookie(signed: str) -> "str | None":
|
|
||||||
"""Return the id if the signature checks out, else None."""
|
|
||||||
if not signed or "." not in signed:
|
|
||||||
return None
|
|
||||||
value, _, sig = signed.rpartition(".")
|
|
||||||
want = hmac.new(_HASH_SECRET, value.encode("utf-8"), hashlib.sha256).hexdigest()[:32]
|
|
||||||
return value if hmac.compare_digest(sig, want) else None
|
|
||||||
|
|
||||||
|
|
||||||
def _today() -> str:
|
|
||||||
return datetime.now(timezone.utc).date().isoformat()
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_db_path() -> Path:
|
|
||||||
explicit = os.environ.get("USAGE_DB_PATH", "").strip()
|
|
||||||
if explicit:
|
|
||||||
return Path(explicit)
|
|
||||||
data = Path("/data")
|
|
||||||
if data.is_dir() and os.access(data, os.W_OK):
|
|
||||||
return data / "s2s-usage.sqlite3"
|
|
||||||
logger.warning("No persistent /data — usage budget falls back to /tmp (per-uptime only).")
|
|
||||||
return Path(tempfile.gettempdir()) / "s2s-usage.sqlite3"
|
|
||||||
|
|
||||||
|
|
||||||
def _connect() -> sqlite3.Connection:
|
|
||||||
con = sqlite3.connect(_db_path, timeout=5.0)
|
|
||||||
con.execute("PRAGMA journal_mode=WAL")
|
|
||||||
con.execute("PRAGMA busy_timeout=5000")
|
|
||||||
return con
|
|
||||||
|
|
||||||
|
|
||||||
def init() -> None:
|
|
||||||
"""Create the schema. Call once at startup."""
|
|
||||||
global _db_path
|
|
||||||
_db_path = _resolve_db_path()
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
con.execute(
|
|
||||||
"""CREATE TABLE IF NOT EXISTS usage_daily (
|
|
||||||
user_key TEXT NOT NULL,
|
|
||||||
day TEXT NOT NULL,
|
|
||||||
spent_sec INTEGER NOT NULL DEFAULT 0,
|
|
||||||
updated_at INTEGER NOT NULL,
|
|
||||||
PRIMARY KEY (user_key, day)
|
|
||||||
)"""
|
|
||||||
)
|
|
||||||
con.execute(
|
|
||||||
"""CREATE TABLE IF NOT EXISTS sessions (
|
|
||||||
session_id TEXT PRIMARY KEY,
|
|
||||||
keys TEXT NOT NULL, -- comma-joined usage_daily keys to debit
|
|
||||||
day TEXT NOT NULL,
|
|
||||||
tier TEXT NOT NULL,
|
|
||||||
grant_ts REAL NOT NULL,
|
|
||||||
last_seen_ts REAL NOT NULL,
|
|
||||||
reserved_sec INTEGER NOT NULL,
|
|
||||||
ended INTEGER NOT NULL DEFAULT 0
|
|
||||||
)"""
|
|
||||||
)
|
|
||||||
logger.info("Usage limiter ready at %s (anon=%ss free=%ss chunk=%ss)", _db_path, ANON_SEC, FREE_SEC, CHUNK_SEC)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Internal helpers (call under _lock) ───────────────────────────────────────
|
|
||||||
|
|
||||||
def _spent(con, key: str, day: str) -> int:
|
|
||||||
row = con.execute(
|
|
||||||
"SELECT spent_sec FROM usage_daily WHERE user_key=? AND day=?", (key, day)
|
|
||||||
).fetchone()
|
|
||||||
return int(row[0]) if row else 0
|
|
||||||
|
|
||||||
|
|
||||||
def _spent_max(con, keys, day: str) -> int:
|
|
||||||
"""OR-match: the most-spent identifier governs."""
|
|
||||||
return max((_spent(con, k, day) for k in keys), default=0)
|
|
||||||
|
|
||||||
|
|
||||||
def _add(con, keys, day: str, delta: int) -> None:
|
|
||||||
now = int(time.time())
|
|
||||||
for k in keys:
|
|
||||||
cur = _spent(con, k, day)
|
|
||||||
nxt = max(0, cur + delta)
|
|
||||||
con.execute(
|
|
||||||
"""INSERT INTO usage_daily (user_key, day, spent_sec, updated_at)
|
|
||||||
VALUES (?, ?, ?, ?)
|
|
||||||
ON CONFLICT(user_key, day) DO UPDATE SET
|
|
||||||
spent_sec = excluded.spent_sec, updated_at = excluded.updated_at""",
|
|
||||||
(k, day, nxt, now),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Public API ────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def remaining(keys, tier: str) -> "int | None":
|
|
||||||
"""Seconds left today for these keys (None = unlimited). No mutation."""
|
|
||||||
budget = budget_for(tier)
|
|
||||||
if budget is None:
|
|
||||||
return None
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
return max(0, budget - _spent_max(con, keys, _today()))
|
|
||||||
|
|
||||||
|
|
||||||
def begin(session_id: str, keys, tier: str) -> int:
|
|
||||||
"""Reserve the first chunk for a new session and record it. Returns the
|
|
||||||
chunk reserved (0 if the budget is already exhausted — the first heartbeat
|
|
||||||
will then expire it). PRO (unlimited) is never tracked; don't call it here."""
|
|
||||||
day = _today()
|
|
||||||
budget = budget_for(tier)
|
|
||||||
now = time.time()
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
avail = budget - _spent_max(con, keys, day) if budget is not None else CHUNK_SEC
|
|
||||||
chunk = max(0, min(CHUNK_SEC, avail))
|
|
||||||
if chunk:
|
|
||||||
_add(con, keys, day, chunk)
|
|
||||||
con.execute(
|
|
||||||
"""INSERT OR REPLACE INTO sessions
|
|
||||||
(session_id, keys, day, tier, grant_ts, last_seen_ts, reserved_sec, ended)
|
|
||||||
VALUES (?, ?, ?, ?, ?, ?, ?, 0)""",
|
|
||||||
(session_id, ",".join(keys), day, tier, now, now, chunk),
|
|
||||||
)
|
|
||||||
return chunk
|
|
||||||
|
|
||||||
|
|
||||||
def heartbeat(session_id: str) -> bool:
|
|
||||||
"""Keep a session alive: extend the reservation toward `elapsed + 1 chunk`,
|
|
||||||
debiting the budget chunk by chunk. Returns True while alive, False once the
|
|
||||||
budget is spent (caller should tear down) or the session is unknown/ended."""
|
|
||||||
now = time.time()
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
row = con.execute(
|
|
||||||
"SELECT keys, day, tier, grant_ts, reserved_sec, ended FROM sessions WHERE session_id=?",
|
|
||||||
(session_id,),
|
|
||||||
).fetchone()
|
|
||||||
if not row or row[5]:
|
|
||||||
return False
|
|
||||||
keys = row[0].split(",")
|
|
||||||
day, tier, grant_ts, reserved = row[1], row[2], row[3], int(row[4])
|
|
||||||
budget = budget_for(tier)
|
|
||||||
elapsed = now - grant_ts
|
|
||||||
|
|
||||||
# Grow the reservation one chunk at a time until it covers elapsed + a
|
|
||||||
# one-chunk lookahead, or the budget runs dry.
|
|
||||||
while reserved < elapsed + CHUNK_SEC:
|
|
||||||
if budget is not None and _spent_max(con, keys, day) >= budget:
|
|
||||||
break
|
|
||||||
_add(con, keys, day, CHUNK_SEC)
|
|
||||||
reserved += CHUNK_SEC
|
|
||||||
|
|
||||||
alive = reserved > elapsed # could we cover the time already elapsed?
|
|
||||||
con.execute(
|
|
||||||
"UPDATE sessions SET last_seen_ts=?, reserved_sec=?, ended=? WHERE session_id=?",
|
|
||||||
(now, reserved, 0 if alive else 1, session_id),
|
|
||||||
)
|
|
||||||
if not alive:
|
|
||||||
_reconcile(con, keys, day, grant_ts, reserved, end_ts=now)
|
|
||||||
return alive
|
|
||||||
|
|
||||||
|
|
||||||
def end(session_id: str) -> None:
|
|
||||||
"""Clean teardown: reconcile to actual elapsed time and refund the unused
|
|
||||||
reservation. Idempotent."""
|
|
||||||
now = time.time()
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
row = con.execute(
|
|
||||||
"SELECT keys, day, grant_ts, reserved_sec, ended FROM sessions WHERE session_id=?",
|
|
||||||
(session_id,),
|
|
||||||
).fetchone()
|
|
||||||
if not row or row[4]:
|
|
||||||
return
|
|
||||||
keys, day, grant_ts, reserved = row[0].split(","), row[1], row[2], int(row[3])
|
|
||||||
_reconcile(con, keys, day, grant_ts, reserved, end_ts=now)
|
|
||||||
con.execute("UPDATE sessions SET ended=1, last_seen_ts=? WHERE session_id=?", (now, session_id))
|
|
||||||
|
|
||||||
|
|
||||||
def sweep() -> None:
|
|
||||||
"""Reap sessions that went silent (crash / closed without a beacon): bill
|
|
||||||
their elapsed time, refund the rest, mark ended. Forfeits ≤ one chunk."""
|
|
||||||
now = time.time()
|
|
||||||
cutoff = now - REAP_AFTER_SEC
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
stale = con.execute(
|
|
||||||
"SELECT session_id, keys, day, grant_ts, last_seen_ts, reserved_sec FROM sessions "
|
|
||||||
"WHERE ended=0 AND last_seen_ts < ?",
|
|
||||||
(cutoff,),
|
|
||||||
).fetchall()
|
|
||||||
for session_id, keys_s, day, grant_ts, last_seen, reserved in stale:
|
|
||||||
_reconcile(con, keys_s.split(","), day, grant_ts, int(reserved), end_ts=last_seen)
|
|
||||||
con.execute("UPDATE sessions SET ended=1 WHERE session_id=?", (session_id,))
|
|
||||||
if stale:
|
|
||||||
logger.debug("swept %d stale session(s)", len(stale))
|
|
||||||
|
|
||||||
|
|
||||||
def _reconcile(con, keys, day: str, grant_ts: float, reserved: int, end_ts: float) -> None:
|
|
||||||
"""Refund reserved-but-unused time. Bill elapsed rounded up to a chunk,
|
|
||||||
capped at what was reserved."""
|
|
||||||
elapsed = max(0.0, end_ts - grant_ts)
|
|
||||||
billed = min(reserved, int(math.ceil(elapsed / CHUNK_SEC) * CHUNK_SEC))
|
|
||||||
refund = reserved - billed
|
|
||||||
if refund > 0:
|
|
||||||
_add(con, keys, day, -refund)
|
|
||||||
-3653
File diff suppressed because it is too large
Load Diff
Generated
-42
@@ -1,42 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "vocero-ui",
|
|
||||||
"lockfileVersion": 3,
|
|
||||||
"requires": true,
|
|
||||||
"packages": {
|
|
||||||
"": {
|
|
||||||
"name": "vocero-ui",
|
|
||||||
"dependencies": {
|
|
||||||
"dompurify": "3.4.16",
|
|
||||||
"marked": "18.0.14"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/@types/trusted-types": {
|
|
||||||
"version": "2.0.7",
|
|
||||||
"resolved": "https://registry.npmjs.org/@types/trusted-types/-/trusted-types-2.0.7.tgz",
|
|
||||||
"integrity": "sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"optional": true
|
|
||||||
},
|
|
||||||
"node_modules/dompurify": {
|
|
||||||
"version": "3.4.16",
|
|
||||||
"resolved": "https://registry.npmjs.org/dompurify/-/dompurify-3.4.16.tgz",
|
|
||||||
"integrity": "sha512-sqo+pNp3qRhCIpbgRi1y8Tgk27Bo2Ry7w0dC1NBeNTdZChWjz9Xb/KOoZbRP/R6pQZ80Qw8YhXw13hWWBbMRnQ==",
|
|
||||||
"license": "(MPL-2.0 OR Apache-2.0)",
|
|
||||||
"optionalDependencies": {
|
|
||||||
"@types/trusted-types": "^2.0.7"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/marked": {
|
|
||||||
"version": "18.0.14",
|
|
||||||
"resolved": "https://registry.npmjs.org/marked/-/marked-18.0.14.tgz",
|
|
||||||
"integrity": "sha512-mBHK6FBHuBAlhgRe88w9F0O1AbwwXJUcQibUbC/QcdTbVGAD7aWza+xt3N6oT/jCZx3/OMeS+8rnuiHZcQ9s7A==",
|
|
||||||
"license": "MIT",
|
|
||||||
"bin": {
|
|
||||||
"marked": "bin/marked.js"
|
|
||||||
},
|
|
||||||
"engines": {
|
|
||||||
"node": ">= 20"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "vocero-ui",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"scripts": {
|
|
||||||
"vendor": "node scripts/vendor-dependencies.mjs"
|
|
||||||
},
|
|
||||||
"dependencies": {
|
|
||||||
"dompurify": "3.4.16",
|
|
||||||
"marked": "18.0.14"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -2,5 +2,3 @@ fastapi==0.115.6
|
|||||||
uvicorn[standard]==0.34.0
|
uvicorn[standard]==0.34.0
|
||||||
httpx==0.28.1
|
httpx==0.28.1
|
||||||
python-dotenv==1.2.2
|
python-dotenv==1.2.2
|
||||||
# [oauth] extra pulls authlib + itsdangerous for attach_huggingface_oauth.
|
|
||||||
huggingface_hub[oauth]>=0.30,<1.0
|
|
||||||
|
|||||||
@@ -1,18 +0,0 @@
|
|||||||
import { copyFile, mkdir } from "node:fs/promises";
|
|
||||||
|
|
||||||
const root = new URL("../", import.meta.url);
|
|
||||||
const assets = [
|
|
||||||
["marked/lib/marked.esm.js", "marked/marked.esm.js"],
|
|
||||||
["marked/lib/marked.esm.js.map", "marked/marked.esm.js.map"],
|
|
||||||
["marked/LICENSE", "marked/LICENSE"],
|
|
||||||
["dompurify/dist/purify.es.mjs", "dompurify/purify.es.mjs"],
|
|
||||||
["dompurify/dist/purify.es.mjs.map", "dompurify/purify.es.mjs.map"],
|
|
||||||
["dompurify/LICENSE", "dompurify/LICENSE"],
|
|
||||||
["dompurify/LICENSE-MPL", "dompurify/LICENSE-MPL"],
|
|
||||||
];
|
|
||||||
|
|
||||||
for (const [source, destination] of assets) {
|
|
||||||
const target = new URL(`vendor/${destination}`, root);
|
|
||||||
await mkdir(new URL("./", target), { recursive: true });
|
|
||||||
await copyFile(new URL(`node_modules/${source}`, root), target);
|
|
||||||
}
|
|
||||||
+30
-277
@@ -1,43 +1,24 @@
|
|||||||
"""
|
"""
|
||||||
Tiny server for the speech-to-speech demo.
|
API and WebSocket proxy for the speech-to-speech frontend.
|
||||||
|
|
||||||
The demo used to ship as a `sdk: static` Space, but the web-search tool needs a
|
Vite serves the React application and forwards backend requests here. Search,
|
||||||
search key the browser must NOT see. A static Space has no runtime process, so it
|
caller-profile, and tool credentials stay server-side. `/api/session` proxies
|
||||||
can't hold a secret the front-end uses. This server fixes that: it serves the
|
load-balancer allocation so the browser receives only the per-session compute
|
||||||
unchanged front-end AND exposes a same-origin `/api/search` proxy that holds the
|
URL it must dial. A pinned direct URL or WebSocket proxy target disables the LB.
|
||||||
Serper key server-side (see docs/adr/0001).
|
|
||||||
|
|
||||||
Everything lives in one container; the speech-to-speech backend stays a separate,
|
|
||||||
load-balanced service the browser talks to over WebSocket as before. The load
|
|
||||||
balancer's address is a secret too (like the Serper key): the browser never sees
|
|
||||||
it. `/api/session` proxies the session handshake server-side so only the
|
|
||||||
per-session compute URL the LB hands back (which the browser must dial) is exposed.
|
|
||||||
|
|
||||||
On the deployed Space the server also meters conversation time by HF login tier
|
|
||||||
(anonymous / signed-in / PRO) — see `limiter.py` and `auth.py`. That whole feature
|
|
||||||
is off unless BOTH `LOAD_BALANCER_URL` and `SPACE_ID` are set, so it runs only on
|
|
||||||
the live Space, never locally (even with the LB exported for testing).
|
|
||||||
|
|
||||||
`SPEECH_TO_SPEECH_URL` overrides everything: when set, the LB logic above is
|
|
||||||
disabled entirely (no session proxy, no queue, no metering, no sign-in) and the
|
|
||||||
browser connects directly to that URL, shown read-only in Settings.
|
|
||||||
|
|
||||||
Endpoints:
|
Endpoints:
|
||||||
GET /api/config -> { search, lb, allowDirect, s2sUrl, auth }
|
GET /api/config -> frontend settings and available integrations
|
||||||
GET /api/me -> login + tier + remaining budget (LB mode only)
|
|
||||||
POST /api/search -> { results, answer } Google via Serper.dev
|
POST /api/search -> { results, answer } Google via Serper.dev
|
||||||
POST /api/session -> proxies <LB>/session: a grant, or a queue ticket
|
POST /api/session -> proxies <LB>/session: a grant, or a queue ticket
|
||||||
GET /api/queue/{id} -> proxies <LB>/queue/{id}: position, or a grant on claim
|
GET /api/queue/{id} -> proxies <LB>/queue/{id}: position, or a grant on claim
|
||||||
DELETE /api/queue/{id} -> leave the queue (explicit "Leave queue" button)
|
DELETE /api/queue/{id} -> leave the queue (explicit "Leave queue" button)
|
||||||
POST /api/queue/end -> leave the queue (sendBeacon on teardown)
|
POST /api/queue/end -> leave the queue (sendBeacon on teardown)
|
||||||
POST /api/session/heartbeat-> extend the reservation; { expired }
|
WS /v1/realtime -> optional speech-service WebSocket proxy
|
||||||
POST /api/session/end -> reconcile + refund (sendBeacon on teardown)
|
GET /docs/* -> shared project documentation
|
||||||
/* -> static files (index.html, main.js, ...)
|
|
||||||
|
|
||||||
When every compute slot is busy the load balancer hands back a queue ticket
|
When every compute slot is busy the load balancer hands back a queue ticket
|
||||||
instead of a grant; the browser polls /api/queue/{id} until it reaches the front
|
instead of a grant; the browser polls /api/queue/{id} until it reaches the front
|
||||||
and a slot frees. Waiting reserves nothing — the daily budget is only reserved at
|
and a slot frees. Queue deadlines and pending-grant timeouts belong to the LB.
|
||||||
the moment a slot is actually claimed (a grant), never while queued.
|
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -58,9 +39,6 @@ from fastapi.responses import JSONResponse, RedirectResponse
|
|||||||
from fastapi.staticfiles import StaticFiles
|
from fastapi.staticfiles import StaticFiles
|
||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
|
|
||||||
import auth
|
|
||||||
import limiter
|
|
||||||
|
|
||||||
logger = logging.getLogger("s2s")
|
logger = logging.getLogger("s2s")
|
||||||
_logging_configured = False
|
_logging_configured = False
|
||||||
if not _logging_configured:
|
if not _logging_configured:
|
||||||
@@ -1402,9 +1380,9 @@ NODE_RED_REASONING_URL = os.environ.get("S2S_REASONING_NODE_RED_URL", "").strip(
|
|||||||
# browser connects to it straight (no load balancer).
|
# browser connects to it straight (no load balancer).
|
||||||
LOAD_BALANCER_URL = os.environ.get("LOAD_BALANCER_URL", "").strip()
|
LOAD_BALANCER_URL = os.environ.get("LOAD_BALANCER_URL", "").strip()
|
||||||
# Direct s2s server URL pinned by the deploy. Takes priority over the load
|
# Direct s2s server URL pinned by the deploy. Takes priority over the load
|
||||||
# balancer: when set, ALL LB logic is disabled (no /api/session proxy, no queue,
|
# balancer: when set, LB allocation and queuing are disabled, and the browser
|
||||||
# no limiter, no sign-in) and the browser connects to this URL directly. Unlike
|
# connects to this URL directly. Unlike the LB address it is NOT a secret —
|
||||||
# the LB address it is NOT a secret — /api/config sends it to the client, which
|
# /api/config sends it to the client, which
|
||||||
# shows it read-only in Settings.
|
# shows it read-only in Settings.
|
||||||
SPEECH_TO_SPEECH_URL = os.environ.get("SPEECH_TO_SPEECH_URL", "").strip()
|
SPEECH_TO_SPEECH_URL = os.environ.get("SPEECH_TO_SPEECH_URL", "").strip()
|
||||||
# Optional same-origin WebSocket relay target. When set, the browser dials this
|
# Optional same-origin WebSocket relay target. When set, the browser dials this
|
||||||
@@ -1413,24 +1391,12 @@ SPEECH_TO_SPEECH_URL = os.environ.get("SPEECH_TO_SPEECH_URL", "").strip()
|
|||||||
S2S_PROXY_TARGET = os.environ.get("S2S_PROXY_TARGET", "").strip()
|
S2S_PROXY_TARGET = os.environ.get("S2S_PROXY_TARGET", "").strip()
|
||||||
if SPEECH_TO_SPEECH_URL or S2S_PROXY_TARGET:
|
if SPEECH_TO_SPEECH_URL or S2S_PROXY_TARGET:
|
||||||
LOAD_BALANCER_URL = ""
|
LOAD_BALANCER_URL = ""
|
||||||
# HF injects SPACE_ID ("owner/space") into every Space runtime; it's absent
|
|
||||||
# locally and on a plain `docker run`. We meter conversation time ONLY on the
|
|
||||||
# deployed Space — i.e. when BOTH the LB is configured AND we're on a Space.
|
|
||||||
# Off-Space (local dev, even with the LB exported) the app still proxies the LB,
|
|
||||||
# but nothing is metered: no budget, no reservations, no sign-in gating.
|
|
||||||
SPACE_ID = os.environ.get("SPACE_ID", "").strip()
|
|
||||||
LIMITER_ENABLED = bool(LOAD_BALANCER_URL) and bool(SPACE_ID)
|
|
||||||
SERPER_URL = "https://google.serper.dev/search"
|
SERPER_URL = "https://google.serper.dev/search"
|
||||||
# Cap results so the tool output stays small enough to feed back to the model.
|
# Cap results so the tool output stays small enough to feed back to the model.
|
||||||
MAX_RESULTS = 5
|
MAX_RESULTS = 5
|
||||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
|
||||||
|
|
||||||
app = FastAPI(title="s2s-demo")
|
app = FastAPI(title="s2s-demo")
|
||||||
|
|
||||||
# Wire HF OAuth before the app serves (no-op unless the OAuth env is present).
|
|
||||||
# Sign-in only matters when we're metering (prod Space), so gate it on that.
|
|
||||||
AUTH_ENABLED = LIMITER_ENABLED and auth.attach(app)
|
|
||||||
|
|
||||||
# Resolve the execution URL for the startup banner using the same normalization
|
# Resolve the execution URL for the startup banner using the same normalization
|
||||||
# as _apipilot_resolve_execute_url.
|
# as _apipilot_resolve_execute_url.
|
||||||
def _resolve_exec_url_for_startup_banner():
|
def _resolve_exec_url_for_startup_banner():
|
||||||
@@ -1444,7 +1410,7 @@ _STARTUP_APIPILOT_EXEC_URL = _resolve_exec_url_for_startup_banner()
|
|||||||
del _resolve_exec_url_for_startup_banner
|
del _resolve_exec_url_for_startup_banner
|
||||||
|
|
||||||
logger.info(
|
logger.info(
|
||||||
"[STARTUP] Other externals — reasoning=%s nodeRed=%s serper=%s lb=%s s2sProxy=%s s2sDirect=%s userAuth=%s reportsApi=%s injectReports=%s reportsHintMax=%d reportsInjectMode=%s injectToday=%s summaryApi=%s summaryText=%s loginApi=%s injectProfile=%s debugPrompt=%s startSessionMode=%s theme=%s auth=%s apipilotToolsUrl=%s apipilotExecuteUrlOverride=%s apipilotServiceToken=%s apipilotTtl=%ds apipilotTimeout=%.1fs apipilotReasoningPass2=%s apipilotApiKeyBypass=%s",
|
"[STARTUP] Other externals — reasoning=%s nodeRed=%s serper=%s lb=%s s2sProxy=%s s2sDirect=%s userAuth=%s reportsApi=%s injectReports=%s reportsHintMax=%d reportsInjectMode=%s injectToday=%s summaryApi=%s summaryText=%s loginApi=%s injectProfile=%s debugPrompt=%s startSessionMode=%s theme=%s apipilotToolsUrl=%s apipilotExecuteUrlOverride=%s apipilotServiceToken=%s apipilotTtl=%ds apipilotTimeout=%.1fs apipilotReasoningPass2=%s apipilotApiKeyBypass=%s",
|
||||||
S2S_REASONING_TYPE,
|
S2S_REASONING_TYPE,
|
||||||
NODE_RED_REASONING_URL or "-",
|
NODE_RED_REASONING_URL or "-",
|
||||||
bool(SERPER_KEY),
|
bool(SERPER_KEY),
|
||||||
@@ -1464,7 +1430,6 @@ logger.info(
|
|||||||
S2S_DEBUG_PROMPT,
|
S2S_DEBUG_PROMPT,
|
||||||
S2S_START_SESSION_MODE or "-",
|
S2S_START_SESSION_MODE or "-",
|
||||||
S2S_THEME or "-",
|
S2S_THEME or "-",
|
||||||
AUTH_ENABLED,
|
|
||||||
S2S_APIPILOT_TOOLS_URL or "-",
|
S2S_APIPILOT_TOOLS_URL or "-",
|
||||||
(S2S_APIPILOT_TOOLS_EXECUTE_URL + f"(resolved:{_STARTUP_APIPILOT_EXEC_URL})") if S2S_APIPILOT_TOOLS_EXECUTE_URL else "-",
|
(S2S_APIPILOT_TOOLS_EXECUTE_URL + f"(resolved:{_STARTUP_APIPILOT_EXEC_URL})") if S2S_APIPILOT_TOOLS_EXECUTE_URL else "-",
|
||||||
"Y" if bool(S2S_APIPILOT_SERVICE_TOKEN) else "-",
|
"Y" if bool(S2S_APIPILOT_SERVICE_TOKEN) else "-",
|
||||||
@@ -1474,10 +1439,6 @@ logger.info(
|
|||||||
("YES head=" + (S2S_APIPILOT_API_KEY[:6] + "..." if len(S2S_APIPILOT_API_KEY) > 6 else S2S_APIPILOT_API_KEY) + " (3 FORMS!)") if S2S_APIPILOT_API_KEY else "-",
|
("YES head=" + (S2S_APIPILOT_API_KEY[:6] + "..." if len(S2S_APIPILOT_API_KEY) > 6 else S2S_APIPILOT_API_KEY) + " (3 FORMS!)") if S2S_APIPILOT_API_KEY else "-",
|
||||||
)
|
)
|
||||||
del _STARTUP_APIPILOT_EXEC_URL
|
del _STARTUP_APIPILOT_EXEC_URL
|
||||||
logger.debug(
|
|
||||||
"[STARTUP] Runtime config — SPACE_ID=%s LIMITER_ENABLED=%s MAX_RESULTS=%d",
|
|
||||||
bool(SPACE_ID), LIMITER_ENABLED, MAX_RESULTS,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def _report_server_debug(hypothesis_id: str, location: str, msg: str, data: dict | None = None) -> None:
|
async def _report_server_debug(hypothesis_id: str, location: str, msg: str, data: dict | None = None) -> None:
|
||||||
@@ -1652,22 +1613,6 @@ _NOISY_CLIENT_EVENT_TYPES = frozenset({
|
|||||||
"input_audio_buffer.speech_started",
|
"input_audio_buffer.speech_started",
|
||||||
"input_audio_buffer.speech_stopped",
|
"input_audio_buffer.speech_stopped",
|
||||||
})
|
})
|
||||||
@app.on_event("startup")
|
|
||||||
async def _startup():
|
|
||||||
"""Stand up the usage DB and a periodic sweeper — metered (prod Space) only."""
|
|
||||||
if not LIMITER_ENABLED:
|
|
||||||
return
|
|
||||||
limiter.init()
|
|
||||||
asyncio.create_task(_sweeper())
|
|
||||||
|
|
||||||
|
|
||||||
async def _sweeper():
|
|
||||||
while True:
|
|
||||||
await asyncio.sleep(limiter.REAP_AFTER_SEC)
|
|
||||||
try:
|
|
||||||
await asyncio.to_thread(limiter.sweep)
|
|
||||||
except Exception as exc: # pragma: no cover - defensive
|
|
||||||
logger.warning("usage sweep failed: %r", exc)
|
|
||||||
|
|
||||||
|
|
||||||
class SearchRequest(BaseModel):
|
class SearchRequest(BaseModel):
|
||||||
@@ -3334,8 +3279,8 @@ def _effective_s2s_url(request: Request) -> str:
|
|||||||
@app.get("/api/config")
|
@app.get("/api/config")
|
||||||
async def config(request: Request):
|
async def config(request: Request):
|
||||||
"""Client bootstrap: whether web search is available, whether the deploy runs
|
"""Client bootstrap: whether web search is available, whether the deploy runs
|
||||||
behind a load balancer (so the browser uses the /api/session proxy + limiter),
|
behind a load balancer (so the browser uses the /api/session proxy),
|
||||||
whether HF sign-in is available, and whether the user may instead set a direct
|
and whether the user may instead set a direct
|
||||||
s2s server URL. The LB address itself is intentionally NOT included.
|
s2s server URL. The LB address itself is intentionally NOT included.
|
||||||
|
|
||||||
2026-09-24: new top-level `apipilot` flag. When True the deploy exposes the
|
2026-09-24: new top-level `apipilot` flag. When True the deploy exposes the
|
||||||
@@ -3351,7 +3296,6 @@ async def config(request: Request):
|
|||||||
"s2sUrl": _effective_s2s_url(request),
|
"s2sUrl": _effective_s2s_url(request),
|
||||||
"primaryPrompt": primary_prompt,
|
"primaryPrompt": primary_prompt,
|
||||||
"reasoningType": S2S_REASONING_TYPE,
|
"reasoningType": S2S_REASONING_TYPE,
|
||||||
"auth": AUTH_ENABLED,
|
|
||||||
"injectProfile": S2S_INJECT_PROFILE,
|
"injectProfile": S2S_INJECT_PROFILE,
|
||||||
"debugPrompt": S2S_DEBUG_PROMPT,
|
"debugPrompt": S2S_DEBUG_PROMPT,
|
||||||
"startSessionMode": S2S_START_SESSION_MODE,
|
"startSessionMode": S2S_START_SESSION_MODE,
|
||||||
@@ -3536,7 +3480,7 @@ async def reasoning(req: ReasoningRequest):
|
|||||||
elif (not phone) and (not needs_reports) and (not episode_id) and (not S2S_LATEST_SUMMARY_TEXT):
|
elif (not phone) and (not needs_reports) and (not episode_id) and (not S2S_LATEST_SUMMARY_TEXT):
|
||||||
logger.info(
|
logger.info(
|
||||||
"[INJECT-PROFILE] ⏭️ REASONING: skipped (phoneNumber missing from request body AND no episode_id). "
|
"[INJECT-PROFILE] ⏭️ REASONING: skipped (phoneNumber missing from request body AND no episode_id). "
|
||||||
"Browser-side issue: check main.js passes voceroUserPhoneNumber and/or episode_id in JSON body. (Note: S2S_REPORT_LIST fetch works even without phone.)"
|
"Browser-side issue: check VoiceController passes phoneNumber and/or episode_id in the JSON body. (Note: S2S_REPORT_LIST fetch works even without phone.)"
|
||||||
)
|
)
|
||||||
elif not can_inject:
|
elif not can_inject:
|
||||||
logger.info(
|
logger.info(
|
||||||
@@ -3813,7 +3757,7 @@ async def reasoning(req: ReasoningRequest):
|
|||||||
|
|
||||||
# ────────────────────────────────────────────────────────────────────
|
# ────────────────────────────────────────────────────────────────────
|
||||||
# Mode A: NODE_RED forwards the complete payload; the flow decides behavior.
|
# Mode A: NODE_RED forwards the complete payload; the flow decides behavior.
|
||||||
# Stage 1 (log-only) returns {answer: "", log_only: true}, allowing main.js
|
# Stage 1 (log-only) returns {answer: "", log_only: true}, allowing the browser
|
||||||
# to leave speech generation to the S2S backend. Later stages can return
|
# to leave speech generation to the S2S backend. Later stages can return
|
||||||
# {answer: "...", log_only: false}. Expected response fields include answer,
|
# {answer: "...", log_only: false}. Expected response fields include answer,
|
||||||
# log_only, usage, debug and latency_ms.
|
# log_only, usage, debug and latency_ms.
|
||||||
@@ -4605,7 +4549,7 @@ async def realtime_proxy(ws: WebSocket):
|
|||||||
pass
|
pass
|
||||||
session_ctx_apipilot_defaults = dict(_apipilot_defaults_p1)
|
session_ctx_apipilot_defaults = dict(_apipilot_defaults_p1)
|
||||||
injected_apipilot_pass1 = _resolve_apipilot_reasoning_context_block(
|
injected_apipilot_pass1 = _resolve_apipilot_reasoning_context_block(
|
||||||
tools=apipilot_tools_pass1, context_defaults=_apipilot_defaults_p1,
|
tools_list=apipilot_tools_pass1, context_defaults=_apipilot_defaults_p1,
|
||||||
) or ""
|
) or ""
|
||||||
if injected_apipilot_pass1.strip():
|
if injected_apipilot_pass1.strip():
|
||||||
logger.info(
|
logger.info(
|
||||||
@@ -5029,40 +4973,6 @@ async def user_reports(phoneNumber: str | None = None):
|
|||||||
return JSONResponse(resp_body)
|
return JSONResponse(resp_body)
|
||||||
|
|
||||||
|
|
||||||
@app.get("/api/me")
|
|
||||||
async def me(request: Request):
|
|
||||||
"""Login state, tier, and remaining daily budget. Only meaningful in LB mode;
|
|
||||||
sets the anonymous tracking cookie when first seen."""
|
|
||||||
client_ip = request.client.host if request.client else "?"
|
|
||||||
logger.debug(
|
|
||||||
"[INTERNAL] /api/me client=%s LIMITER=%s",
|
|
||||||
client_ip, LIMITER_ENABLED,
|
|
||||||
)
|
|
||||||
if not LIMITER_ENABLED:
|
|
||||||
return {"enabled": False}
|
|
||||||
view = auth.user_view(request)
|
|
||||||
tier, keys, set_cookie = auth.resolve_identity(request)
|
|
||||||
unlimited = limiter.budget_for(tier) is None
|
|
||||||
rem = None if unlimited else await asyncio.to_thread(limiter.remaining, keys, tier)
|
|
||||||
logger.debug(
|
|
||||||
"[INTERNAL] /api/me client=%s tier=%s unlimited=%s remaining=%s setCookie=%s",
|
|
||||||
client_ip, tier, unlimited, rem, bool(set_cookie),
|
|
||||||
)
|
|
||||||
out = {
|
|
||||||
"enabled": True,
|
|
||||||
"auth": AUTH_ENABLED,
|
|
||||||
**view,
|
|
||||||
"remainingSec": rem,
|
|
||||||
"limitSec": limiter.budget_for(tier),
|
|
||||||
"loginUrl": auth.OAUTH_LOGIN_PATH if AUTH_ENABLED else None,
|
|
||||||
"logoutUrl": auth.OAUTH_LOGOUT_PATH if AUTH_ENABLED else None,
|
|
||||||
}
|
|
||||||
resp = JSONResponse(out)
|
|
||||||
if set_cookie:
|
|
||||||
auth.set_anon_cookie(resp, set_cookie)
|
|
||||||
return resp
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/api/search")
|
@app.post("/api/search")
|
||||||
async def search(req: SearchRequest):
|
async def search(req: SearchRequest):
|
||||||
"""Proxy a Google search via Serper.dev. The key stays on the server unless
|
"""Proxy a Google search via Serper.dev. The key stays on the server unless
|
||||||
@@ -5136,44 +5046,17 @@ async def search(req: SearchRequest):
|
|||||||
|
|
||||||
@app.post("/api/session")
|
@app.post("/api/session")
|
||||||
async def session(request: Request):
|
async def session(request: Request):
|
||||||
"""Proxy the session handshake to the load balancer, keeping its URL secret,
|
"""Relay a session grant or queue ticket, keeping the LB address server-side.
|
||||||
and meter conversation time by tier.
|
|
||||||
|
|
||||||
The browser POSTs here (same-origin); we resolve the caller's tier, refuse if
|
The grant's `connect_url` contains the compute host and short-lived token the
|
||||||
today's budget is already spent (402), otherwise POST <LOAD_BALANCER_URL>/session
|
browser needs to connect directly to the allocated speech service."""
|
||||||
and relay the JSON back. The LB body carries a per-session `connect_url`
|
|
||||||
(compute host + short-lived token) the browser must dial directly — that one
|
|
||||||
URL is unavoidably exposed, but the stable load-balancer address is not. On a
|
|
||||||
successful grant we reserve the first time chunk against the day's budget."""
|
|
||||||
client_ip = request.client.host if request.client else "?"
|
client_ip = request.client.host if request.client else "?"
|
||||||
if not LOAD_BALANCER_URL:
|
if not LOAD_BALANCER_URL:
|
||||||
# No LB configured — this deploy is direct-mode only; the browser should
|
# No LB configured — this deploy is direct-mode only; the browser should
|
||||||
# never call this. 404 so it's indistinguishable from a missing route.
|
# never call this. 404 so it's indistinguishable from a missing route.
|
||||||
raise HTTPException(status_code=404, detail="No encontrado.")
|
raise HTTPException(status_code=404, detail="No encontrado.")
|
||||||
|
|
||||||
tier, keys, set_cookie = auth.resolve_identity(request)
|
logger.debug("[EXT][LB] session: client=%s", client_ip)
|
||||||
# Metering runs only on the deployed Space; off-Space the LB still proxies but
|
|
||||||
# nothing is tracked. Within metering, unlimited tiers (pro, org) aren't either.
|
|
||||||
tracked = LIMITER_ENABLED and limiter.budget_for(tier) is not None
|
|
||||||
|
|
||||||
logger.debug(
|
|
||||||
"[EXT][LB] session: client=%s tier=%s tracked=%s LIMITER=%s",
|
|
||||||
client_ip, tier, tracked, LIMITER_ENABLED,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Refuse before troubling the LB if the day's budget is already gone. Done
|
|
||||||
# here (at enqueue) so we never put a user who can't talk into the queue.
|
|
||||||
if tracked:
|
|
||||||
rem = await asyncio.to_thread(limiter.remaining, keys, tier)
|
|
||||||
logger.debug("[EXT][LB] session: client=%s budget remaining=%s", client_ip, rem)
|
|
||||||
if rem is not None and rem <= 0:
|
|
||||||
resp = JSONResponse(
|
|
||||||
{"tier": tier, "reason": "limit", "remainingSec": 0}, status_code=402
|
|
||||||
)
|
|
||||||
if set_cookie:
|
|
||||||
auth.set_anon_cookie(resp, set_cookie)
|
|
||||||
return resp
|
|
||||||
|
|
||||||
url = f"{LOAD_BALANCER_URL.rstrip('/')}/session"
|
url = f"{LOAD_BALANCER_URL.rstrip('/')}/session"
|
||||||
logger.debug("[EXT][LB] session POST %s (handshake)", url)
|
logger.debug("[EXT][LB] session POST %s (handshake)", url)
|
||||||
try:
|
try:
|
||||||
@@ -5193,10 +5076,7 @@ async def session(request: Request):
|
|||||||
body = _safe_json(lb)
|
body = _safe_json(lb)
|
||||||
if body.get("state") == "at_capacity":
|
if body.get("state") == "at_capacity":
|
||||||
logger.debug("[EXT][LB] session: at_capacity — queue full")
|
logger.debug("[EXT][LB] session: at_capacity — queue full")
|
||||||
resp = JSONResponse({"state": "at_capacity"}, status_code=503)
|
return JSONResponse({"state": "at_capacity"}, status_code=503)
|
||||||
if set_cookie:
|
|
||||||
auth.set_anon_cookie(resp, set_cookie)
|
|
||||||
return resp
|
|
||||||
|
|
||||||
if lb.status_code != 200:
|
if lb.status_code != 200:
|
||||||
# The LB's error body may name the reason (e.g. capacity); it carries no
|
# The LB's error body may name the reason (e.g. capacity); it carries no
|
||||||
@@ -5213,36 +5093,24 @@ async def session(request: Request):
|
|||||||
state, session_id, bool(connect_url), connect_url,
|
state, session_id, bool(connect_url), connect_url,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Busy pool: the LB queued us. Relay the ticket untouched — crucially with NO
|
# Busy pool: the LB queued us. Relay the ticket for subsequent polling.
|
||||||
# reservation, so waiting in line never costs the day's budget.
|
|
||||||
if data.get("state") == "queued":
|
if data.get("state") == "queued":
|
||||||
data["tier"] = tier
|
|
||||||
logger.debug("[EXT][LB] session: queued queueId=%s position=%s",
|
logger.debug("[EXT][LB] session: queued queueId=%s position=%s",
|
||||||
data.get("queue_id") or data.get("queueId"),
|
data.get("queue_id") or data.get("queueId"),
|
||||||
data.get("position"))
|
data.get("position"))
|
||||||
resp = JSONResponse(data)
|
return JSONResponse(data)
|
||||||
if set_cookie:
|
|
||||||
auth.set_anon_cookie(resp, set_cookie)
|
|
||||||
return resp
|
|
||||||
|
|
||||||
# A slot was free: reserve the first chunk now and return the grant.
|
|
||||||
return await _finalize_grant(data, keys, tier, tracked, set_cookie)
|
|
||||||
|
|
||||||
|
|
||||||
@app.get("/api/queue/{queue_id}")
|
@app.get("/api/queue/{queue_id}")
|
||||||
async def queue_status(queue_id: str, request: Request):
|
async def queue_status(queue_id: str, request: Request):
|
||||||
"""Poll a waiting ticket: relay the position, or — when the head of the line
|
"""Relay a waiting ticket's position or its newly claimed session grant."""
|
||||||
claims a freed slot — reserve the budget now and return the grant. Re-checks the
|
|
||||||
daily budget at claim, since a multi-minute wait could have spent it elsewhere."""
|
|
||||||
client_ip = request.client.host if request.client else "?"
|
client_ip = request.client.host if request.client else "?"
|
||||||
if not LOAD_BALANCER_URL:
|
if not LOAD_BALANCER_URL:
|
||||||
raise HTTPException(status_code=404, detail="No encontrado.")
|
raise HTTPException(status_code=404, detail="No encontrado.")
|
||||||
|
|
||||||
tier, keys, set_cookie = auth.resolve_identity(request)
|
|
||||||
tracked = LIMITER_ENABLED and limiter.budget_for(tier) is not None
|
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"[EXT][LB] queue poll: client=%s queueId=%s tier=%s",
|
"[EXT][LB] queue poll: client=%s queueId=%s",
|
||||||
client_ip, queue_id[:16], tier,
|
client_ip, queue_id[:16],
|
||||||
)
|
)
|
||||||
|
|
||||||
url = f"{LOAD_BALANCER_URL.rstrip('/')}/queue/{queue_id}"
|
url = f"{LOAD_BALANCER_URL.rstrip('/')}/queue/{queue_id}"
|
||||||
@@ -5262,10 +5130,7 @@ async def queue_status(queue_id: str, request: Request):
|
|||||||
# Ticket unknown/expired (reaped after we stopped polling). Tell the client
|
# Ticket unknown/expired (reaped after we stopped polling). Tell the client
|
||||||
# to start over rather than spin.
|
# to start over rather than spin.
|
||||||
logger.debug("[EXT][LB] queue: queueId=%s EXPIRED (404)", queue_id[:16])
|
logger.debug("[EXT][LB] queue: queueId=%s EXPIRED (404)", queue_id[:16])
|
||||||
resp = JSONResponse({"state": "expired"}, status_code=404)
|
return JSONResponse({"state": "expired"}, status_code=404)
|
||||||
if set_cookie:
|
|
||||||
auth.set_anon_cookie(resp, set_cookie)
|
|
||||||
return resp
|
|
||||||
|
|
||||||
if lb.status_code != 200:
|
if lb.status_code != 200:
|
||||||
logger.warning("Queue poll failed %s: %s", lb.status_code, lb.text[:300])
|
logger.warning("Queue poll failed %s: %s", lb.status_code, lb.text[:300])
|
||||||
@@ -5280,31 +5145,7 @@ async def queue_status(queue_id: str, request: Request):
|
|||||||
data.get("session_id") or data.get("sessionId"),
|
data.get("session_id") or data.get("sessionId"),
|
||||||
)
|
)
|
||||||
|
|
||||||
if data.get("state") == "queued":
|
return JSONResponse(data)
|
||||||
data["tier"] = tier
|
|
||||||
resp = JSONResponse(data)
|
|
||||||
if set_cookie:
|
|
||||||
auth.set_anon_cookie(resp, set_cookie)
|
|
||||||
return resp
|
|
||||||
|
|
||||||
# Claimed a slot. Re-check the budget: it may have been spent in another tab
|
|
||||||
# during the wait. If so, refuse — the just-claimed slot is now a pending
|
|
||||||
# session on the LB and its pending-timeout reaper reclaims it shortly.
|
|
||||||
if tracked:
|
|
||||||
rem = await asyncio.to_thread(limiter.remaining, keys, tier)
|
|
||||||
logger.debug(
|
|
||||||
"[EXT][LB] queue CLAIMED queueId=%s budget remaining=%s",
|
|
||||||
queue_id[:16], rem,
|
|
||||||
)
|
|
||||||
if rem is not None and rem <= 0:
|
|
||||||
resp = JSONResponse(
|
|
||||||
{"tier": tier, "reason": "limit", "remainingSec": 0}, status_code=402
|
|
||||||
)
|
|
||||||
if set_cookie:
|
|
||||||
auth.set_anon_cookie(resp, set_cookie)
|
|
||||||
return resp
|
|
||||||
|
|
||||||
return await _finalize_grant(data, keys, tier, tracked, set_cookie)
|
|
||||||
|
|
||||||
|
|
||||||
@app.delete("/api/queue/{queue_id}")
|
@app.delete("/api/queue/{queue_id}")
|
||||||
@@ -5330,35 +5171,6 @@ async def queue_end(request: Request):
|
|||||||
return {"ok": True}
|
return {"ok": True}
|
||||||
|
|
||||||
|
|
||||||
async def _finalize_grant(data, keys, tier, tracked, set_cookie):
|
|
||||||
"""Shared grant tail (fast path or queue claim): reserve the first chunk, attach
|
|
||||||
the metering fields the client needs, and set the anon cookie."""
|
|
||||||
remaining = None
|
|
||||||
session_id = data.get("session_id")
|
|
||||||
if tracked and session_id:
|
|
||||||
logger.debug(
|
|
||||||
"[EXT][LB] grant FINALIZE sessionId=%s tier=%s tracked=%s",
|
|
||||||
session_id, tier, tracked,
|
|
||||||
)
|
|
||||||
await asyncio.to_thread(limiter.begin, session_id, keys, tier)
|
|
||||||
remaining = await asyncio.to_thread(limiter.remaining, keys, tier)
|
|
||||||
logger.debug(
|
|
||||||
"[EXT][LB] grant RESERVED sessionId=%s remaining=%s",
|
|
||||||
session_id, remaining,
|
|
||||||
)
|
|
||||||
|
|
||||||
data.update({
|
|
||||||
"tier": tier,
|
|
||||||
"limited": tracked,
|
|
||||||
"remainingSec": remaining,
|
|
||||||
"heartbeatSec": limiter.HEARTBEAT_SEC,
|
|
||||||
})
|
|
||||||
resp = JSONResponse(data)
|
|
||||||
if set_cookie:
|
|
||||||
auth.set_anon_cookie(resp, set_cookie)
|
|
||||||
return resp
|
|
||||||
|
|
||||||
|
|
||||||
async def _lb_leave(queue_id: str) -> None:
|
async def _lb_leave(queue_id: str) -> None:
|
||||||
"""Best-effort: tell the LB to drop a waiting ticket."""
|
"""Best-effort: tell the LB to drop a waiting ticket."""
|
||||||
url = f"{LOAD_BALANCER_URL.rstrip('/')}/queue/{queue_id}"
|
url = f"{LOAD_BALANCER_URL.rstrip('/')}/queue/{queue_id}"
|
||||||
@@ -6480,60 +6292,6 @@ async def _queue_id(request: Request) -> str:
|
|||||||
return (data or {}).get("queueId", "") if isinstance(data, dict) else ""
|
return (data or {}).get("queueId", "") if isinstance(data, dict) else ""
|
||||||
|
|
||||||
|
|
||||||
async def _session_id(request: Request) -> str:
|
|
||||||
"""Pull `sessionId` from a JSON body, tolerating sendBeacon's blob posts."""
|
|
||||||
try:
|
|
||||||
data = await request.json()
|
|
||||||
except Exception:
|
|
||||||
return ""
|
|
||||||
return (data or {}).get("sessionId", "") if isinstance(data, dict) else ""
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/api/session/heartbeat")
|
|
||||||
async def session_heartbeat(request: Request):
|
|
||||||
"""Extend the live reservation one chunk at a time. `expired` once the day's
|
|
||||||
budget is spent — the client then tears down."""
|
|
||||||
client_ip = request.client.host if request.client else "?"
|
|
||||||
if not LIMITER_ENABLED:
|
|
||||||
raise HTTPException(status_code=404, detail="No encontrado.")
|
|
||||||
sid = await _session_id(request)
|
|
||||||
alive = bool(sid) and await asyncio.to_thread(limiter.heartbeat, sid)
|
|
||||||
logger.debug(
|
|
||||||
"[INTERNAL] /api/session/heartbeat client=%s sessionId=%s alive=%s",
|
|
||||||
client_ip, (sid or "")[:16], alive,
|
|
||||||
)
|
|
||||||
return {"expired": not alive}
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/api/session/end")
|
|
||||||
async def session_end(request: Request):
|
|
||||||
"""Clean teardown: reconcile to real elapsed time and refund the unused
|
|
||||||
chunk. Sent via navigator.sendBeacon, so it must succeed without a response."""
|
|
||||||
client_ip = request.client.host if request.client else "?"
|
|
||||||
if not LIMITER_ENABLED:
|
|
||||||
raise HTTPException(status_code=404, detail="No encontrado.")
|
|
||||||
sid = await _session_id(request)
|
|
||||||
logger.debug(
|
|
||||||
"[INTERNAL] /api/session/end client=%s sessionId=%s",
|
|
||||||
client_ip, (sid or "")[:16],
|
|
||||||
)
|
|
||||||
if sid:
|
|
||||||
await asyncio.to_thread(limiter.end, sid)
|
|
||||||
return {"ok": True}
|
|
||||||
|
|
||||||
|
|
||||||
@app.middleware("http")
|
|
||||||
async def no_cache_static_assets(request: Request, call_next):
|
|
||||||
response = await call_next(request)
|
|
||||||
path = request.url.path or ""
|
|
||||||
nocache_ext = (".html", ".js", ".mjs", ".css", ".wasm", ".worklet.js")
|
|
||||||
if any(path.endswith(ext) for ext in nocache_ext) or path == "/" or path.endswith("/"):
|
|
||||||
response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate, max-age=0"
|
|
||||||
response.headers["Pragma"] = "no-cache"
|
|
||||||
response.headers["Expires"] = "0"
|
|
||||||
return response
|
|
||||||
|
|
||||||
|
|
||||||
# Preserve the former proposal URLs after moving the document out of the client.
|
# Preserve the former proposal URLs after moving the document out of the client.
|
||||||
@app.get("/project", include_in_schema=False)
|
@app.get("/project", include_in_schema=False)
|
||||||
@app.get("/project/", include_in_schema=False)
|
@app.get("/project/", include_in_schema=False)
|
||||||
@@ -6545,8 +6303,3 @@ async def project_documentation():
|
|||||||
|
|
||||||
# The existing FastAPI /docs page remains registered ahead of this mount.
|
# The existing FastAPI /docs page remains registered ahead of this mount.
|
||||||
app.mount("/docs", StaticFiles(directory=os.path.join(ROOT_DIR, "docs"), html=True), name="documentation")
|
app.mount("/docs", StaticFiles(directory=os.path.join(ROOT_DIR, "docs"), html=True), name="documentation")
|
||||||
|
|
||||||
|
|
||||||
# Static front-end. Registered last so the /api routes win. `html=True` serves
|
|
||||||
# index.html at "/". The repo is public anyway, so serving the dir is fine.
|
|
||||||
app.mount("/", StaticFiles(directory=HERE, html=True), name="static")
|
|
||||||
|
|||||||
@@ -0,0 +1,272 @@
|
|||||||
|
"""Backend contract checks with local HTTP fixtures and a loopback voice server.
|
||||||
|
|
||||||
|
Run from the repository root: .venv/bin/python -m unittest discover -s s2s-client/tests
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
import unittest
|
||||||
|
from unittest.mock import ANY, AsyncMock, Mock, patch
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from websockets.sync.server import serve
|
||||||
|
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
# Never consume deployment credentials or contact services from a developer's .env.
|
||||||
|
with patch.dict(os.environ, {}, clear=True), patch("dotenv.load_dotenv", return_value=False):
|
||||||
|
import server
|
||||||
|
|
||||||
|
|
||||||
|
GRANT = {
|
||||||
|
"session_id": "local-session",
|
||||||
|
"connect_url": "ws://voice.test/v1/realtime?token=fixture",
|
||||||
|
"pending_timeout_s": 60,
|
||||||
|
}
|
||||||
|
TICKET = {"state": "queued", "queue_id": "local-ticket", "position": 2, "poll_interval_s": 1}
|
||||||
|
ASYNC_CLIENT = httpx.AsyncClient
|
||||||
|
|
||||||
|
|
||||||
|
class BackendContractTests(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.upstream = Mock(side_effect=AssertionError("Unexpected upstream HTTP request"))
|
||||||
|
self.enterContext(patch.object(server, "LOAD_BALANCER_URL", "http://lb.test"))
|
||||||
|
self.enterContext(patch.object(
|
||||||
|
server.httpx, "AsyncClient",
|
||||||
|
side_effect=lambda **kwargs: ASYNC_CLIENT(
|
||||||
|
transport=httpx.MockTransport(self.upstream), **kwargs,
|
||||||
|
),
|
||||||
|
))
|
||||||
|
self.client = self.enterContext(TestClient(server.app))
|
||||||
|
|
||||||
|
def respond(self, *responses):
|
||||||
|
self.upstream.side_effect = iter(responses)
|
||||||
|
|
||||||
|
def test_bootstrap_and_removed_account_contract(self):
|
||||||
|
response = self.client.get("/api/config")
|
||||||
|
self.assertEqual(response.status_code, 200)
|
||||||
|
self.assertTrue(response.json()["lb"])
|
||||||
|
self.assertFalse(response.json()["allowDirect"])
|
||||||
|
self.assertNotIn("auth", response.json())
|
||||||
|
paths = self.client.get("/openapi.json").json()["paths"]
|
||||||
|
for method, path in (
|
||||||
|
("GET", "/api/me"), ("POST", "/api/session/heartbeat"), ("POST", "/api/session/end"),
|
||||||
|
):
|
||||||
|
self.assertNotIn(path, paths)
|
||||||
|
self.assertEqual(self.client.request(method, path).status_code, 404)
|
||||||
|
self.assertFalse(any(getattr(route, "path", "").startswith("/oauth") for route in server.app.routes))
|
||||||
|
self.upstream.assert_not_called()
|
||||||
|
|
||||||
|
def test_only_documentation_is_served_as_static_content(self):
|
||||||
|
for path in ("/", "/index.html", "/main.js", "/style.css", "/server.py", "/requirements.txt"):
|
||||||
|
self.assertEqual(self.client.get(path).status_code, 404)
|
||||||
|
self.assertEqual(self.client.get("/docs").status_code, 200)
|
||||||
|
self.assertEqual(self.client.get("/docs/README.md").status_code, 200)
|
||||||
|
for path in ("/project/", "/public/docs.html"):
|
||||||
|
response = self.client.get(path, follow_redirects=False)
|
||||||
|
self.assertEqual(response.status_code, 307)
|
||||||
|
self.assertEqual(response.headers["location"], "/docs/proposals/vocero-architecture.html")
|
||||||
|
self.upstream.assert_not_called()
|
||||||
|
|
||||||
|
def test_session_grant_passes_through_without_extra_fields_or_cookies(self):
|
||||||
|
self.respond(httpx.Response(200, json=GRANT))
|
||||||
|
response = self.client.post("/api/session", json={})
|
||||||
|
self.assertEqual(response.status_code, 200)
|
||||||
|
self.assertEqual(response.json(), GRANT)
|
||||||
|
self.assertNotIn("set-cookie", response.headers)
|
||||||
|
request = self.upstream.call_args.args[0]
|
||||||
|
self.assertEqual((request.method, str(request.url)), ("POST", "http://lb.test/session"))
|
||||||
|
self.assertEqual(json.loads(request.content), {})
|
||||||
|
|
||||||
|
def test_queue_polling_returns_positions_and_then_grant(self):
|
||||||
|
self.respond(
|
||||||
|
httpx.Response(200, json=TICKET),
|
||||||
|
httpx.Response(200, json={**TICKET, "position": 1}),
|
||||||
|
httpx.Response(200, json=GRANT),
|
||||||
|
)
|
||||||
|
self.assertEqual(self.client.post("/api/session").json(), TICKET)
|
||||||
|
self.assertEqual(self.client.get("/api/queue/local-ticket").json(), {**TICKET, "position": 1})
|
||||||
|
response = self.client.get("/api/queue/local-ticket")
|
||||||
|
self.assertEqual(response.json(), GRANT)
|
||||||
|
self.assertNotIn("set-cookie", response.headers)
|
||||||
|
self.assertEqual(
|
||||||
|
[(call.args[0].method, call.args[0].url.path) for call in self.upstream.call_args_list],
|
||||||
|
[("POST", "/session"), ("GET", "/queue/local-ticket"), ("GET", "/queue/local-ticket")],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_capacity_and_expired_queue_responses_remain(self):
|
||||||
|
self.respond(httpx.Response(503, json={"state": "at_capacity"}), httpx.Response(404))
|
||||||
|
response = self.client.post("/api/session")
|
||||||
|
self.assertEqual((response.status_code, response.json()), (503, {"state": "at_capacity"}))
|
||||||
|
response = self.client.get("/api/queue/local-ticket")
|
||||||
|
self.assertEqual((response.status_code, response.json()), (404, {"state": "expired"}))
|
||||||
|
|
||||||
|
def test_explicit_and_beacon_queue_cancellation(self):
|
||||||
|
self.respond(httpx.Response(204), httpx.Response(204))
|
||||||
|
self.assertEqual(self.client.delete("/api/queue/local-ticket").json(), {"ok": True})
|
||||||
|
# sendBeacon posts a JSON blob, not a form or a session identifier.
|
||||||
|
response = self.client.post(
|
||||||
|
"/api/queue/end", content=json.dumps({"queueId": "local-ticket"}),
|
||||||
|
headers={"Content-Type": "application/json"},
|
||||||
|
)
|
||||||
|
self.assertEqual(response.json(), {"ok": True})
|
||||||
|
for call in self.upstream.call_args_list:
|
||||||
|
self.assertEqual((call.args[0].method, call.args[0].url.path), ("DELETE", "/queue/local-ticket"))
|
||||||
|
self.assertEqual(self.upstream.call_count, 2)
|
||||||
|
|
||||||
|
def test_empty_beacon_is_harmless_and_cancellation_is_best_effort(self):
|
||||||
|
self.assertEqual(self.client.post("/api/queue/end", content="").json(), {"ok": True})
|
||||||
|
self.upstream.assert_not_called()
|
||||||
|
self.upstream.side_effect = httpx.ConnectError("Local fixture unavailable")
|
||||||
|
self.assertEqual(self.client.delete("/api/queue/local-ticket").json(), {"ok": True})
|
||||||
|
|
||||||
|
def test_upstream_failures_are_still_reported(self):
|
||||||
|
self.upstream.side_effect = httpx.ConnectError("Local fixture unavailable")
|
||||||
|
self.assertEqual(self.client.post("/api/session").status_code, 502)
|
||||||
|
self.assertEqual(self.client.get("/api/queue/local-ticket").status_code, 502)
|
||||||
|
self.respond(httpx.Response(429), httpx.Response(500))
|
||||||
|
self.assertEqual(self.client.post("/api/session").status_code, 502)
|
||||||
|
self.assertEqual(self.client.get("/api/queue/local-ticket").status_code, 502)
|
||||||
|
|
||||||
|
def test_direct_mode_disables_lb_and_preserves_pinned_voice_url(self):
|
||||||
|
with patch.object(server, "LOAD_BALANCER_URL", ""), patch.object(
|
||||||
|
server, "SPEECH_TO_SPEECH_URL", GRANT["connect_url"],
|
||||||
|
):
|
||||||
|
config = self.client.get("/api/config").json()
|
||||||
|
self.assertEqual(config["s2sUrl"], GRANT["connect_url"])
|
||||||
|
self.assertFalse(config["lb"])
|
||||||
|
self.assertTrue(config["allowDirect"])
|
||||||
|
for method, path in (
|
||||||
|
("POST", "/api/session"), ("GET", "/api/queue/local-ticket"),
|
||||||
|
("DELETE", "/api/queue/local-ticket"), ("POST", "/api/queue/end"),
|
||||||
|
):
|
||||||
|
self.assertEqual(self.client.request(method, path).status_code, 404)
|
||||||
|
self.upstream.assert_not_called()
|
||||||
|
|
||||||
|
def test_caller_profile_still_uses_upstream_login_token(self):
|
||||||
|
profile = {"result": {"name": "Local Caller"}}
|
||||||
|
self.respond(
|
||||||
|
httpx.Response(200, json={"token": "fixture-token"}),
|
||||||
|
httpx.Response(200, json=profile),
|
||||||
|
)
|
||||||
|
with patch.object(server, "S2S_LOGIN_BY_PHONE", "http://profile.test/login"), patch.object(
|
||||||
|
server, "VOCERO_USER_PHONE_AUTH_URL", "http://profile.test/user",
|
||||||
|
):
|
||||||
|
response = self.client.get("/api/user/phone", params={"phoneNumber": "+56000000000"})
|
||||||
|
self.assertEqual(response.json(), profile)
|
||||||
|
login, lookup = [call.args[0] for call in self.upstream.call_args_list]
|
||||||
|
self.assertEqual(json.loads(login.content), {"phoneNumber": "+56000000000"})
|
||||||
|
self.assertEqual(lookup.headers["authorization"], "Bearer fixture-token")
|
||||||
|
self.assertEqual(lookup.url.params["phoneNumber"], "+56000000000")
|
||||||
|
|
||||||
|
def test_caller_auth_headers_remain_independent(self):
|
||||||
|
request = server.Request({
|
||||||
|
"type": "http", "headers": [
|
||||||
|
(b"authorization", b"Bearer fixture-token"), (b"x-demo-session", b"fixture-demo"),
|
||||||
|
],
|
||||||
|
})
|
||||||
|
self.assertEqual(server._apipilot_parse_authorization(request), ("Bearer fixture-token", "fixture-demo"))
|
||||||
|
|
||||||
|
|
||||||
|
class VoiceProxyTests(unittest.TestCase):
|
||||||
|
def test_apipilot_catalog_and_caller_defaults_reach_the_voice_session(self):
|
||||||
|
closed = threading.Event()
|
||||||
|
catalog = AsyncMock(return_value=[{
|
||||||
|
"name": "get_caller_briefing",
|
||||||
|
"description": "Retrieve the caller's local test briefing.",
|
||||||
|
"requires_auth": True,
|
||||||
|
"parameters": {"required": ["phone", "episode_id", "email"]},
|
||||||
|
}])
|
||||||
|
login = AsyncMock(return_value=("fixture-token", {}))
|
||||||
|
profile = AsyncMock(return_value={"user": {"email": "caller@example.test"}})
|
||||||
|
|
||||||
|
def voice(socket):
|
||||||
|
try:
|
||||||
|
socket.send(json.dumps({"type": "session.created"}))
|
||||||
|
for frame in socket:
|
||||||
|
socket.send(frame)
|
||||||
|
finally:
|
||||||
|
closed.set()
|
||||||
|
|
||||||
|
with serve(voice, "127.0.0.1", 0) as upstream:
|
||||||
|
thread = threading.Thread(target=upstream.serve_forever, daemon=True)
|
||||||
|
thread.start()
|
||||||
|
self.addCleanup(thread.join, 5)
|
||||||
|
target = f"ws://127.0.0.1:{upstream.socket.getsockname()[1]}/v1/realtime"
|
||||||
|
with (
|
||||||
|
patch.object(server, "S2S_PROXY_TARGET", target),
|
||||||
|
patch.object(server, "S2S_INJECT_PROFILE", False),
|
||||||
|
patch.object(server, "S2S_APIPILOT_TOOLS_URL", "http://catalog.test/tools"),
|
||||||
|
patch.object(server, "S2S_LOGIN_BY_PHONE", "http://profile.test/login"),
|
||||||
|
patch.object(server, "VOCERO_USER_PHONE_AUTH_URL", "http://profile.test/user"),
|
||||||
|
patch.object(server, "_apipilot_fetch_tools", catalog),
|
||||||
|
patch.object(server, "_login_by_phone", login),
|
||||||
|
patch.object(server, "_fetch_user_profile_for_phone", profile),
|
||||||
|
patch.object(server.httpx, "AsyncClient", side_effect=AssertionError("Unexpected HTTP request")),
|
||||||
|
TestClient(server.app) as client,
|
||||||
|
):
|
||||||
|
with client.websocket_connect(
|
||||||
|
"/v1/realtime?phoneNumber=%2B56000000000&episode_id=fixture-episode",
|
||||||
|
) as socket:
|
||||||
|
self.assertEqual(socket.receive_json(), {"type": "session.created"})
|
||||||
|
socket.send_json({"type": "session.update", "session": {"instructions": "Local fixture"}})
|
||||||
|
update = socket.receive_json()
|
||||||
|
self.assertEqual(update["type"], "session.update")
|
||||||
|
instructions = update["session"]["instructions"]
|
||||||
|
for expected in (
|
||||||
|
"Local fixture", server.APIPILOT_PASS1_LIST_MARKER, "name: get_caller_briefing",
|
||||||
|
'phone = "+56000000000"', 'episode_id = "fixture-episode"',
|
||||||
|
'email = "caller@example.test"',
|
||||||
|
):
|
||||||
|
self.assertIn(expected, instructions)
|
||||||
|
# Configuration injection must preserve ordinary audio forwarding.
|
||||||
|
audio = {"type": "input_audio_buffer.append", "audio": "AAA="}
|
||||||
|
socket.send_json(audio)
|
||||||
|
self.assertEqual(socket.receive_json(), audio)
|
||||||
|
self.assertTrue(closed.wait(3), "Client disconnect must close the upstream WebSocket")
|
||||||
|
login.assert_awaited_once_with("+56000000000")
|
||||||
|
profile.assert_awaited_once_with("+56000000000", auth_token="fixture-token")
|
||||||
|
catalog.assert_awaited_once_with(auth_header="Bearer fixture-token", demo_session=ANY, refresh=False)
|
||||||
|
|
||||||
|
def test_proxy_voice_frames_keepalive_and_client_disconnect(self):
|
||||||
|
closed = threading.Event()
|
||||||
|
received = []
|
||||||
|
keepalive = threading.Event()
|
||||||
|
|
||||||
|
def voice(socket):
|
||||||
|
try:
|
||||||
|
socket.send(json.dumps({"type": "session.created"}))
|
||||||
|
if socket.ping().wait(3):
|
||||||
|
keepalive.set()
|
||||||
|
for frame in socket:
|
||||||
|
received.append(frame)
|
||||||
|
socket.send(frame)
|
||||||
|
finally:
|
||||||
|
closed.set()
|
||||||
|
|
||||||
|
with serve(voice, "127.0.0.1", 0) as upstream:
|
||||||
|
thread = threading.Thread(target=upstream.serve_forever, daemon=True)
|
||||||
|
thread.start()
|
||||||
|
self.addCleanup(thread.join, 5)
|
||||||
|
target = f"ws://127.0.0.1:{upstream.socket.getsockname()[1]}/v1/realtime"
|
||||||
|
with patch.object(server, "S2S_PROXY_TARGET", target), TestClient(server.app) as client:
|
||||||
|
config = client.get("/api/config", headers={"host": "localhost:19420"}).json()
|
||||||
|
self.assertEqual(config["s2sUrl"], "ws://localhost:19420/v1/realtime")
|
||||||
|
with client.websocket_connect("/v1/realtime") as socket:
|
||||||
|
self.assertEqual(socket.receive_json(), {"type": "session.created"})
|
||||||
|
update = {"type": "session.update", "session": {"instructions": "Local fixture"}}
|
||||||
|
socket.send_json(update)
|
||||||
|
self.assertEqual(socket.receive_json(), update)
|
||||||
|
socket.send_bytes(b"\x00\x01\x02\x03")
|
||||||
|
self.assertEqual(socket.receive_bytes(), b"\x00\x01\x02\x03")
|
||||||
|
self.assertTrue(keepalive.wait(3))
|
||||||
|
self.assertTrue(closed.wait(3), "Client disconnect must close the upstream WebSocket")
|
||||||
|
self.assertEqual(len(received), 2)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -1,189 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* Account — the HF login chip and the daily-limit modal.
|
|
||||||
*
|
|
||||||
* Reads `/api/me` to learn the current tier (anonymous / signed-in / PRO) and
|
|
||||||
* remaining daily talk-time, renders a sign-in pill or a signed-in chip with a
|
|
||||||
* small popover (tier, remaining, sign out, upgrade), and shows the limit modal
|
|
||||||
* when a conversation is refused or cut. The time metering itself lives in
|
|
||||||
* main.js (heartbeat loop) + the server; this module is just the surface.
|
|
||||||
*
|
|
||||||
* Inert unless the deploy is in LB mode (`/api/me` → `{enabled:true}`).
|
|
||||||
*/
|
|
||||||
|
|
||||||
import { $, escHtml } from "./dom.js";
|
|
||||||
|
|
||||||
const PRO_URL = "https://huggingface.co/subscribe/pro";
|
|
||||||
|
|
||||||
// Official multi-color Hugging Face logo, used in the badge + sign-in CTA.
|
|
||||||
const HF_MARK = `<svg class="hf-logo" viewBox="0 0 95 88" fill="none" aria-hidden="true"><path fill="#FFD21E" d="M47.21 76.5a34.75 34.75 0 1 0 0-69.5 34.75 34.75 0 0 0 0 69.5Z"/><path fill="#FF9D0B" d="M81.96 41.75a34.75 34.75 0 1 0-69.5 0 34.75 34.75 0 0 0 69.5 0Zm-73.5 0a38.75 38.75 0 1 1 77.5 0 38.75 38.75 0 0 1-77.5 0Z"/><path fill="#3A3B45" d="M58.5 32.3c1.28.44 1.78 3.06 3.07 2.38a5 5 0 1 0-6.76-2.07c.61 1.15 2.55-.72 3.7-.32ZM34.95 32.3c-1.28.44-1.79 3.06-3.07 2.38a5 5 0 1 1 6.76-2.07c-.61 1.15-2.56-.72-3.7-.32Z"/><path fill="#FF323D" d="M46.96 56.29c9.83 0 13-8.76 13-13.26 0-2.34-1.57-1.6-4.09-.36-2.33 1.15-5.46 2.74-8.9 2.74-7.19 0-13-6.88-13-2.38s3.16 13.26 13 13.26Z"/><path fill="#3A3B45" fill-rule="evenodd" d="M39.43 54a8.7 8.7 0 0 1 5.3-4.49c.4-.12.81.57 1.24 1.28.4.68.82 1.37 1.24 1.37.45 0 .9-.68 1.33-1.35.45-.7.89-1.38 1.32-1.25a8.61 8.61 0 0 1 5 4.17c3.73-2.94 5.1-7.74 5.1-10.7 0-2.34-1.57-1.6-4.09-.36l-.14.07c-2.31 1.15-5.39 2.67-8.77 2.67s-6.45-1.52-8.77-2.67c-2.6-1.29-4.23-2.1-4.23.29 0 3.05 1.46 8.06 5.47 10.97Z" clip-rule="evenodd"/><path fill="#FF9D0B" d="M70.71 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM24.21 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM17.52 48c-1.62 0-3.06.66-4.07 1.87a5.97 5.97 0 0 0-1.33 3.76 7.1 7.1 0 0 0-1.94-.3c-1.55 0-2.95.59-3.94 1.66a5.8 5.8 0 0 0-.8 7 5.3 5.3 0 0 0-1.79 2.82c-.24.9-.48 2.8.8 4.74a5.22 5.22 0 0 0-.37 5.02c1.02 2.32 3.57 4.14 8.52 6.1 3.07 1.22 5.89 2 5.91 2.01a44.33 44.33 0 0 0 10.93 1.6c5.86 0 10.05-1.8 12.46-5.34 3.88-5.69 3.33-10.9-1.7-15.92-2.77-2.78-4.62-6.87-5-7.77-.78-2.66-2.84-5.62-6.25-5.62a5.7 5.7 0 0 0-4.6 2.46c-1-1.26-1.98-2.25-2.86-2.82A7.4 7.4 0 0 0 17.52 48Zm0 4c.51 0 1.14.22 1.82.65 2.14 1.36 6.25 8.43 7.76 11.18.5.92 1.37 1.31 2.14 1.31 1.55 0 2.75-1.53.15-3.48-3.92-2.93-2.55-7.72-.68-8.01.08-.02.17-.02.24-.02 1.7 0 2.45 2.93 2.45 2.93s2.2 5.52 5.98 9.3c3.77 3.77 3.97 6.8 1.22 10.83-1.88 2.75-5.47 3.58-9.16 3.58-3.81 0-7.73-.9-9.92-1.46-.11-.03-13.45-3.8-11.76-7 .28-.54.75-.76 1.34-.76 2.38 0 6.7 3.54 8.57 3.54.41 0 .7-.17.83-.6.79-2.85-12.06-4.05-10.98-8.17.2-.73.71-1.02 1.44-1.02 3.14 0 10.2 5.53 11.68 5.53.11 0 .2-.03.24-.1.74-1.2.33-2.04-4.9-5.2-5.21-3.16-8.88-5.06-6.8-7.33.24-.26.58-.38 1-.38 3.17 0 10.66 6.82 10.66 6.82s2.02 2.1 3.25 2.1c.28 0 .52-.1.68-.38.86-1.46-8.06-8.22-8.56-11.01-.34-1.9.24-2.85 1.31-2.85Z"/><path fill="#FFD21E" d="M38.6 76.69c2.75-4.04 2.55-7.07-1.22-10.84-3.78-3.77-5.98-9.3-5.98-9.3s-.82-3.2-2.69-2.9c-1.87.3-3.24 5.08.68 8.01 3.91 2.93-.78 4.92-2.29 2.17-1.5-2.75-5.62-9.82-7.76-11.18-2.13-1.35-3.63-.6-3.13 2.2.5 2.79 9.43 9.55 8.56 11-.87 1.47-3.93-1.71-3.93-1.71s-9.57-8.71-11.66-6.44c-2.08 2.27 1.59 4.17 6.8 7.33 5.23 3.16 5.64 4 4.9 5.2-.75 1.2-12.28-8.53-13.36-4.4-1.08 4.11 11.77 5.3 10.98 8.15-.8 2.85-9.06-5.38-10.74-2.18-1.7 3.21 11.65 6.98 11.76 7.01 4.3 1.12 15.25 3.49 19.08-2.12Z"/><path fill="#FF9D0B" d="M77.4 48c1.62 0 3.07.66 4.07 1.87a5.97 5.97 0 0 1 1.33 3.76 7.1 7.1 0 0 1 1.95-.3c1.55 0 2.95.59 3.94 1.66a5.8 5.8 0 0 1 .8 7 5.3 5.3 0 0 1 1.78 2.82c.24.9.48 2.8-.8 4.74a5.22 5.22 0 0 1 .37 5.02c-1.02 2.32-3.57 4.14-8.51 6.1-3.08 1.22-5.9 2-5.92 2.01a44.33 44.33 0 0 1-10.93 1.6c-5.86 0-10.05-1.8-12.46-5.34-3.88-5.69-3.33-10.9 1.7-15.92 2.78-2.78 4.63-6.87 5.01-7.77.78-2.66 2.83-5.62 6.24-5.62a5.7 5.7 0 0 1 4.6 2.46c1-1.26 1.98-2.25 2.87-2.82A7.4 7.4 0 0 1 77.4 48Zm0 4c-.51 0-1.13.22-1.82.65-2.13 1.36-6.25 8.43-7.76 11.18a2.43 2.43 0 0 1-2.14 1.31c-1.54 0-2.75-1.53-.14-3.48 3.91-2.93 2.54-7.72.67-8.01a1.54 1.54 0 0 0-.24-.02c-1.7 0-2.45 2.93-2.45 2.93s-2.2 5.52-5.97 9.3c-3.78 3.77-3.98 6.8-1.22 10.83 1.87 2.75 5.47 3.58 9.15 3.58 3.82 0 7.73-.9 9.93-1.46.1-.03 13.45-3.8 11.76-7-.29-.54-.75-.76-1.34-.76-2.38 0-6.71 3.54-8.57 3.54-.42 0-.71-.17-.83-.6-.8-2.85 12.05-4.05 10.97-8.17-.19-.73-.7-1.02-1.44-1.02-3.14 0-10.2 5.53-11.68 5.53-.1 0-.19-.03-.23-.1-.74-1.2-.34-2.04 4.88-5.2 5.23-3.16 8.9-5.06 6.8-7.33-.23-.26-.57-.38-.98-.38-3.18 0-10.67 6.82-10.67 6.82s-2.02 2.1-3.24 2.1a.74.74 0 0 1-.68-.38c-.87-1.46 8.05-8.22 8.55-11.01.34-1.9-.24-2.85-1.31-2.85Z"/><path fill="#FFD21E" d="M56.33 76.69c-2.75-4.04-2.56-7.07 1.22-10.84 3.77-3.77 5.97-9.3 5.97-9.3s.82-3.2 2.7-2.9c1.86.3 3.23 5.08-.68 8.01-3.92 2.93.78 4.92 2.28 2.17 1.51-2.75 5.63-9.82 7.76-11.18 2.13-1.35 3.64-.6 3.13 2.2-.5 2.79-9.42 9.55-8.55 11 .86 1.47 3.92-1.71 3.92-1.71s9.58-8.71 11.66-6.44c2.08 2.27-1.58 4.17-6.8 7.33-5.23 3.16-5.63 4-4.9 5.2.75 1.2 12.28-8.53 13.36-4.4 1.08 4.11-11.76 5.3-10.97 8.15.8 2.85 9.05-5.38 10.74-2.18 1.69 3.21-11.65 6.98-11.76 7.01-4.31 1.12-15.26 3.49-19.08-2.12Z"/></svg>`;
|
|
||||||
|
|
||||||
/** @param {number} sec @returns {string} "m:ss" */
|
|
||||||
function fmt(sec) {
|
|
||||||
const s = Math.max(0, Math.round(sec));
|
|
||||||
return `${Math.floor(s / 60)}:${String(s % 60).padStart(2, "0")}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
export class Account {
|
|
||||||
constructor() {
|
|
||||||
/** @type {HTMLElement} */
|
|
||||||
this._root = $("#account");
|
|
||||||
/** @type {HTMLDialogElement} */
|
|
||||||
this._modal = $("#limit-modal");
|
|
||||||
this._modalTitle = $("#limit-title");
|
|
||||||
this._modalMsg = $("#limit-msg");
|
|
||||||
this._modalNote = $("#limit-note");
|
|
||||||
/** @type {HTMLAnchorElement} */
|
|
||||||
this._modalCta = /** @type {any} */ ($("#limit-cta"));
|
|
||||||
|
|
||||||
/** @type {{enabled:boolean, auth?:boolean, loggedIn?:boolean, username?:string, avatar?:string, tier?:string, remainingSec?:number|null, limitSec?:number|null, loginUrl?:string|null, logoutUrl?:string|null}} */
|
|
||||||
this._me = { enabled: false };
|
|
||||||
this._popoverOpen = false;
|
|
||||||
|
|
||||||
$("#limit-close").addEventListener("click", () => this._modal.close());
|
|
||||||
this._modal.addEventListener("click", (e) => {
|
|
||||||
if (e.target === this._modal) this._modal.close();
|
|
||||||
});
|
|
||||||
// Close the popover on an outside click.
|
|
||||||
document.addEventListener("click", (e) => {
|
|
||||||
if (this._popoverOpen && !this._root.contains(/** @type {Node} */ (e.target))) {
|
|
||||||
this._closePopover();
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
get tier() {
|
|
||||||
return this._me.tier || "anon";
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Fetch `/api/me` and (re)render the chip. Safe to call repeatedly (load,
|
|
||||||
* after the OAuth redirect, after a conversation ends). */
|
|
||||||
async refresh() {
|
|
||||||
try {
|
|
||||||
const res = await fetch("api/me");
|
|
||||||
this._me = res.ok ? await res.json() : { enabled: false };
|
|
||||||
} catch {
|
|
||||||
this._me = { enabled: false };
|
|
||||||
}
|
|
||||||
this._render();
|
|
||||||
}
|
|
||||||
|
|
||||||
_render() {
|
|
||||||
const me = this._me;
|
|
||||||
if (!me.enabled) {
|
|
||||||
this._root.hidden = true;
|
|
||||||
this._root.innerHTML = "";
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
this._root.hidden = false;
|
|
||||||
|
|
||||||
if (!me.loggedIn) {
|
|
||||||
// Signed-out: a sign-in pill (only when OAuth is actually available).
|
|
||||||
if (me.auth && me.loginUrl) {
|
|
||||||
this._root.innerHTML = `<a class="signin-pill" href="${escHtml(me.loginUrl)}" title="Inicia sesión para obtener más tiempo"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M15 3h4a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2h-4"/><polyline points="10 17 15 12 10 7"/><line x1="15" y1="12" x2="3" y2="12"/></svg><span>Iniciar sesión</span></a>`;
|
|
||||||
} else {
|
|
||||||
this._root.innerHTML = "";
|
|
||||||
this._root.hidden = true;
|
|
||||||
}
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Signed-in: avatar + handle chip that toggles a popover.
|
|
||||||
const isPro = me.tier === "pro";
|
|
||||||
// Org members get unlimited usage too, but aren't PRO — don't brand them so.
|
|
||||||
const isUnlimited = isPro || me.tier === "org";
|
|
||||||
const avatar = me.avatar
|
|
||||||
? `<img class="account-avatar" src="${escHtml(me.avatar)}" alt="" />`
|
|
||||||
: `<span class="account-avatar account-avatar-fallback">${escHtml((me.username || "?")[0].toUpperCase())}</span>`;
|
|
||||||
const remaining =
|
|
||||||
isUnlimited || me.remainingSec == null
|
|
||||||
? "Sin límite"
|
|
||||||
: `${fmt(me.remainingSec)} restantes hoy`;
|
|
||||||
const tierLabel = isPro ? "PRO" : isUnlimited ? "Equipo" : "Gratis";
|
|
||||||
|
|
||||||
this._root.innerHTML = `
|
|
||||||
<button id="account-chip" class="account-chip" aria-haspopup="true" aria-expanded="false">
|
|
||||||
${avatar}
|
|
||||||
<span class="account-handle">${escHtml(me.username || "tú")}</span>
|
|
||||||
${isPro
|
|
||||||
? '<span class="account-pro">PRO</span>'
|
|
||||||
: me.tier === "org"
|
|
||||||
? '<span class="account-pro account-team">EQUIPO</span>'
|
|
||||||
: ""}
|
|
||||||
</button>
|
|
||||||
<div id="account-pop" class="account-pop" hidden>
|
|
||||||
<div class="account-pop-row account-pop-name">${escHtml(me.username || "tú")}</div>
|
|
||||||
<div class="account-pop-row account-pop-meta">
|
|
||||||
<span class="account-tier">${tierLabel}</span>
|
|
||||||
<span class="account-remaining">${escHtml(remaining)}</span>
|
|
||||||
</div>
|
|
||||||
${isUnlimited ? "" : `<a class="account-pop-link" href="${PRO_URL}" target="_blank" rel="noopener">Pasar a PRO</a>`}
|
|
||||||
<a class="account-pop-link account-signout" href="${escHtml(me.logoutUrl || "#")}">Cerrar sesión</a>
|
|
||||||
</div>`;
|
|
||||||
|
|
||||||
const chip = $("#account-chip");
|
|
||||||
chip.addEventListener("click", (e) => {
|
|
||||||
e.stopPropagation();
|
|
||||||
this._popoverOpen ? this._closePopover() : this._openPopover();
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
_openPopover() {
|
|
||||||
const pop = document.getElementById("account-pop");
|
|
||||||
const chip = document.getElementById("account-chip");
|
|
||||||
if (!pop || !chip) return;
|
|
||||||
pop.hidden = false;
|
|
||||||
chip.setAttribute("aria-expanded", "true");
|
|
||||||
this._popoverOpen = true;
|
|
||||||
}
|
|
||||||
|
|
||||||
_closePopover() {
|
|
||||||
const pop = document.getElementById("account-pop");
|
|
||||||
const chip = document.getElementById("account-chip");
|
|
||||||
if (pop) pop.hidden = true;
|
|
||||||
if (chip) chip.setAttribute("aria-expanded", "false");
|
|
||||||
this._popoverOpen = false;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Show the limit modal for a tier — used both when a conversation is refused
|
|
||||||
* at start (402) and when a live one is cut (heartbeat `expired`).
|
|
||||||
* @param {string} [tier]
|
|
||||||
*/
|
|
||||||
showLimit(tier = this.tier) {
|
|
||||||
const canSignIn = this._me.auth && this._me.loginUrl;
|
|
||||||
this._modalTitle.textContent = "¡Gracias por conversar!";
|
|
||||||
if (tier === "anon") {
|
|
||||||
this._modalMsg.textContent =
|
|
||||||
"Las conversaciones de invitados duran 5 minutos. Inicia sesión con Hugging Face para obtener 10 minutos gratis al día. Los miembros PRO pueden conversar sin límite.";
|
|
||||||
this._modalNote.textContent = "Tus minutos gratis se renuevan mañana.";
|
|
||||||
if (canSignIn) {
|
|
||||||
this._modalCta.innerHTML = `${HF_MARK}<span>Iniciar sesión con Hugging Face</span>`;
|
|
||||||
this._modalCta.href = /** @type {string} */ (this._me.loginUrl);
|
|
||||||
this._modalCta.hidden = false;
|
|
||||||
} else {
|
|
||||||
this._modalCta.hidden = true;
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
// Signed-in, non-PRO.
|
|
||||||
this._modalMsg.textContent =
|
|
||||||
"Ya has usado tus 10 minutos de hoy. Pasa a PRO para conversar sin límite y apoyar la IA de código abierto.";
|
|
||||||
this._modalNote.textContent = "También puedes volver mañana. Tus minutos se renuevan cada día.";
|
|
||||||
this._modalCta.innerHTML = "<span>Pasar a PRO</span>";
|
|
||||||
this._modalCta.href = PRO_URL;
|
|
||||||
this._modalCta.hidden = false;
|
|
||||||
}
|
|
||||||
if (!this._modal.open) this._modal.showModal();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Show a warm "we're at capacity" message when even the waiting line is full.
|
|
||||||
* Reuses the limit modal shell; no call-to-action, just reassurance. */
|
|
||||||
showBusy() {
|
|
||||||
this._modalTitle.textContent = "¡Estamos al límite de abrazos! 🤗";
|
|
||||||
this._modalMsg.textContent =
|
|
||||||
"Todos los lugares y la cola están ocupados. ¡Demasiado cariño! Tómate un café y vuelve en un minuto.";
|
|
||||||
this._modalNote.textContent = "Normalmente se libera un lugar en pocos minutos.";
|
|
||||||
this._modalCta.hidden = true;
|
|
||||||
if (!this._modal.open) this._modal.showModal();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,835 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* ChatView — owns the whole conversation surface: the slide-in history panel,
|
|
||||||
* the ephemeral on-orb bubbles, and all the transcript/tool/streaming
|
|
||||||
* bookkeeping. main.js wires the realtime client's events straight to the
|
|
||||||
* `on*` methods here and otherwise doesn't touch chat state.
|
|
||||||
*
|
|
||||||
* Two parallel surfaces share one shape (see `_buildMessageEl`):
|
|
||||||
* - ephemeral bubbles (`.bubble` / `.bubble-*`) fade on a timer
|
|
||||||
* - persistent history (`.hist-msg` / `.hist-*`) the durable panel log
|
|
||||||
*
|
|
||||||
* Keying:
|
|
||||||
* - user transcripts by the server's `item_id` — a speculative continuation
|
|
||||||
* REUSES it, so both segments land in one row/bubble; deltas are CUMULATIVE
|
|
||||||
* (each carries the full sentence so far), so we replace text wholesale.
|
|
||||||
* - assistant transcripts by `response_id`, so a cancelled speculative reply
|
|
||||||
* can be marked interrupted without erasing what was already shown.
|
|
||||||
*/
|
|
||||||
|
|
||||||
import { $, escHtml, DEBUG } from "./dom.js";
|
|
||||||
import { renderMarkdown } from "./markdown.js";
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @param {string} hypothesisId
|
|
||||||
* @param {string} location
|
|
||||||
* @param {string} msg
|
|
||||||
* @param {Record<string, unknown>} [data]
|
|
||||||
*/
|
|
||||||
function reportChatDebug(hypothesisId, location, msg, data = {}) {
|
|
||||||
void hypothesisId;
|
|
||||||
void location;
|
|
||||||
void msg;
|
|
||||||
void data;
|
|
||||||
}
|
|
||||||
|
|
||||||
const WRENCH_PATH = `<path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/>`;
|
|
||||||
const CHAT_BUBBLE_SVG = `<svg width="28" height="28" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/></svg>`;
|
|
||||||
const EMPTY_STATE_HTML = `<div id="chat-empty" class="chat-empty">${CHAT_BUBBLE_SVG}<span class="chat-empty-title">Aún no hay mensajes</span><span class="chat-empty-hint">Toca el círculo y empieza a hablar</span></div>`;
|
|
||||||
/**
|
|
||||||
* @param {string} prefix
|
|
||||||
* @param {string} [sep]
|
|
||||||
* @returns {string} safe filename prefix (a-z 0-9 _ - no spaces / slashes / unicode),
|
|
||||||
* sanitised so any OS accepts it.
|
|
||||||
*/
|
|
||||||
function _safeName(prefix, sep = "-") {
|
|
||||||
const base = String(prefix || "").toLowerCase().trim()
|
|
||||||
.normalize("NFKD").replace(/[^\x20-\x7E]/g, "")
|
|
||||||
.replace(/[^a-z0-9]+/gi, sep || "-")
|
|
||||||
.replace(new RegExp(`^${sep}+|${sep}+$`, "g"), "");
|
|
||||||
return base || "vocero";
|
|
||||||
}
|
|
||||||
|
|
||||||
export class ChatView {
|
|
||||||
constructor() {
|
|
||||||
// In the V2 layout the chat panel lives inline in #app, shown permanently
|
|
||||||
// below the orb — there is NO toggle button, close button, backdrop, or
|
|
||||||
// unread badge any more. The strict $() helper THROWS when the selector is
|
|
||||||
// missing, so for the optional overlay-era refs we MUST use plain
|
|
||||||
// document.querySelector (returns null when absent). Only the
|
|
||||||
// always-present inline refs go through $().
|
|
||||||
/** @type {HTMLButtonElement | null} */
|
|
||||||
this._chatBtn = /** @type {HTMLButtonElement | null} */ (document.querySelector("#chat-btn"));
|
|
||||||
/** @type {HTMLSpanElement | null} */
|
|
||||||
this._chatBadge = /** @type {HTMLSpanElement | null} */ (document.querySelector("#chat-badge"));
|
|
||||||
/** @type {HTMLElement} */
|
|
||||||
this._chatPanel = $("#chat-panel");
|
|
||||||
/** @type {HTMLElement | null} */
|
|
||||||
this._chatPanelBackdrop = /** @type {HTMLElement | null} */ (document.querySelector("#chat-panel-backdrop"));
|
|
||||||
/** @type {HTMLButtonElement | null} */
|
|
||||||
this._chatPanelClose = /** @type {HTMLButtonElement | null} */ (document.querySelector("#chat-panel-close"));
|
|
||||||
/** @type {HTMLElement} */
|
|
||||||
this._chatHistory = $("#chat-history");
|
|
||||||
/** @type {HTMLElement} */
|
|
||||||
this._bubbleStack = $("#bubble-stack");
|
|
||||||
/** @type {HTMLButtonElement | null} Export conversation download button in header. */
|
|
||||||
this._exportBtn = /** @type {HTMLButtonElement | null} */ (document.getElementById("chat-export-btn"));
|
|
||||||
if (this._exportBtn) {
|
|
||||||
this._exportBtn.addEventListener("click", () => this.exportConversation());
|
|
||||||
}
|
|
||||||
|
|
||||||
// Inline chat is always "open" (permanently visible on screen).
|
|
||||||
this._panelOpen = true;
|
|
||||||
// ── Always-bottom scroll bookkeeping ─────────────────────────────────
|
|
||||||
// Active timer for 4-step retry chain (0/30/100/250ms snap-to-bottom).
|
|
||||||
/** @type {number | null} */
|
|
||||||
this._scrollTimerId = null;
|
|
||||||
// Current step index inside a retry chain (0..3).
|
|
||||||
this._scrollStep = 0;
|
|
||||||
|
|
||||||
// ── User transcript state (keyed by item_id) ───────────────────────────
|
|
||||||
/** @type {Map<string, HTMLElement>} */
|
|
||||||
this._userHistByItem = new Map();
|
|
||||||
/** @type {HTMLElement | null} */
|
|
||||||
this._activeUserBubble = null;
|
|
||||||
this._activeUserItemId = "";
|
|
||||||
// Monotonic counter for synthesizing unique keys when the server omits an
|
|
||||||
// item_id / response_id, so id-less messages never collapse onto each other.
|
|
||||||
this._anonSeq = 0;
|
|
||||||
|
|
||||||
// ── Assistant transcript state (keyed by response_id) ──────────────────
|
|
||||||
/** @type {Map<string, { bubble: HTMLElement, hist: HTMLElement }>} */
|
|
||||||
this._asstByResp = new Map();
|
|
||||||
/** Original transcript text is retained independently of rendered markup. @type {WeakMap<HTMLElement, string>} */
|
|
||||||
this._messageText = new WeakMap();
|
|
||||||
|
|
||||||
// ── Ephemeral bubble auto-dismiss ──────────────────────────────────────
|
|
||||||
// Per-element expiry (epoch ms). A bubble fades once its expiry passes —
|
|
||||||
// but only in stack order (see _reapBubbles). Refreshing the expiry keeps a
|
|
||||||
// bubble alive while it updates (e.g. the live user utterance).
|
|
||||||
/** @type {WeakMap<HTMLElement, number>} */
|
|
||||||
this._bubbleExpiry = new WeakMap();
|
|
||||||
// Single pending reaper handle: one timer for the whole stack (not one per
|
|
||||||
// bubble) so dismissal is strictly oldest-first regardless of per-bubble delays.
|
|
||||||
this._reaperHandle = 0;
|
|
||||||
|
|
||||||
// ── New inline-chat handlers: only wire when old overlay buttons exist.
|
|
||||||
if (this._chatBtn) this._chatBtn.addEventListener("click", () => (this._panelOpen ? this._closePanel() : this._openPanel()));
|
|
||||||
if (this._chatPanelClose) this._chatPanelClose.addEventListener("click", () => this._closePanel());
|
|
||||||
if (this._chatPanelBackdrop) this._chatPanelBackdrop.addEventListener("click", () => this._closePanel());
|
|
||||||
document.addEventListener("keydown", (e) => {
|
|
||||||
// Old behaviour (ESC closes overlay) is a no-op on the inline permanent
|
|
||||||
// panel; only execute it when a close button actually exists.
|
|
||||||
if (e.key === "Escape" && this._chatPanelClose && this._panelOpen) this._closePanel();
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── User name / initials helpers ────────────────────────────────────────
|
|
||||||
/** @returns {string} display name for the current user (e.g. "Alessandro").
|
|
||||||
* Reuses extractVoceroUserName + initialsForName exposed on window by main.js
|
|
||||||
* so we don't duplicate the nested-object lookup logic (profile/user/customer
|
|
||||||
* keys, first+last name fallback, etc.). Falls back to "Usuario" (Spanish)
|
|
||||||
* when no profile has been resolved yet — never returns "" because the
|
|
||||||
* white-theme chat label is rendered into a pseudo-element that would show
|
|
||||||
* a lonely timestamp otherwise.
|
|
||||||
*/
|
|
||||||
_resolveUserName() {
|
|
||||||
const ex = typeof window._s2sExtractName === "function" ? window._s2sExtractName : null;
|
|
||||||
const profile =
|
|
||||||
/** @type {Record<string, unknown> | null} */ (/** @type {unknown} */ (window.voceroUserProfile)) || null;
|
|
||||||
const full = ex ? ex(profile) : "";
|
|
||||||
if (!full) return "Usuario";
|
|
||||||
// Return ONLY the first token so the narrow chat label line fits:
|
|
||||||
// "Alessandro Valenti" → "Alessandro", matches the screenshot style.
|
|
||||||
const first = String(full).trim().split(/\s+/)[0] || full;
|
|
||||||
return first;
|
|
||||||
}
|
|
||||||
/** @returns {string} 2-letter uppercase initials, e.g. "Alessandro Valenti" → "AV". */
|
|
||||||
_resolveUserInitials() {
|
|
||||||
const mk = typeof window._s2sInitialsFor === "function" ? window._s2sInitialsFor : null;
|
|
||||||
const profile =
|
|
||||||
/** @type {Record<string, unknown> | null} */ (/** @type {unknown} */ (window.voceroUserProfile)) || null;
|
|
||||||
const ex = typeof window._s2sExtractName === "function" ? window._s2sExtractName : null;
|
|
||||||
const full = ex ? ex(profile) : "";
|
|
||||||
if (mk && full) {
|
|
||||||
const init = mk(full);
|
|
||||||
if (init) return init;
|
|
||||||
}
|
|
||||||
return "AV";
|
|
||||||
}
|
|
||||||
/** @returns {string} "HH:MM" 24h using the current locale, no seconds. */
|
|
||||||
_nowStamp() {
|
|
||||||
try {
|
|
||||||
return new Date().toLocaleTimeString([], { hour: "2-digit", minute: "2-digit" });
|
|
||||||
} catch (_) {
|
|
||||||
const d = new Date();
|
|
||||||
const hh = String(d.getHours()).padStart(2, "0");
|
|
||||||
const mm = String(d.getMinutes()).padStart(2, "0");
|
|
||||||
return `${hh}:${mm}`;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/** Assistant display name — fixed Spanish label, same as the topbar brand. */
|
|
||||||
get _assistantName() {
|
|
||||||
return "Vocero";
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Panel ───────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
_openPanel() {
|
|
||||||
this._panelOpen = true;
|
|
||||||
if (this._chatPanel) this._chatPanel.classList.add("open");
|
|
||||||
if (this._chatBadge) this._chatBadge.classList.remove("visible");
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
_closePanel() {
|
|
||||||
// Closing disabled on the new inline-permanent layout; guard so if the
|
|
||||||
// overlay UI is still in use by another variant it continues to work.
|
|
||||||
if (!this._chatPanelClose) return;
|
|
||||||
this._panelOpen = false;
|
|
||||||
if (this._chatPanel) this._chatPanel.classList.remove("open");
|
|
||||||
}
|
|
||||||
|
|
||||||
// Scroll the chat history to place the LAST message's FINAL LINE fully above
|
|
||||||
// the sticky footer bar.
|
|
||||||
//
|
|
||||||
// DESIGN NOTES (2026-09-04, after user report "scroll non scrolla
|
|
||||||
// sufficientemente"):
|
|
||||||
// 1. NO early-return on "already queued". The previous coalescing flag was
|
|
||||||
// the root cause of tail-clipping: the 1st incoming delta would queue a
|
|
||||||
// rAF, every subsequent delta (up to ~30/frame) was SKIPPED, then the
|
|
||||||
// rAF ran against the scrollHeight of the 1st delta only — so the tail
|
|
||||||
// words appended later were never accounted for.
|
|
||||||
// 2. Instead we CANCEL any pending timer chain and RESCHEDULE from scratch
|
|
||||||
// every single time `_scrollToBottom()` is called. This means the very
|
|
||||||
// last append/update call always "wins" the chain — so we scroll
|
|
||||||
// against the final grown content, not an intermediate snapshot.
|
|
||||||
// 3. We run 4 scroll attempts staggered at [0ms, 30ms, 100ms, 250ms].
|
|
||||||
// Layout is async: line-wrap, font metrics, the scroll anchoring reset,
|
|
||||||
// and tool-row height expansions all resolve in later micro/task frames.
|
|
||||||
// Attempt 0 = immediate post-append; 30 = after one layout-pass; 100 =
|
|
||||||
// after any Chrome scroll-anchor reflow; 250 = after React-like deferred
|
|
||||||
// re-renders (if any). At 250ms we are guaranteed stable scrollHeight.
|
|
||||||
// 4. Instead of scrollTop = scrollHeight we use scrollTo({top: 9999999}).
|
|
||||||
// The browser clamps automatically; using a giant literal avoids any
|
|
||||||
// race where scrollHeight was read before a layout commit propagated.
|
|
||||||
_scrollToBottom() {
|
|
||||||
if (!this._chatHistory) return;
|
|
||||||
const h = this._chatHistory;
|
|
||||||
// Cancel any in-flight attempt chain and start fresh — guarantees the LAST
|
|
||||||
// caller wins (so if 100 tokens arrive in 1 frame we don't run 100 chains,
|
|
||||||
// we only run ONE chain: the one scheduled from the 100th token append).
|
|
||||||
if (this._scrollTimerId != null) {
|
|
||||||
clearTimeout(this._scrollTimerId);
|
|
||||||
this._scrollTimerId = null;
|
|
||||||
}
|
|
||||||
// Also reset any per-chain tracking so we re-run all 4 steps from the new
|
|
||||||
// point-in-time content height.
|
|
||||||
this._scrollStep = 0;
|
|
||||||
|
|
||||||
const snap = () => {
|
|
||||||
// Use Number.MAX_SAFE_INTEGER: browsers clamp to the real max
|
|
||||||
// (scrollHeight - clientHeight) automatically. This avoids ANY race
|
|
||||||
// where we read scrollHeight at an intermediate layout commit, then
|
|
||||||
// text wraps to more lines later and the scroll delta is short by a
|
|
||||||
// few pixels (the exact bug reported by the user where the last line
|
|
||||||
// was ½ visible behind the floating footer bar).
|
|
||||||
const BIG = Number.MAX_SAFE_INTEGER;
|
|
||||||
try { h.scrollTo({ top: BIG, behavior: "instant" }); }
|
|
||||||
catch (_) {
|
|
||||||
// Fallback for browsers without instant behavior option support
|
|
||||||
h.scrollTop = BIG;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
const step = () => {
|
|
||||||
snap();
|
|
||||||
this._scrollStep++;
|
|
||||||
// 5 attempts, staggered: next delay in ms per step index
|
|
||||||
// step 0 (already run above): immediate
|
|
||||||
// step 1: +30ms after one layout pass
|
|
||||||
// step 2: +70ms after line-wrap reflow (most-common "one word left on
|
|
||||||
// line 7" tail-clipping gets fixed here)
|
|
||||||
// step 3: +150ms after Chrome scroll-anchor reflow
|
|
||||||
// step 4: +250ms final after deferred tool-row / image layout
|
|
||||||
// stop: step 5 reached → exit.
|
|
||||||
const nextDelay = [30, 70, 150, 250][this._scrollStep];
|
|
||||||
if (nextDelay != null) {
|
|
||||||
this._scrollTimerId = setTimeout(step, nextDelay);
|
|
||||||
} else {
|
|
||||||
// Step 4 done → release guard. The id is kept so a new incoming call
|
|
||||||
// can cancel a mid-flight chain cleanly.
|
|
||||||
this._scrollTimerId = null;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Attempt 0 — IMMEDIATELY (same microtask as append — catches 90% of cases).
|
|
||||||
step();
|
|
||||||
}
|
|
||||||
|
|
||||||
_markUnread() {
|
|
||||||
// New inline layout: the chat is always visible so no "unread badge" UX.
|
|
||||||
// If we're running the old overlay variant we still show it.
|
|
||||||
if (this._panelOpen) {
|
|
||||||
this._scrollToBottom();
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
if (this._chatBadge) this._chatBadge.classList.add("visible");
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Shared rendering ──────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Build a role-labelled message element. Ephemeral bubbles and persistent
|
|
||||||
* history rows share the same shape and differ only in their class prefix
|
|
||||||
* (`bubble`/`bubble-*` vs `hist-msg`/`hist-*`).
|
|
||||||
*
|
|
||||||
* White-theme label + avatar data attributes (populated on both the outer
|
|
||||||
* wrapper and the inner body):
|
|
||||||
* wrapper .{hist-msg|bubble}.user → data-user-name / data-user-initials / data-timestamp
|
|
||||||
* wrapper .{hist-msg|bubble}.assistant → data-asst-name / data-timestamp
|
|
||||||
* inner .{hist|bubble}-body → data-user-name | data-asst-name + data-timestamp
|
|
||||||
*
|
|
||||||
* The .{hist|bubble}-body::before pseudo-element in style.css uses attr()
|
|
||||||
* to render the "Name HH:MM" label aligned to the bubble corner; the
|
|
||||||
* wrapper .user::before pseudo uses attr(data-user-initials) to paint the
|
|
||||||
* circular avatar's 2-letter monogram.
|
|
||||||
*
|
|
||||||
* @param {{ container: string, prefix: string, role: "user"|"assistant", text: string, partial?: boolean }} o
|
|
||||||
* @returns {HTMLElement}
|
|
||||||
*/
|
|
||||||
_buildMessageEl({ container, prefix, role, text, partial = false }) {
|
|
||||||
const el = document.createElement("div");
|
|
||||||
el.className = `${container} ${role}`;
|
|
||||||
const label = role === "user" ? "Tú" : "Asistente";
|
|
||||||
const stamp = this._nowStamp();
|
|
||||||
if (role === "user") {
|
|
||||||
const uName = this._resolveUserName();
|
|
||||||
const uInit = this._resolveUserInitials();
|
|
||||||
el.setAttribute("data-user-name", uName);
|
|
||||||
el.setAttribute("data-user-initials", uInit);
|
|
||||||
el.setAttribute("data-timestamp", stamp);
|
|
||||||
} else {
|
|
||||||
el.setAttribute("data-asst-name", this._assistantName);
|
|
||||||
el.setAttribute("data-timestamp", stamp);
|
|
||||||
}
|
|
||||||
el.innerHTML = `<div class="${prefix}-role">${label}</div><div class="${prefix}-body${partial ? " partial" : ""}"></div>`;
|
|
||||||
const body = /** @type {HTMLDivElement} */ (el.lastElementChild);
|
|
||||||
if (body) {
|
|
||||||
this._renderMessageBody(body, text, role === "assistant");
|
|
||||||
if (role === "user") {
|
|
||||||
body.setAttribute("data-user-name", el.getAttribute("data-user-name") || "");
|
|
||||||
} else {
|
|
||||||
body.setAttribute("data-asst-name", this._assistantName);
|
|
||||||
}
|
|
||||||
body.setAttribute("data-timestamp", stamp);
|
|
||||||
}
|
|
||||||
return el;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {HTMLElement} body @param {string} text @param {boolean} assistant */
|
|
||||||
_renderMessageBody(body, text, assistant) {
|
|
||||||
this._messageText.set(body, text);
|
|
||||||
body.classList.toggle("markdown-content", assistant);
|
|
||||||
if (assistant) body.replaceChildren(renderMarkdown(text));
|
|
||||||
else body.textContent = text;
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Ephemeral bubbles ───────────────────────────────────────────────────
|
|
||||||
|
|
||||||
/** @param {"user"|"assistant"|"tool"} role @param {string} text @returns {HTMLElement} */
|
|
||||||
_spawnBubble(role, text) {
|
|
||||||
let el;
|
|
||||||
if (role === "tool") {
|
|
||||||
el = document.createElement("div");
|
|
||||||
el.className = "bubble tool";
|
|
||||||
el.innerHTML = `<svg class="bubble-tool-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">${WRENCH_PATH}</svg><span class="bubble-tool-text">${escHtml(text)}</span>`;
|
|
||||||
} else {
|
|
||||||
el = this._buildMessageEl({ container: "bubble", prefix: "bubble", role, text });
|
|
||||||
}
|
|
||||||
this._bubbleStack.appendChild(el);
|
|
||||||
// Cap the stack at 3, but never evict the bubble the caller is still
|
|
||||||
// actively updating (the live user bubble) — drop the next-oldest instead.
|
|
||||||
const visible = /** @type {HTMLElement[]} */ ([...this._bubbleStack.querySelectorAll(".bubble:not(.out)")]);
|
|
||||||
if (visible.length > 3) {
|
|
||||||
this._dismissBubble(visible.find((b) => b !== this._activeUserBubble) ?? visible[0]);
|
|
||||||
}
|
|
||||||
requestAnimationFrame(() => el.classList.add("in"));
|
|
||||||
return el;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {HTMLElement} el @param {string} text */
|
|
||||||
_updateBubbleText(el, text) {
|
|
||||||
const t = /** @type {HTMLElement | null} */ (el.querySelector(".bubble-body"));
|
|
||||||
if (t) this._renderMessageBody(t, text, el.classList.contains("assistant"));
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {HTMLElement} el */
|
|
||||||
_dismissBubble(el) {
|
|
||||||
if (!el || el.classList.contains("out")) return; // idempotent
|
|
||||||
this._bubbleExpiry.delete(el);
|
|
||||||
el.classList.remove("in");
|
|
||||||
el.classList.add("out");
|
|
||||||
const remove = () => el.remove();
|
|
||||||
el.addEventListener("transitionend", remove, { once: true });
|
|
||||||
// Fallback: transitionend never fires if the bubble's visual state didn't
|
|
||||||
// change (dismissed pre-paint) or the tab is backgrounded. The transition
|
|
||||||
// is 0.3s, so force removal a little after.
|
|
||||||
setTimeout(remove, 400);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Fade bubbles whose expiry has passed — strictly oldest-first. We walk the
|
|
||||||
* stack top (oldest) to bottom (newest) and stop at the first bubble still
|
|
||||||
* alive: nothing newer may leave while an older bubble is still on screen. A
|
|
||||||
* bubble that keeps updating pushes its own expiry forward, so it (and
|
|
||||||
* everything behind it) stays put until it finally goes quiet.
|
|
||||||
*/
|
|
||||||
_reapBubbles() {
|
|
||||||
this._reaperHandle = 0;
|
|
||||||
const now = Date.now();
|
|
||||||
const visible = /** @type {HTMLElement[]} */ ([...this._bubbleStack.querySelectorAll(".bubble:not(.out)")]);
|
|
||||||
let nextWake = Infinity;
|
|
||||||
for (const el of visible) {
|
|
||||||
const exp = this._bubbleExpiry.get(el) ?? now; // no expiry recorded → treat as due
|
|
||||||
if (exp <= now) {
|
|
||||||
this._dismissBubble(el);
|
|
||||||
} else {
|
|
||||||
// Oldest survivor isn't due yet; stop so nothing newer leaves before it.
|
|
||||||
nextWake = exp;
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (nextWake !== Infinity) {
|
|
||||||
this._reaperHandle = setTimeout(() => this._reapBubbles(), Math.max(50, nextWake - Date.now()));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* (Re)arm a bubble's auto-dismiss by pushing its expiry out by `delay`.
|
|
||||||
* Calling it again resets the countdown — so a bubble that keeps updating
|
|
||||||
* stays on screen and only fades once it goes quiet. Removal is ordered by the
|
|
||||||
* shared reaper, so the oldest bubble always disappears first.
|
|
||||||
* @param {HTMLElement} el @param {number} [delay]
|
|
||||||
*/
|
|
||||||
_bumpDismiss(el, delay = 4000) {
|
|
||||||
this._bubbleExpiry.set(el, Date.now() + delay);
|
|
||||||
if (!this._reaperHandle) this._reaperHandle = setTimeout(() => this._reapBubbles(), delay);
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── History ───────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
/** Render the empty-state placeholder into the history panel. */
|
|
||||||
renderEmptyState() {
|
|
||||||
this._chatHistory.innerHTML = EMPTY_STATE_HTML;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Reset the panel to the empty state and clear the unread badge. */
|
|
||||||
clear() {
|
|
||||||
this.renderEmptyState();
|
|
||||||
if (this._chatBadge) this._chatBadge.classList.remove("visible");
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {"user"|"assistant"} role @param {string} text @param {boolean} partial @returns {HTMLElement} */
|
|
||||||
_appendHistMsg(role, text, partial) {
|
|
||||||
const empty = this._chatHistory.querySelector(".chat-empty");
|
|
||||||
if (empty) empty.remove();
|
|
||||||
const el = this._buildMessageEl({ container: "hist-msg", prefix: "hist", role, text, partial });
|
|
||||||
this._chatHistory.appendChild(el);
|
|
||||||
this._scrollToBottom();
|
|
||||||
return el;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {HTMLElement | null} el @param {string} text @param {boolean} partial */
|
|
||||||
_updateHistMsg(el, text, partial) {
|
|
||||||
if (!el) return;
|
|
||||||
const body = /** @type {HTMLElement | null} */ (el.querySelector(".hist-body"));
|
|
||||||
if (!body) return;
|
|
||||||
this._renderMessageBody(body, text, el.classList.contains("assistant"));
|
|
||||||
body.classList.toggle("partial", partial);
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Append a tool-call row to the conversation. We only add it once the tool
|
|
||||||
* has run, so the expandable toggle carries BOTH the call input and its result.
|
|
||||||
* @param {string} name @param {string} argsJson @param {string} output
|
|
||||||
*/
|
|
||||||
_appendHistTool(name, argsJson, output) {
|
|
||||||
const empty = this._chatHistory.querySelector(".chat-empty");
|
|
||||||
if (empty) empty.remove();
|
|
||||||
let pretty = argsJson;
|
|
||||||
try { pretty = JSON.stringify(JSON.parse(argsJson), null, 2); } catch {}
|
|
||||||
const el = document.createElement("div");
|
|
||||||
el.className = "hist-msg tool";
|
|
||||||
el.innerHTML = `
|
|
||||||
<div class="hist-role">Llamada a herramienta</div>
|
|
||||||
<button class="hist-tool-header" aria-expanded="false">
|
|
||||||
<svg class="hist-tool-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">${WRENCH_PATH}</svg>
|
|
||||||
<span class="hist-tool-name">${escHtml(name)}</span>
|
|
||||||
<svg class="hist-tool-chevron" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="6 9 12 15 18 9"/></svg>
|
|
||||||
</button>
|
|
||||||
<div class="hist-tool-body">
|
|
||||||
<div class="hist-tool-label">Entrada</div>
|
|
||||||
<div class="hist-tool-block">${escHtml(pretty)}</div>
|
|
||||||
<div class="hist-tool-label">Salida</div>
|
|
||||||
<div class="hist-tool-block hist-tool-output">${escHtml(output || "(sin salida)")}</div>
|
|
||||||
</div>
|
|
||||||
`;
|
|
||||||
const header = /** @type {HTMLButtonElement} */ (el.querySelector(".hist-tool-header"));
|
|
||||||
const body = /** @type {HTMLDivElement} */ (el.querySelector(".hist-tool-body"));
|
|
||||||
header.addEventListener("click", () => {
|
|
||||||
const expanded = header.getAttribute("aria-expanded") === "true";
|
|
||||||
header.setAttribute("aria-expanded", String(!expanded));
|
|
||||||
body.classList.toggle("open", !expanded);
|
|
||||||
});
|
|
||||||
this._chatHistory.appendChild(el);
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Tag an assistant history row as interrupted (user barged in mid-reply).
|
|
||||||
* @param {HTMLElement | null} hist */
|
|
||||||
_markHistInterrupted(hist) {
|
|
||||||
if (!hist || hist.querySelector(".hist-note")) return;
|
|
||||||
hist.classList.add("interrupted");
|
|
||||||
const note = document.createElement("div");
|
|
||||||
note.className = "hist-note";
|
|
||||||
note.textContent = "Interrumpido";
|
|
||||||
hist.appendChild(note);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Render a captured webcam frame in the transcript (the camera tool result).
|
|
||||||
* @param {string} dataUrl */
|
|
||||||
_appendHistImage(dataUrl) {
|
|
||||||
const empty = this._chatHistory.querySelector(".chat-empty");
|
|
||||||
if (empty) empty.remove();
|
|
||||||
const el = document.createElement("div");
|
|
||||||
el.className = "hist-msg tool";
|
|
||||||
el.innerHTML = `<div class="hist-role">Captura</div><img class="hist-image" alt="Captura de la cámara enviada al modelo" />`;
|
|
||||||
const img = /** @type {HTMLImageElement} */ (el.querySelector("img"));
|
|
||||||
img.src = dataUrl;
|
|
||||||
this._chatHistory.appendChild(el);
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Reset all streaming bookkeeping for session start / teardown. Pass
|
|
||||||
* `dismiss` to also fade any bubbles still on screen.
|
|
||||||
* @param {{ dismiss?: boolean }} [opts]
|
|
||||||
*/
|
|
||||||
reset(opts) {
|
|
||||||
if (opts?.dismiss) {
|
|
||||||
if (this._activeUserBubble) this._dismissBubble(this._activeUserBubble);
|
|
||||||
for (const { bubble } of this._asstByResp.values()) this._dismissBubble(bubble);
|
|
||||||
}
|
|
||||||
this._userHistByItem.clear();
|
|
||||||
this._activeUserBubble = null;
|
|
||||||
this._activeUserItemId = "";
|
|
||||||
this._asstByResp.clear();
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Export / download conversation ───────────────────────────────────────
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Serialize the current persistent chat history (.hist-msg rows inside
|
|
||||||
* #chat-history) into a structured JSON object. Covers:
|
|
||||||
* - user / assistant text messages (role, text, timestamp, name, initials)
|
|
||||||
* - tool-call expandable rows (role=tool, name, args, output)
|
|
||||||
* - webcam / snapshot images (role=image, dataUrl)
|
|
||||||
* Empty state rows (.chat-empty) and ephemeral orb bubbles are ignored.
|
|
||||||
* @returns {{
|
|
||||||
* exportedAt: string,
|
|
||||||
* messageCount: number,
|
|
||||||
* messages: Array<{ type: string; timestamp?: string; [k: string]: unknown }>,
|
|
||||||
* }}
|
|
||||||
*/
|
|
||||||
_serializeConversation() {
|
|
||||||
/** @type {Array<{ type: string; timestamp?: string; [k: string]: unknown }>} */
|
|
||||||
const messages = [];
|
|
||||||
if (!this._chatHistory) return { exportedAt: new Date().toISOString(), messageCount: 0, messages };
|
|
||||||
const rows = /** @type {HTMLElement[]} */ ([...this._chatHistory.querySelectorAll(":scope > .hist-msg")]);
|
|
||||||
for (const row of rows) {
|
|
||||||
const ts = row.getAttribute("data-timestamp") || undefined;
|
|
||||||
if (row.classList.contains("user") || row.classList.contains("assistant")) {
|
|
||||||
const body = /** @type {HTMLElement | null} */ (row.querySelector(".hist-body"));
|
|
||||||
if (!body) continue;
|
|
||||||
const role = row.classList.contains("user") ? "user" : "assistant";
|
|
||||||
const out = /** @type {{ type: string; text: string; timestamp?: string; name?: string; initials?: string; interrupted?: boolean }} */ ({
|
|
||||||
type: role,
|
|
||||||
text: this._messageText.get(body) ?? body.textContent ?? "",
|
|
||||||
});
|
|
||||||
if (ts) out.timestamp = ts;
|
|
||||||
if (role === "user") {
|
|
||||||
const n = row.getAttribute("data-user-name");
|
|
||||||
const i = row.getAttribute("data-user-initials");
|
|
||||||
if (n) out.name = n;
|
|
||||||
if (i) out.initials = i;
|
|
||||||
}
|
|
||||||
if (row.classList.contains("interrupted")) out.interrupted = true;
|
|
||||||
messages.push(out);
|
|
||||||
} else if (row.classList.contains("tool")) {
|
|
||||||
const img = /** @type {HTMLImageElement | null} */ (row.querySelector("img.hist-image"));
|
|
||||||
if (img) {
|
|
||||||
messages.push({ type: "image", timestamp: ts, dataUrl: img.src || "" });
|
|
||||||
} else {
|
|
||||||
const nameEl = /** @type {HTMLElement | null} */ (row.querySelector(".hist-tool-name"));
|
|
||||||
const blocks = /** @type {HTMLElement[]} */ ([...row.querySelectorAll(".hist-tool-block")]);
|
|
||||||
const args = blocks[0]?.textContent || "";
|
|
||||||
const output = blocks[1]?.textContent || "";
|
|
||||||
messages.push({
|
|
||||||
type: "tool",
|
|
||||||
timestamp: ts,
|
|
||||||
name: nameEl?.textContent || "",
|
|
||||||
args,
|
|
||||||
output,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
exportedAt: new Date().toISOString(),
|
|
||||||
messageCount: messages.length,
|
|
||||||
messages,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Pretty-print the JSON conversation structure to a human-readable plain
|
|
||||||
* text log (one entry per "HH:MM Name: text"). Tool calls are rendered
|
|
||||||
* as indented blocks (tool name followed by input/output sections).
|
|
||||||
* @param {ReturnType<ChatView["_serializeConversation"]>} conv
|
|
||||||
* @returns {string}
|
|
||||||
*/
|
|
||||||
_conversationToText(conv) {
|
|
||||||
const lines = [];
|
|
||||||
const date = (() => { try { return new Date(conv.exportedAt).toLocaleString(); } catch { return conv.exportedAt; } })();
|
|
||||||
lines.push(`Conversación Vocero — ${date}`);
|
|
||||||
lines.push(`Mensajes totales: ${conv.messageCount}`);
|
|
||||||
lines.push("—".repeat(52));
|
|
||||||
lines.push("");
|
|
||||||
for (const m of conv.messages) {
|
|
||||||
if (m.type === "user" || m.type === "assistant") {
|
|
||||||
const label = m.type === "user" ? (m.name ? String(m.name) : "Usuario") : "Vocero";
|
|
||||||
const head = [m.timestamp, label].filter(Boolean).join(" ");
|
|
||||||
lines.push(`${head}${m.interrupted ? " [interrumpida]" : ""}:`);
|
|
||||||
lines.push(String(m.text || "").trim());
|
|
||||||
lines.push("");
|
|
||||||
} else if (m.type === "tool") {
|
|
||||||
const head = [m.timestamp, `HERRAMIENTA: ${String(m.name || "").trim()}`].filter(Boolean).join(" ");
|
|
||||||
lines.push(head);
|
|
||||||
const args = String(m.args || "").trim();
|
|
||||||
const output = String(m.output || "").trim();
|
|
||||||
if (args) { lines.push(" Entrada:"); lines.push(" " + args.split("\n").join("\n ")); }
|
|
||||||
if (output) { lines.push(" Salida:"); lines.push(" " + output.split("\n").join("\n ")); }
|
|
||||||
lines.push("");
|
|
||||||
} else if (m.type === "image") {
|
|
||||||
const head = [m.timestamp, "CAPTURA (imagen de la cámara enviada al modelo)"].filter(Boolean).join(" ");
|
|
||||||
lines.push(head);
|
|
||||||
lines.push(" " + String(m.dataUrl || "").slice(0, 80) + (String(m.dataUrl || "").length > 80 ? "…" : ""));
|
|
||||||
lines.push("");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return lines.join("\n");
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Trigger a browser "download" for the given string content. Uses the
|
|
||||||
* classic Blob+URL+a.click pattern — no backend call, fully client-side,
|
|
||||||
* works offline and never leaks the conversation content anywhere.
|
|
||||||
* @param {string} content
|
|
||||||
* @param {string} filename
|
|
||||||
* @param {string} mime
|
|
||||||
*/
|
|
||||||
_downloadBlob(content, filename, mime) {
|
|
||||||
const blob = new Blob([content], { type: mime });
|
|
||||||
const url = URL.createObjectURL(blob);
|
|
||||||
try {
|
|
||||||
const a = document.createElement("a");
|
|
||||||
a.href = url;
|
|
||||||
a.download = filename;
|
|
||||||
a.rel = "noopener";
|
|
||||||
document.body.appendChild(a);
|
|
||||||
a.click();
|
|
||||||
setTimeout(() => a.remove(), 0);
|
|
||||||
} finally {
|
|
||||||
setTimeout(() => URL.revokeObjectURL(url), 4000);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Public entry-point wired to the #chat-export-btn in the chat header.
|
|
||||||
* Serializes the current conversation and produces TWO downloads:
|
|
||||||
* 1. vocero-conversacion-YYYYMMDD-HHMMSS.json (structured data)
|
|
||||||
* 2. vocero-conversacion-YYYYMMDD-HHMMSS.txt (human-readable log)
|
|
||||||
* UI feedback: briefly swaps the download icon for a ✓ completion tick then
|
|
||||||
* restores the original SVG so the user gets a visual ack that the export
|
|
||||||
* completed (idempotent / errors are logged to console — never throws).
|
|
||||||
*/
|
|
||||||
exportConversation() {
|
|
||||||
try {
|
|
||||||
const conv = this._serializeConversation();
|
|
||||||
const profile = /** @type {Record<string, unknown> | null} */ (/** @type {unknown} */ (window.voceroUserProfile)) || null;
|
|
||||||
const ex = typeof window._s2sExtractName === "function" ? window._s2sExtractName : null;
|
|
||||||
const userName = ex && profile ? String(ex(profile) || "").trim() : "";
|
|
||||||
const stampBase = (() => {
|
|
||||||
const d = new Date();
|
|
||||||
const pad = (n, w = 2) => String(n).padStart(w, "0");
|
|
||||||
return `${d.getFullYear()}${pad(d.getMonth() + 1)}${pad(d.getDate())}-${pad(d.getHours())}${pad(d.getMinutes())}${pad(d.getSeconds())}`;
|
|
||||||
})();
|
|
||||||
const prefix = _safeName(`vocero-conversacion${userName ? "-" + userName : ""}`, "-");
|
|
||||||
const jsonName = `${prefix}-${stampBase}.json`;
|
|
||||||
const txtName = `${prefix}-${stampBase}.txt`;
|
|
||||||
const jsonText = JSON.stringify(conv, null, 2);
|
|
||||||
const plainText = this._conversationToText(conv);
|
|
||||||
if (conv.messageCount === 0) {
|
|
||||||
// Don't offer downloads of empty logs; flash the button briefly to
|
|
||||||
// signal "pressed" without spamming the user with empty files.
|
|
||||||
if (this._exportBtn) {
|
|
||||||
this._exportBtn.classList.add("chat-export-empty");
|
|
||||||
setTimeout(() => this._exportBtn?.classList.remove("chat-export-empty"), 600);
|
|
||||||
}
|
|
||||||
if (typeof window !== "undefined" && typeof console !== "undefined") {
|
|
||||||
console.info("[ui/chat.js:exportConversation] No messages yet — export skipped.");
|
|
||||||
}
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
this._downloadBlob(jsonText, jsonName, "application/json;charset=utf-8");
|
|
||||||
this._downloadBlob(plainText, txtName, "text/plain;charset=utf-8");
|
|
||||||
// Quick visual OK: paint the button green for 650ms. Avoids modifying
|
|
||||||
// innerHTML (we'd otherwise lose the SVG on mobile fast-clicks).
|
|
||||||
if (this._exportBtn) {
|
|
||||||
this._exportBtn.classList.add("chat-export-ok");
|
|
||||||
setTimeout(() => this._exportBtn?.classList.remove("chat-export-ok"), 650);
|
|
||||||
}
|
|
||||||
} catch (err) {
|
|
||||||
if (typeof window !== "undefined" && typeof console !== "undefined") {
|
|
||||||
console.error("[ui/chat.js:exportConversation] Conversation export failed:", err);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Client event handlers ─────────────────────────────────────────────────
|
|
||||||
|
|
||||||
/**
|
|
||||||
* A streamed transcript delta (user or assistant).
|
|
||||||
* @param {{ role: "user" | "assistant"; text: string; partial: boolean; itemId?: string; responseId?: string }} d
|
|
||||||
*/
|
|
||||||
onTranscript(d) {
|
|
||||||
if (DEBUG) console.debug(`[ui] transcript role=${d.role} partial=${d.partial} item=${d.itemId} resp=${d.responseId} text=${JSON.stringify(d.text)}`);
|
|
||||||
|
|
||||||
if (d.role === "user") {
|
|
||||||
// Group by item_id: a speculative continuation reuses the same id, so it
|
|
||||||
// updates the same row/bubble. A missing id falls back to the active item
|
|
||||||
// (same utterance) or a fresh unique key, never a shared sentinel that
|
|
||||||
// would collapse distinct turns into one row.
|
|
||||||
const id = d.itemId || this._activeUserItemId || `_u${++this._anonSeq}`;
|
|
||||||
const text = d.text;
|
|
||||||
|
|
||||||
let hist = this._userHistByItem.get(id);
|
|
||||||
if (!hist) {
|
|
||||||
hist = this._appendHistMsg("user", text, d.partial);
|
|
||||||
this._userHistByItem.set(id, hist);
|
|
||||||
} else {
|
|
||||||
this._updateHistMsg(hist, text, d.partial);
|
|
||||||
}
|
|
||||||
|
|
||||||
// One ephemeral bubble per active item. Purely timer-based: the timer is
|
|
||||||
// refreshed on every delta, so it stays while the user keeps talking and
|
|
||||||
// fades a few seconds after they stop — no dependency on a response ever
|
|
||||||
// arriving, so it can never get stuck.
|
|
||||||
if (this._activeUserItemId !== id || !this._activeUserBubble) {
|
|
||||||
this._activeUserBubble = this._spawnBubble("user", text);
|
|
||||||
this._activeUserItemId = id;
|
|
||||||
} else {
|
|
||||||
this._updateBubbleText(this._activeUserBubble, text);
|
|
||||||
}
|
|
||||||
this._bumpDismiss(this._activeUserBubble, 6000);
|
|
||||||
this._markUnread();
|
|
||||||
} else if (d.role === "assistant") {
|
|
||||||
// Assistant transcript arrives once, as the full text, keyed by
|
|
||||||
// response_id so a cancelled speculative response can be removed later. A
|
|
||||||
// missing id gets a unique key so two id-less replies never collide.
|
|
||||||
const rid = d.responseId || `_a${++this._anonSeq}`;
|
|
||||||
const entry = this._asstByResp.get(rid);
|
|
||||||
if (!entry) {
|
|
||||||
const bubble = this._spawnBubble("assistant", d.text);
|
|
||||||
this._asstByResp.set(rid, { bubble, hist: this._appendHistMsg("assistant", d.text, false) });
|
|
||||||
this._bumpDismiss(bubble);
|
|
||||||
} else {
|
|
||||||
this._updateBubbleText(entry.bubble, d.text);
|
|
||||||
this._updateHistMsg(entry.hist, d.text, false);
|
|
||||||
this._bumpDismiss(entry.bubble);
|
|
||||||
}
|
|
||||||
this._markUnread();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* A response closed (completed or cancelled).
|
|
||||||
* @param {{ responseId: string; status: string; audible?: boolean; transcript?: string }} detail
|
|
||||||
*/
|
|
||||||
onResponseFinished(detail) {
|
|
||||||
const { responseId, status, audible, transcript } = detail;
|
|
||||||
if (DEBUG) console.debug(`[ui] response-finished resp=${responseId} status=${status} audible=${audible} known=${this._asstByResp.has(responseId)}`);
|
|
||||||
// Without an id we can't target a specific response; the bubble will
|
|
||||||
// auto-dismiss on its own timer regardless.
|
|
||||||
if (!responseId) return;
|
|
||||||
const entry = this._asstByResp.get(responseId);
|
|
||||||
|
|
||||||
if (status === "cancelled") {
|
|
||||||
// Keep every transcript that was received — mark it interrupted rather
|
|
||||||
// than erasing it. If the `*.transcript.done` never fired, build the row
|
|
||||||
// from the text carried in response.done.
|
|
||||||
let hist = entry?.hist ?? null;
|
|
||||||
if (!hist && transcript) {
|
|
||||||
hist = this._appendHistMsg("assistant", transcript, false);
|
|
||||||
} else if (hist && transcript) {
|
|
||||||
this._updateHistMsg(hist, transcript, false);
|
|
||||||
}
|
|
||||||
if (hist) this._markHistInterrupted(hist);
|
|
||||||
this._asstByResp.delete(responseId);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Any other terminal close (completed / failed / incomplete / …): just
|
|
||||||
// release the map entry. The bubble already auto-dismisses on its timer and
|
|
||||||
// the history row persists as the conversation log. Crucially we do NOT
|
|
||||||
// touch user state here — that lifecycle is fully independent.
|
|
||||||
this._asstByResp.delete(responseId);
|
|
||||||
// ALWAYS re-anchor to the last line when a response ends, since the final
|
|
||||||
// punctuation line + any trailing tool-result row pushes the tail further
|
|
||||||
// down than the last streamed delta did.
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The model called a tool — show an ephemeral "running" bubble.
|
|
||||||
* @param {string} name */
|
|
||||||
onToolCall(name) {
|
|
||||||
this._bumpDismiss(this._spawnBubble("tool", name));
|
|
||||||
this._markUnread();
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The tool finished — append its call+result row (and any captured image).
|
|
||||||
* @param {string} name @param {string} argsJson @param {string} output @param {string} [image] */
|
|
||||||
onToolResult(name, argsJson, output, image) {
|
|
||||||
const startedAt = performance.now();
|
|
||||||
this._appendHistTool(name, argsJson, output);
|
|
||||||
if (image) this._appendHistImage(image);
|
|
||||||
// Tool rows can be tall (big JSON payloads, tables). Force a scroll after
|
|
||||||
// insert so the start of the result isn't hidden off-screen above the
|
|
||||||
// sticky footer; we want the LAST line of the answer always in view.
|
|
||||||
this._scrollToBottom();
|
|
||||||
this._markUnread();
|
|
||||||
// #region debug-point C:chat-tool-render
|
|
||||||
reportChatDebug("C", "ui/chat.js:onToolResult", "rendered tool result row", {
|
|
||||||
name,
|
|
||||||
argsLength: argsJson.length,
|
|
||||||
outputLength: output.length,
|
|
||||||
renderMs: Math.round(performance.now() - startedAt),
|
|
||||||
});
|
|
||||||
// #endregion
|
|
||||||
if (image) this._appendHistImage(image); // show the captured frame below the call
|
|
||||||
this._markUnread();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/** Small shared helpers used across the UI modules: a strict query selector,
|
|
||||||
* HTML escaping for text we drop into innerHTML, error-string trimming, and
|
|
||||||
* the opt-in debug flag. */
|
|
||||||
|
|
||||||
/** Opt-in tracing: `localStorage.setItem("s2s.debug", "1")` then reload. */
|
|
||||||
export const DEBUG = (() => {
|
|
||||||
try {
|
|
||||||
return localStorage.getItem("s2s.debug") === "1";
|
|
||||||
} catch {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
})();
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Query a single element, throwing if it's missing (so a broken selector fails
|
|
||||||
* loudly at startup rather than as a later null-deref).
|
|
||||||
* @template {HTMLElement} T
|
|
||||||
* @param {string} selector
|
|
||||||
* @returns {T}
|
|
||||||
*/
|
|
||||||
export function $(selector) {
|
|
||||||
const el = document.querySelector(selector);
|
|
||||||
if (!el) throw new Error(`Missing element: ${selector}`);
|
|
||||||
return /** @type {T} */ (el);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {string} s @returns {string} */
|
|
||||||
export function escHtml(s) {
|
|
||||||
return s.replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">");
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Trim a long error message to fit the orb caption. @param {string} text */
|
|
||||||
export function truncateError(text) {
|
|
||||||
if (text.length <= 90) return text;
|
|
||||||
return text.slice(0, 87) + "…";
|
|
||||||
}
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
import { Marked } from "../vendor/marked/marked.esm.js";
|
|
||||||
import DOMPurify from "../vendor/dompurify/purify.es.mjs";
|
|
||||||
import { escHtml } from "./dom.js";
|
|
||||||
|
|
||||||
const markdown = new Marked({
|
|
||||||
gfm: true,
|
|
||||||
breaks: true,
|
|
||||||
renderer: {
|
|
||||||
// Treat embedded HTML as text. Replies can format Markdown, not UI controls.
|
|
||||||
html: ({ text }) => escHtml(text),
|
|
||||||
checkbox: ({ checked }) => checked ? "☑ " : "☐ ",
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
/** Parse assistant text into passive, sanitized message content. @param {string} text */
|
|
||||||
export function renderMarkdown(text) {
|
|
||||||
if (!DOMPurify.isSupported) return document.createTextNode(text);
|
|
||||||
try {
|
|
||||||
const html = markdown.parse(text, { async: false });
|
|
||||||
const fragment = DOMPurify.sanitize(html, {
|
|
||||||
ALLOWED_TAGS: [
|
|
||||||
"p", "br", "strong", "em", "del", "a", "img", "ul", "ol", "li",
|
|
||||||
"h1", "h2", "h3", "h4", "h5", "h6", "blockquote", "pre", "code", "hr",
|
|
||||||
"table", "thead", "tbody", "tr", "th", "td",
|
|
||||||
],
|
|
||||||
ALLOWED_ATTR: ["href", "title", "src", "alt", "start", "align"],
|
|
||||||
ALLOW_DATA_ATTR: false,
|
|
||||||
ALLOW_ARIA_ATTR: false,
|
|
||||||
RETURN_DOM_FRAGMENT: true,
|
|
||||||
});
|
|
||||||
for (const link of fragment.querySelectorAll("a")) {
|
|
||||||
if (!link.hasAttribute("href")) {
|
|
||||||
link.replaceWith(...link.childNodes);
|
|
||||||
} else {
|
|
||||||
link.target = "_blank";
|
|
||||||
link.rel = "noopener noreferrer";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for (const image of fragment.querySelectorAll("img")) image.loading = "lazy";
|
|
||||||
for (const table of fragment.querySelectorAll("table")) {
|
|
||||||
const wrapper = document.createElement("div");
|
|
||||||
wrapper.className = "markdown-table";
|
|
||||||
wrapper.tabIndex = 0;
|
|
||||||
wrapper.setAttribute("role", "region");
|
|
||||||
wrapper.setAttribute("aria-label", "Tabla de la respuesta");
|
|
||||||
table.replaceWith(wrapper);
|
|
||||||
wrapper.appendChild(table);
|
|
||||||
}
|
|
||||||
for (const code of fragment.querySelectorAll("pre")) code.tabIndex = 0;
|
|
||||||
return fragment;
|
|
||||||
} catch (err) {
|
|
||||||
console.warn("[ui] Markdown rendering failed", err);
|
|
||||||
return document.createTextNode(text);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Vendored
-202
@@ -1,202 +0,0 @@
|
|||||||
|
|
||||||
Apache License
|
|
||||||
Version 2.0, January 2004
|
|
||||||
http://www.apache.org/licenses/
|
|
||||||
|
|
||||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
||||||
|
|
||||||
1. Definitions.
|
|
||||||
|
|
||||||
"License" shall mean the terms and conditions for use, reproduction,
|
|
||||||
and distribution as defined by Sections 1 through 9 of this document.
|
|
||||||
|
|
||||||
"Licensor" shall mean the copyright owner or entity authorized by
|
|
||||||
the copyright owner that is granting the License.
|
|
||||||
|
|
||||||
"Legal Entity" shall mean the union of the acting entity and all
|
|
||||||
other entities that control, are controlled by, or are under common
|
|
||||||
control with that entity. For the purposes of this definition,
|
|
||||||
"control" means (i) the power, direct or indirect, to cause the
|
|
||||||
direction or management of such entity, whether by contract or
|
|
||||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
||||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
||||||
|
|
||||||
"You" (or "Your") shall mean an individual or Legal Entity
|
|
||||||
exercising permissions granted by this License.
|
|
||||||
|
|
||||||
"Source" form shall mean the preferred form for making modifications,
|
|
||||||
including but not limited to software source code, documentation
|
|
||||||
source, and configuration files.
|
|
||||||
|
|
||||||
"Object" form shall mean any form resulting from mechanical
|
|
||||||
transformation or translation of a Source form, including but
|
|
||||||
not limited to compiled object code, generated documentation,
|
|
||||||
and conversions to other media types.
|
|
||||||
|
|
||||||
"Work" shall mean the work of authorship, whether in Source or
|
|
||||||
Object form, made available under the License, as indicated by a
|
|
||||||
copyright notice that is included in or attached to the work
|
|
||||||
(an example is provided in the Appendix below).
|
|
||||||
|
|
||||||
"Derivative Works" shall mean any work, whether in Source or Object
|
|
||||||
form, that is based on (or derived from) the Work and for which the
|
|
||||||
editorial revisions, annotations, elaborations, or other modifications
|
|
||||||
represent, as a whole, an original work of authorship. For the purposes
|
|
||||||
of this License, Derivative Works shall not include works that remain
|
|
||||||
separable from, or merely link (or bind by name) to the interfaces of,
|
|
||||||
the Work and Derivative Works thereof.
|
|
||||||
|
|
||||||
"Contribution" shall mean any work of authorship, including
|
|
||||||
the original version of the Work and any modifications or additions
|
|
||||||
to that Work or Derivative Works thereof, that is intentionally
|
|
||||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
||||||
or by an individual or Legal Entity authorized to submit on behalf of
|
|
||||||
the copyright owner. For the purposes of this definition, "submitted"
|
|
||||||
means any form of electronic, verbal, or written communication sent
|
|
||||||
to the Licensor or its representatives, including but not limited to
|
|
||||||
communication on electronic mailing lists, source code control systems,
|
|
||||||
and issue tracking systems that are managed by, or on behalf of, the
|
|
||||||
Licensor for the purpose of discussing and improving the Work, but
|
|
||||||
excluding communication that is conspicuously marked or otherwise
|
|
||||||
designated in writing by the copyright owner as "Not a Contribution."
|
|
||||||
|
|
||||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
||||||
on behalf of whom a Contribution has been received by Licensor and
|
|
||||||
subsequently incorporated within the Work.
|
|
||||||
|
|
||||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
||||||
this License, each Contributor hereby grants to You a perpetual,
|
|
||||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
||||||
copyright license to reproduce, prepare Derivative Works of,
|
|
||||||
publicly display, publicly perform, sublicense, and distribute the
|
|
||||||
Work and such Derivative Works in Source or Object form.
|
|
||||||
|
|
||||||
3. Grant of Patent License. Subject to the terms and conditions of
|
|
||||||
this License, each Contributor hereby grants to You a perpetual,
|
|
||||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
||||||
(except as stated in this section) patent license to make, have made,
|
|
||||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
||||||
where such license applies only to those patent claims licensable
|
|
||||||
by such Contributor that are necessarily infringed by their
|
|
||||||
Contribution(s) alone or by combination of their Contribution(s)
|
|
||||||
with the Work to which such Contribution(s) was submitted. If You
|
|
||||||
institute patent litigation against any entity (including a
|
|
||||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
||||||
or a Contribution incorporated within the Work constitutes direct
|
|
||||||
or contributory patent infringement, then any patent licenses
|
|
||||||
granted to You under this License for that Work shall terminate
|
|
||||||
as of the date such litigation is filed.
|
|
||||||
|
|
||||||
4. Redistribution. You may reproduce and distribute copies of the
|
|
||||||
Work or Derivative Works thereof in any medium, with or without
|
|
||||||
modifications, and in Source or Object form, provided that You
|
|
||||||
meet the following conditions:
|
|
||||||
|
|
||||||
(a) You must give any other recipients of the Work or
|
|
||||||
Derivative Works a copy of this License; and
|
|
||||||
|
|
||||||
(b) You must cause any modified files to carry prominent notices
|
|
||||||
stating that You changed the files; and
|
|
||||||
|
|
||||||
(c) You must retain, in the Source form of any Derivative Works
|
|
||||||
that You distribute, all copyright, patent, trademark, and
|
|
||||||
attribution notices from the Source form of the Work,
|
|
||||||
excluding those notices that do not pertain to any part of
|
|
||||||
the Derivative Works; and
|
|
||||||
|
|
||||||
(d) If the Work includes a "NOTICE" text file as part of its
|
|
||||||
distribution, then any Derivative Works that You distribute must
|
|
||||||
include a readable copy of the attribution notices contained
|
|
||||||
within such NOTICE file, excluding those notices that do not
|
|
||||||
pertain to any part of the Derivative Works, in at least one
|
|
||||||
of the following places: within a NOTICE text file distributed
|
|
||||||
as part of the Derivative Works; within the Source form or
|
|
||||||
documentation, if provided along with the Derivative Works; or,
|
|
||||||
within a display generated by the Derivative Works, if and
|
|
||||||
wherever such third-party notices normally appear. The contents
|
|
||||||
of the NOTICE file are for informational purposes only and
|
|
||||||
do not modify the License. You may add Your own attribution
|
|
||||||
notices within Derivative Works that You distribute, alongside
|
|
||||||
or as an addendum to the NOTICE text from the Work, provided
|
|
||||||
that such additional attribution notices cannot be construed
|
|
||||||
as modifying the License.
|
|
||||||
|
|
||||||
You may add Your own copyright statement to Your modifications and
|
|
||||||
may provide additional or different license terms and conditions
|
|
||||||
for use, reproduction, or distribution of Your modifications, or
|
|
||||||
for any such Derivative Works as a whole, provided Your use,
|
|
||||||
reproduction, and distribution of the Work otherwise complies with
|
|
||||||
the conditions stated in this License.
|
|
||||||
|
|
||||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
||||||
any Contribution intentionally submitted for inclusion in the Work
|
|
||||||
by You to the Licensor shall be under the terms and conditions of
|
|
||||||
this License, without any additional terms or conditions.
|
|
||||||
Notwithstanding the above, nothing herein shall supersede or modify
|
|
||||||
the terms of any separate license agreement you may have executed
|
|
||||||
with Licensor regarding such Contributions.
|
|
||||||
|
|
||||||
6. Trademarks. This License does not grant permission to use the trade
|
|
||||||
names, trademarks, service marks, or product names of the Licensor,
|
|
||||||
except as required for reasonable and customary use in describing the
|
|
||||||
origin of the Work and reproducing the content of the NOTICE file.
|
|
||||||
|
|
||||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
||||||
agreed to in writing, Licensor provides the Work (and each
|
|
||||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
||||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
||||||
implied, including, without limitation, any warranties or conditions
|
|
||||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
||||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
||||||
appropriateness of using or redistributing the Work and assume any
|
|
||||||
risks associated with Your exercise of permissions under this License.
|
|
||||||
|
|
||||||
8. Limitation of Liability. In no event and under no legal theory,
|
|
||||||
whether in tort (including negligence), contract, or otherwise,
|
|
||||||
unless required by applicable law (such as deliberate and grossly
|
|
||||||
negligent acts) or agreed to in writing, shall any Contributor be
|
|
||||||
liable to You for damages, including any direct, indirect, special,
|
|
||||||
incidental, or consequential damages of any character arising as a
|
|
||||||
result of this License or out of the use or inability to use the
|
|
||||||
Work (including but not limited to damages for loss of goodwill,
|
|
||||||
work stoppage, computer failure or malfunction, or any and all
|
|
||||||
other commercial damages or losses), even if such Contributor
|
|
||||||
has been advised of the possibility of such damages.
|
|
||||||
|
|
||||||
9. Accepting Warranty or Additional Liability. While redistributing
|
|
||||||
the Work or Derivative Works thereof, You may choose to offer,
|
|
||||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
||||||
or other liability obligations and/or rights consistent with this
|
|
||||||
License. However, in accepting such obligations, You may act only
|
|
||||||
on Your own behalf and on Your sole responsibility, not on behalf
|
|
||||||
of any other Contributor, and only if You agree to indemnify,
|
|
||||||
defend, and hold each Contributor harmless for any liability
|
|
||||||
incurred by, or claims asserted against, such Contributor by reason
|
|
||||||
of your accepting any such warranty or additional liability.
|
|
||||||
|
|
||||||
END OF TERMS AND CONDITIONS
|
|
||||||
|
|
||||||
APPENDIX: How to apply the Apache License to your work.
|
|
||||||
|
|
||||||
To apply the Apache License to your work, attach the following
|
|
||||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
||||||
replaced with your own identifying information. (Don't include
|
|
||||||
the brackets!) The text should be enclosed in the appropriate
|
|
||||||
comment syntax for the file format. We also recommend that a
|
|
||||||
file or class name and description of purpose be included on the
|
|
||||||
same "printed page" as the copyright notice for easier
|
|
||||||
identification within third-party archives.
|
|
||||||
|
|
||||||
Copyright [yyyy] [name of copyright owner]
|
|
||||||
|
|
||||||
Licensed under the Apache License, Version 2.0 (the "License");
|
|
||||||
you may not use this file except in compliance with the License.
|
|
||||||
You may obtain a copy of the License at
|
|
||||||
|
|
||||||
http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
|
|
||||||
Unless required by applicable law or agreed to in writing, software
|
|
||||||
distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
See the License for the specific language governing permissions and
|
|
||||||
limitations under the License.
|
|
||||||
-373
@@ -1,373 +0,0 @@
|
|||||||
Mozilla Public License Version 2.0
|
|
||||||
==================================
|
|
||||||
|
|
||||||
1. Definitions
|
|
||||||
--------------
|
|
||||||
|
|
||||||
1.1. "Contributor"
|
|
||||||
means each individual or legal entity that creates, contributes to
|
|
||||||
the creation of, or owns Covered Software.
|
|
||||||
|
|
||||||
1.2. "Contributor Version"
|
|
||||||
means the combination of the Contributions of others (if any) used
|
|
||||||
by a Contributor and that particular Contributor's Contribution.
|
|
||||||
|
|
||||||
1.3. "Contribution"
|
|
||||||
means Covered Software of a particular Contributor.
|
|
||||||
|
|
||||||
1.4. "Covered Software"
|
|
||||||
means Source Code Form to which the initial Contributor has attached
|
|
||||||
the notice in Exhibit A, the Executable Form of such Source Code
|
|
||||||
Form, and Modifications of such Source Code Form, in each case
|
|
||||||
including portions thereof.
|
|
||||||
|
|
||||||
1.5. "Incompatible With Secondary Licenses"
|
|
||||||
means
|
|
||||||
|
|
||||||
(a) that the initial Contributor has attached the notice described
|
|
||||||
in Exhibit B to the Covered Software; or
|
|
||||||
|
|
||||||
(b) that the Covered Software was made available under the terms of
|
|
||||||
version 1.1 or earlier of the License, but not also under the
|
|
||||||
terms of a Secondary License.
|
|
||||||
|
|
||||||
1.6. "Executable Form"
|
|
||||||
means any form of the work other than Source Code Form.
|
|
||||||
|
|
||||||
1.7. "Larger Work"
|
|
||||||
means a work that combines Covered Software with other material, in
|
|
||||||
a separate file or files, that is not Covered Software.
|
|
||||||
|
|
||||||
1.8. "License"
|
|
||||||
means this document.
|
|
||||||
|
|
||||||
1.9. "Licensable"
|
|
||||||
means having the right to grant, to the maximum extent possible,
|
|
||||||
whether at the time of the initial grant or subsequently, any and
|
|
||||||
all of the rights conveyed by this License.
|
|
||||||
|
|
||||||
1.10. "Modifications"
|
|
||||||
means any of the following:
|
|
||||||
|
|
||||||
(a) any file in Source Code Form that results from an addition to,
|
|
||||||
deletion from, or modification of the contents of Covered
|
|
||||||
Software; or
|
|
||||||
|
|
||||||
(b) any new file in Source Code Form that contains any Covered
|
|
||||||
Software.
|
|
||||||
|
|
||||||
1.11. "Patent Claims" of a Contributor
|
|
||||||
means any patent claim(s), including without limitation, method,
|
|
||||||
process, and apparatus claims, in any patent Licensable by such
|
|
||||||
Contributor that would be infringed, but for the grant of the
|
|
||||||
License, by the making, using, selling, offering for sale, having
|
|
||||||
made, import, or transfer of either its Contributions or its
|
|
||||||
Contributor Version.
|
|
||||||
|
|
||||||
1.12. "Secondary License"
|
|
||||||
means either the GNU General Public License, Version 2.0, the GNU
|
|
||||||
Lesser General Public License, Version 2.1, the GNU Affero General
|
|
||||||
Public License, Version 3.0, or any later versions of those
|
|
||||||
licenses.
|
|
||||||
|
|
||||||
1.13. "Source Code Form"
|
|
||||||
means the form of the work preferred for making modifications.
|
|
||||||
|
|
||||||
1.14. "You" (or "Your")
|
|
||||||
means an individual or a legal entity exercising rights under this
|
|
||||||
License. For legal entities, "You" includes any entity that
|
|
||||||
controls, is controlled by, or is under common control with You. For
|
|
||||||
purposes of this definition, "control" means (a) the power, direct
|
|
||||||
or indirect, to cause the direction or management of such entity,
|
|
||||||
whether by contract or otherwise, or (b) ownership of more than
|
|
||||||
fifty percent (50%) of the outstanding shares or beneficial
|
|
||||||
ownership of such entity.
|
|
||||||
|
|
||||||
2. License Grants and Conditions
|
|
||||||
--------------------------------
|
|
||||||
|
|
||||||
2.1. Grants
|
|
||||||
|
|
||||||
Each Contributor hereby grants You a world-wide, royalty-free,
|
|
||||||
non-exclusive license:
|
|
||||||
|
|
||||||
(a) under intellectual property rights (other than patent or trademark)
|
|
||||||
Licensable by such Contributor to use, reproduce, make available,
|
|
||||||
modify, display, perform, distribute, and otherwise exploit its
|
|
||||||
Contributions, either on an unmodified basis, with Modifications, or
|
|
||||||
as part of a Larger Work; and
|
|
||||||
|
|
||||||
(b) under Patent Claims of such Contributor to make, use, sell, offer
|
|
||||||
for sale, have made, import, and otherwise transfer either its
|
|
||||||
Contributions or its Contributor Version.
|
|
||||||
|
|
||||||
2.2. Effective Date
|
|
||||||
|
|
||||||
The licenses granted in Section 2.1 with respect to any Contribution
|
|
||||||
become effective for each Contribution on the date the Contributor first
|
|
||||||
distributes such Contribution.
|
|
||||||
|
|
||||||
2.3. Limitations on Grant Scope
|
|
||||||
|
|
||||||
The licenses granted in this Section 2 are the only rights granted under
|
|
||||||
this License. No additional rights or licenses will be implied from the
|
|
||||||
distribution or licensing of Covered Software under this License.
|
|
||||||
Notwithstanding Section 2.1(b) above, no patent license is granted by a
|
|
||||||
Contributor:
|
|
||||||
|
|
||||||
(a) for any code that a Contributor has removed from Covered Software;
|
|
||||||
or
|
|
||||||
|
|
||||||
(b) for infringements caused by: (i) Your and any other third party's
|
|
||||||
modifications of Covered Software, or (ii) the combination of its
|
|
||||||
Contributions with other software (except as part of its Contributor
|
|
||||||
Version); or
|
|
||||||
|
|
||||||
(c) under Patent Claims infringed by Covered Software in the absence of
|
|
||||||
its Contributions.
|
|
||||||
|
|
||||||
This License does not grant any rights in the trademarks, service marks,
|
|
||||||
or logos of any Contributor (except as may be necessary to comply with
|
|
||||||
the notice requirements in Section 3.4).
|
|
||||||
|
|
||||||
2.4. Subsequent Licenses
|
|
||||||
|
|
||||||
No Contributor makes additional grants as a result of Your choice to
|
|
||||||
distribute the Covered Software under a subsequent version of this
|
|
||||||
License (see Section 10.2) or under the terms of a Secondary License (if
|
|
||||||
permitted under the terms of Section 3.3).
|
|
||||||
|
|
||||||
2.5. Representation
|
|
||||||
|
|
||||||
Each Contributor represents that the Contributor believes its
|
|
||||||
Contributions are its original creation(s) or it has sufficient rights
|
|
||||||
to grant the rights to its Contributions conveyed by this License.
|
|
||||||
|
|
||||||
2.6. Fair Use
|
|
||||||
|
|
||||||
This License is not intended to limit any rights You have under
|
|
||||||
applicable copyright doctrines of fair use, fair dealing, or other
|
|
||||||
equivalents.
|
|
||||||
|
|
||||||
2.7. Conditions
|
|
||||||
|
|
||||||
Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted
|
|
||||||
in Section 2.1.
|
|
||||||
|
|
||||||
3. Responsibilities
|
|
||||||
-------------------
|
|
||||||
|
|
||||||
3.1. Distribution of Source Form
|
|
||||||
|
|
||||||
All distribution of Covered Software in Source Code Form, including any
|
|
||||||
Modifications that You create or to which You contribute, must be under
|
|
||||||
the terms of this License. You must inform recipients that the Source
|
|
||||||
Code Form of the Covered Software is governed by the terms of this
|
|
||||||
License, and how they can obtain a copy of this License. You may not
|
|
||||||
attempt to alter or restrict the recipients' rights in the Source Code
|
|
||||||
Form.
|
|
||||||
|
|
||||||
3.2. Distribution of Executable Form
|
|
||||||
|
|
||||||
If You distribute Covered Software in Executable Form then:
|
|
||||||
|
|
||||||
(a) such Covered Software must also be made available in Source Code
|
|
||||||
Form, as described in Section 3.1, and You must inform recipients of
|
|
||||||
the Executable Form how they can obtain a copy of such Source Code
|
|
||||||
Form by reasonable means in a timely manner, at a charge no more
|
|
||||||
than the cost of distribution to the recipient; and
|
|
||||||
|
|
||||||
(b) You may distribute such Executable Form under the terms of this
|
|
||||||
License, or sublicense it under different terms, provided that the
|
|
||||||
license for the Executable Form does not attempt to limit or alter
|
|
||||||
the recipients' rights in the Source Code Form under this License.
|
|
||||||
|
|
||||||
3.3. Distribution of a Larger Work
|
|
||||||
|
|
||||||
You may create and distribute a Larger Work under terms of Your choice,
|
|
||||||
provided that You also comply with the requirements of this License for
|
|
||||||
the Covered Software. If the Larger Work is a combination of Covered
|
|
||||||
Software with a work governed by one or more Secondary Licenses, and the
|
|
||||||
Covered Software is not Incompatible With Secondary Licenses, this
|
|
||||||
License permits You to additionally distribute such Covered Software
|
|
||||||
under the terms of such Secondary License(s), so that the recipient of
|
|
||||||
the Larger Work may, at their option, further distribute the Covered
|
|
||||||
Software under the terms of either this License or such Secondary
|
|
||||||
License(s).
|
|
||||||
|
|
||||||
3.4. Notices
|
|
||||||
|
|
||||||
You may not remove or alter the substance of any license notices
|
|
||||||
(including copyright notices, patent notices, disclaimers of warranty,
|
|
||||||
or limitations of liability) contained within the Source Code Form of
|
|
||||||
the Covered Software, except that You may alter any license notices to
|
|
||||||
the extent required to remedy known factual inaccuracies.
|
|
||||||
|
|
||||||
3.5. Application of Additional Terms
|
|
||||||
|
|
||||||
You may choose to offer, and to charge a fee for, warranty, support,
|
|
||||||
indemnity or liability obligations to one or more recipients of Covered
|
|
||||||
Software. However, You may do so only on Your own behalf, and not on
|
|
||||||
behalf of any Contributor. You must make it absolutely clear that any
|
|
||||||
such warranty, support, indemnity, or liability obligation is offered by
|
|
||||||
You alone, and You hereby agree to indemnify every Contributor for any
|
|
||||||
liability incurred by such Contributor as a result of warranty, support,
|
|
||||||
indemnity or liability terms You offer. You may include additional
|
|
||||||
disclaimers of warranty and limitations of liability specific to any
|
|
||||||
jurisdiction.
|
|
||||||
|
|
||||||
4. Inability to Comply Due to Statute or Regulation
|
|
||||||
---------------------------------------------------
|
|
||||||
|
|
||||||
If it is impossible for You to comply with any of the terms of this
|
|
||||||
License with respect to some or all of the Covered Software due to
|
|
||||||
statute, judicial order, or regulation then You must: (a) comply with
|
|
||||||
the terms of this License to the maximum extent possible; and (b)
|
|
||||||
describe the limitations and the code they affect. Such description must
|
|
||||||
be placed in a text file included with all distributions of the Covered
|
|
||||||
Software under this License. Except to the extent prohibited by statute
|
|
||||||
or regulation, such description must be sufficiently detailed for a
|
|
||||||
recipient of ordinary skill to be able to understand it.
|
|
||||||
|
|
||||||
5. Termination
|
|
||||||
--------------
|
|
||||||
|
|
||||||
5.1. The rights granted under this License will terminate automatically
|
|
||||||
if You fail to comply with any of its terms. However, if You become
|
|
||||||
compliant, then the rights granted under this License from a particular
|
|
||||||
Contributor are reinstated (a) provisionally, unless and until such
|
|
||||||
Contributor explicitly and finally terminates Your grants, and (b) on an
|
|
||||||
ongoing basis, if such Contributor fails to notify You of the
|
|
||||||
non-compliance by some reasonable means prior to 60 days after You have
|
|
||||||
come back into compliance. Moreover, Your grants from a particular
|
|
||||||
Contributor are reinstated on an ongoing basis if such Contributor
|
|
||||||
notifies You of the non-compliance by some reasonable means, this is the
|
|
||||||
first time You have received notice of non-compliance with this License
|
|
||||||
from such Contributor, and You become compliant prior to 30 days after
|
|
||||||
Your receipt of the notice.
|
|
||||||
|
|
||||||
5.2. If You initiate litigation against any entity by asserting a patent
|
|
||||||
infringement claim (excluding declaratory judgment actions,
|
|
||||||
counter-claims, and cross-claims) alleging that a Contributor Version
|
|
||||||
directly or indirectly infringes any patent, then the rights granted to
|
|
||||||
You by any and all Contributors for the Covered Software under Section
|
|
||||||
2.1 of this License shall terminate.
|
|
||||||
|
|
||||||
5.3. In the event of termination under Sections 5.1 or 5.2 above, all
|
|
||||||
end user license agreements (excluding distributors and resellers) which
|
|
||||||
have been validly granted by You or Your distributors under this License
|
|
||||||
prior to termination shall survive termination.
|
|
||||||
|
|
||||||
************************************************************************
|
|
||||||
* *
|
|
||||||
* 6. Disclaimer of Warranty *
|
|
||||||
* ------------------------- *
|
|
||||||
* *
|
|
||||||
* Covered Software is provided under this License on an "as is" *
|
|
||||||
* basis, without warranty of any kind, either expressed, implied, or *
|
|
||||||
* statutory, including, without limitation, warranties that the *
|
|
||||||
* Covered Software is free of defects, merchantable, fit for a *
|
|
||||||
* particular purpose or non-infringing. The entire risk as to the *
|
|
||||||
* quality and performance of the Covered Software is with You. *
|
|
||||||
* Should any Covered Software prove defective in any respect, You *
|
|
||||||
* (not any Contributor) assume the cost of any necessary servicing, *
|
|
||||||
* repair, or correction. This disclaimer of warranty constitutes an *
|
|
||||||
* essential part of this License. No use of any Covered Software is *
|
|
||||||
* authorized under this License except under this disclaimer. *
|
|
||||||
* *
|
|
||||||
************************************************************************
|
|
||||||
|
|
||||||
************************************************************************
|
|
||||||
* *
|
|
||||||
* 7. Limitation of Liability *
|
|
||||||
* -------------------------- *
|
|
||||||
* *
|
|
||||||
* Under no circumstances and under no legal theory, whether tort *
|
|
||||||
* (including negligence), contract, or otherwise, shall any *
|
|
||||||
* Contributor, or anyone who distributes Covered Software as *
|
|
||||||
* permitted above, be liable to You for any direct, indirect, *
|
|
||||||
* special, incidental, or consequential damages of any character *
|
|
||||||
* including, without limitation, damages for lost profits, loss of *
|
|
||||||
* goodwill, work stoppage, computer failure or malfunction, or any *
|
|
||||||
* and all other commercial damages or losses, even if such party *
|
|
||||||
* shall have been informed of the possibility of such damages. This *
|
|
||||||
* limitation of liability shall not apply to liability for death or *
|
|
||||||
* personal injury resulting from such party's negligence to the *
|
|
||||||
* extent applicable law prohibits such limitation. Some *
|
|
||||||
* jurisdictions do not allow the exclusion or limitation of *
|
|
||||||
* incidental or consequential damages, so this exclusion and *
|
|
||||||
* limitation may not apply to You. *
|
|
||||||
* *
|
|
||||||
************************************************************************
|
|
||||||
|
|
||||||
8. Litigation
|
|
||||||
-------------
|
|
||||||
|
|
||||||
Any litigation relating to this License may be brought only in the
|
|
||||||
courts of a jurisdiction where the defendant maintains its principal
|
|
||||||
place of business and such litigation shall be governed by laws of that
|
|
||||||
jurisdiction, without reference to its conflict-of-law provisions.
|
|
||||||
Nothing in this Section shall prevent a party's ability to bring
|
|
||||||
cross-claims or counter-claims.
|
|
||||||
|
|
||||||
9. Miscellaneous
|
|
||||||
----------------
|
|
||||||
|
|
||||||
This License represents the complete agreement concerning the subject
|
|
||||||
matter hereof. If any provision of this License is held to be
|
|
||||||
unenforceable, such provision shall be reformed only to the extent
|
|
||||||
necessary to make it enforceable. Any law or regulation which provides
|
|
||||||
that the language of a contract shall be construed against the drafter
|
|
||||||
shall not be used to construe this License against a Contributor.
|
|
||||||
|
|
||||||
10. Versions of the License
|
|
||||||
---------------------------
|
|
||||||
|
|
||||||
10.1. New Versions
|
|
||||||
|
|
||||||
Mozilla Foundation is the license steward. Except as provided in Section
|
|
||||||
10.3, no one other than the license steward has the right to modify or
|
|
||||||
publish new versions of this License. Each version will be given a
|
|
||||||
distinguishing version number.
|
|
||||||
|
|
||||||
10.2. Effect of New Versions
|
|
||||||
|
|
||||||
You may distribute the Covered Software under the terms of the version
|
|
||||||
of the License under which You originally received the Covered Software,
|
|
||||||
or under the terms of any subsequent version published by the license
|
|
||||||
steward.
|
|
||||||
|
|
||||||
10.3. Modified Versions
|
|
||||||
|
|
||||||
If you create software not governed by this License, and you want to
|
|
||||||
create a new license for such software, you may create and use a
|
|
||||||
modified version of this License if you rename the license and remove
|
|
||||||
any references to the name of the license steward (except to note that
|
|
||||||
such modified license differs from this License).
|
|
||||||
|
|
||||||
10.4. Distributing Source Code Form that is Incompatible With Secondary
|
|
||||||
Licenses
|
|
||||||
|
|
||||||
If You choose to distribute Source Code Form that is Incompatible With
|
|
||||||
Secondary Licenses under the terms of this version of the License, the
|
|
||||||
notice described in Exhibit B of this License must be attached.
|
|
||||||
|
|
||||||
Exhibit A - Source Code Form License Notice
|
|
||||||
-------------------------------------------
|
|
||||||
|
|
||||||
This Source Code Form is subject to the terms of the Mozilla Public
|
|
||||||
License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
||||||
file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
||||||
|
|
||||||
If it is not possible or desirable to put the notice in a particular
|
|
||||||
file, then You may include the notice in a location (such as a LICENSE
|
|
||||||
file in a relevant directory) where a recipient would be likely to look
|
|
||||||
for such a notice.
|
|
||||||
|
|
||||||
You may add additional accurate notices of copyright ownership.
|
|
||||||
|
|
||||||
Exhibit B - "Incompatible With Secondary Licenses" Notice
|
|
||||||
---------------------------------------------------------
|
|
||||||
|
|
||||||
This Source Code Form is "Incompatible With Secondary Licenses", as
|
|
||||||
defined by the Mozilla Public License, v. 2.0.
|
|
||||||
-2464
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
Vendored
-44
@@ -1,44 +0,0 @@
|
|||||||
# License information
|
|
||||||
|
|
||||||
## Contribution License Agreement
|
|
||||||
|
|
||||||
If you contribute code to this project, you are implicitly allowing your code
|
|
||||||
to be distributed under the MIT license. You are also implicitly verifying that
|
|
||||||
all code is your original work. `</legalese>`
|
|
||||||
|
|
||||||
## Marked
|
|
||||||
|
|
||||||
Copyright (c) 2018+, MarkedJS (https://github.com/markedjs/)
|
|
||||||
Copyright (c) 2011-2018, Christopher Jeffrey (https://github.com/chjj/)
|
|
||||||
|
|
||||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
||||||
of this software and associated documentation files (the "Software"), to deal
|
|
||||||
in the Software without restriction, including without limitation the rights
|
|
||||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
||||||
copies of the Software, and to permit persons to whom the Software is
|
|
||||||
furnished to do so, subject to the following conditions:
|
|
||||||
|
|
||||||
The above copyright notice and this permission notice shall be included in
|
|
||||||
all copies or substantial portions of the Software.
|
|
||||||
|
|
||||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
||||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
||||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
||||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
||||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
||||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
||||||
THE SOFTWARE.
|
|
||||||
|
|
||||||
## Markdown
|
|
||||||
|
|
||||||
Copyright © 2004, John Gruber
|
|
||||||
http://daringfireball.net/
|
|
||||||
All rights reserved.
|
|
||||||
|
|
||||||
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
|
||||||
|
|
||||||
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
|
|
||||||
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
|
|
||||||
* Neither the name “Markdown” nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
|
|
||||||
|
|
||||||
This software is provided by the copyright holders and contributors “as is” and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the copyright owner or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage.
|
|
||||||
-80
File diff suppressed because one or more lines are too long
-7
File diff suppressed because one or more lines are too long
@@ -1,186 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* AudioWorkletProcessor that plays back Float32 mono samples received from
|
|
||||||
* the main thread, upsampling whatever incoming rate the server uses
|
|
||||||
* (typically 24 kHz PCM16) to the AudioContext rate (typically 48 kHz).
|
|
||||||
*
|
|
||||||
* Lifecycle / messaging:
|
|
||||||
*
|
|
||||||
* main -> worklet:
|
|
||||||
* { kind: "config", inputRate: 24000 } one-shot at startup
|
|
||||||
* { kind: "audio", samples: Float32Array } (transferable) per chunk
|
|
||||||
* { kind: "clear" } wipe queue (barge-in)
|
|
||||||
*
|
|
||||||
* worklet -> main:
|
|
||||||
* { kind: "stats", queuedMs, played } every ~250 ms
|
|
||||||
* { kind: "underrun" } every time the queue
|
|
||||||
* runs dry mid-playback
|
|
||||||
*
|
|
||||||
* Underrun strategy: output silence. We do NOT hold the last sample (that
|
|
||||||
* tends to produce audible clicks/buzzes when long gaps appear between
|
|
||||||
* TTS chunks). A short ramp-out + ramp-in at boundaries would be nicer but
|
|
||||||
* the server's 30 ms cadence makes underruns visible only at end of turn.
|
|
||||||
*/
|
|
||||||
|
|
||||||
const STATS_INTERVAL_FRAMES = 12000;
|
|
||||||
const FADE_FRAMES = 32;
|
|
||||||
const START_BUFFER_MS = 90;
|
|
||||||
|
|
||||||
class AudioPlaybackProcessor extends AudioWorkletProcessor {
|
|
||||||
constructor() {
|
|
||||||
super();
|
|
||||||
this._inputRate = 24000;
|
|
||||||
this._stepRatio = this._inputRate / sampleRate;
|
|
||||||
this._queue = [];
|
|
||||||
this._readIdx = 0;
|
|
||||||
this._fracPos = 0;
|
|
||||||
this._playing = false;
|
|
||||||
this._framesSinceStats = 0;
|
|
||||||
this._totalPlayed = 0;
|
|
||||||
this._fadeIn = 0;
|
|
||||||
this._fadeOut = 0;
|
|
||||||
this._lastSample = 0;
|
|
||||||
this._startBuffered = false;
|
|
||||||
|
|
||||||
this.port.onmessage = (e) => {
|
|
||||||
const data = e.data;
|
|
||||||
if (!data || typeof data !== "object") return;
|
|
||||||
switch (data.kind) {
|
|
||||||
case "config":
|
|
||||||
if (typeof data.inputRate === "number" && data.inputRate > 0) {
|
|
||||||
this._inputRate = data.inputRate;
|
|
||||||
this._stepRatio = this._inputRate / sampleRate;
|
|
||||||
}
|
|
||||||
break;
|
|
||||||
case "audio":
|
|
||||||
if (data.samples instanceof Float32Array && data.samples.length > 0) {
|
|
||||||
this._queue.push(data.samples);
|
|
||||||
if (!this._playing && this._queuedMs() >= START_BUFFER_MS) {
|
|
||||||
this._playing = true;
|
|
||||||
this._startBuffered = true;
|
|
||||||
this._fadeIn = FADE_FRAMES;
|
|
||||||
this._fadeOut = 0;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
break;
|
|
||||||
case "clear":
|
|
||||||
this._queue.length = 0;
|
|
||||||
this._readIdx = 0;
|
|
||||||
this._fracPos = 0;
|
|
||||||
this._fadeOut = FADE_FRAMES;
|
|
||||||
this._startBuffered = false;
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
_queuedSamples() {
|
|
||||||
let total = -this._readIdx;
|
|
||||||
for (const buf of this._queue) total += buf.length;
|
|
||||||
return Math.max(0, total);
|
|
||||||
}
|
|
||||||
|
|
||||||
_queuedMs() {
|
|
||||||
return (this._queuedSamples() / this._inputRate) * 1000;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Linear-interp read at the current fractional position. */
|
|
||||||
_readInterpolated() {
|
|
||||||
if (this._queue.length === 0) return null;
|
|
||||||
const head = this._queue[0];
|
|
||||||
const idx = this._readIdx;
|
|
||||||
const frac = this._fracPos;
|
|
||||||
|
|
||||||
let a = head[idx];
|
|
||||||
let b;
|
|
||||||
if (idx + 1 < head.length) {
|
|
||||||
b = head[idx + 1];
|
|
||||||
} else if (this._queue.length > 1) {
|
|
||||||
b = this._queue[1][0];
|
|
||||||
} else {
|
|
||||||
b = a;
|
|
||||||
}
|
|
||||||
return a + (b - a) * frac;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Advance the read position by `stepRatio`; pop consumed buffers. */
|
|
||||||
_advance() {
|
|
||||||
this._fracPos += this._stepRatio;
|
|
||||||
while (this._fracPos >= 1) {
|
|
||||||
this._fracPos -= 1;
|
|
||||||
this._readIdx += 1;
|
|
||||||
}
|
|
||||||
while (this._queue.length > 0 && this._readIdx >= this._queue[0].length) {
|
|
||||||
this._readIdx -= this._queue[0].length;
|
|
||||||
this._queue.shift();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
process(_, outputs) {
|
|
||||||
const channels = outputs[0];
|
|
||||||
if (!channels || channels.length === 0) return true;
|
|
||||||
const out = channels[0];
|
|
||||||
const stereo = channels.length > 1 ? channels[1] : null;
|
|
||||||
|
|
||||||
for (let i = 0; i < out.length; i++) {
|
|
||||||
let sample = 0;
|
|
||||||
|
|
||||||
if (!this._playing && this._queue.length > 0 && this._queuedMs() >= START_BUFFER_MS) {
|
|
||||||
this._playing = true;
|
|
||||||
this._startBuffered = true;
|
|
||||||
this._fadeIn = FADE_FRAMES;
|
|
||||||
this._fadeOut = 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (this._playing) {
|
|
||||||
const v = this._readInterpolated();
|
|
||||||
if (v === null) {
|
|
||||||
// Underrun: try to ramp out cleanly to avoid clicks.
|
|
||||||
sample = this._lastSample * Math.max(0, 1 - 1 / FADE_FRAMES);
|
|
||||||
this._lastSample = sample;
|
|
||||||
if (Math.abs(sample) < 1e-4) {
|
|
||||||
this._playing = false;
|
|
||||||
this._startBuffered = false;
|
|
||||||
this._lastSample = 0;
|
|
||||||
this.port.postMessage({ kind: "underrun" });
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
sample = v;
|
|
||||||
this._lastSample = v;
|
|
||||||
this._advance();
|
|
||||||
}
|
|
||||||
|
|
||||||
if (this._fadeIn > 0) {
|
|
||||||
const gain = 1 - this._fadeIn / FADE_FRAMES;
|
|
||||||
sample *= gain;
|
|
||||||
this._fadeIn -= 1;
|
|
||||||
}
|
|
||||||
if (this._fadeOut > 0) {
|
|
||||||
const gain = this._fadeOut / FADE_FRAMES;
|
|
||||||
sample *= gain;
|
|
||||||
this._fadeOut -= 1;
|
|
||||||
if (this._fadeOut === 0) {
|
|
||||||
this._playing = false;
|
|
||||||
this._startBuffered = false;
|
|
||||||
this._lastSample = 0;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
this._totalPlayed += 1;
|
|
||||||
}
|
|
||||||
|
|
||||||
out[i] = sample;
|
|
||||||
if (stereo) stereo[i] = sample;
|
|
||||||
}
|
|
||||||
|
|
||||||
this._framesSinceStats += out.length;
|
|
||||||
if (this._framesSinceStats >= STATS_INTERVAL_FRAMES) {
|
|
||||||
this._framesSinceStats = 0;
|
|
||||||
this.port.postMessage({ kind: "stats", queuedMs: this._queuedMs(), played: this._totalPlayed });
|
|
||||||
}
|
|
||||||
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
registerProcessor("audio-playback", AudioPlaybackProcessor);
|
|
||||||
@@ -1,159 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* AudioWorkletProcessor that resamples the AudioContext rate (typically 48 kHz)
|
|
||||||
* down to 16 kHz, packs the result as little-endian Int16 PCM, and posts it
|
|
||||||
* back to the main thread in fixed-size chunks.
|
|
||||||
*
|
|
||||||
* The Hugging Face speech-to-speech WebSocket route expects the
|
|
||||||
* `input_audio_buffer.append` payload at 16 kHz PCM16 mono.
|
|
||||||
*
|
|
||||||
* Design notes:
|
|
||||||
* - 48 -> 16 is an exact 3:1 ratio so we use a 3-tap boxcar average as a
|
|
||||||
* cheap low-pass before decimating. Good enough for voice STT; we lose
|
|
||||||
* a tiny bit of >8 kHz content which the pipeline discards anyway.
|
|
||||||
* - Output frames are emitted at the cadence dictated by `chunkMs`
|
|
||||||
* (default 40 ms = 640 samples = 1280 bytes). The OpenAI Realtime
|
|
||||||
* server batches incoming audio so the cadence is flexible; 20-100 ms
|
|
||||||
* is the sweet spot.
|
|
||||||
* - Float -> Int16 saturates to [-1, 1] before scaling.
|
|
||||||
* - Optional noise gate: per-chunk RMS decides open/closed against a
|
|
||||||
* threshold; the gain ramps (fast attack, hold, slow release) so word
|
|
||||||
* onsets aren't clipped and quiet tails don't click. The gate only
|
|
||||||
* affects the audio we SEND; the main-thread visualiser taps the raw
|
|
||||||
* mic separately. We post the chunk RMS up every frame so the Settings
|
|
||||||
* mic meter can show the live level against the threshold.
|
|
||||||
*/
|
|
||||||
|
|
||||||
const TARGET_RATE = 16000;
|
|
||||||
const DEFAULT_CHUNK_MS = 40;
|
|
||||||
// Gate envelope timing (fixed; only the threshold is user-tunable).
|
|
||||||
const GATE_ATTACK_MS = 5; // open almost instantly so word onsets survive
|
|
||||||
const GATE_HOLD_MS = 250; // stay open this long after the level drops back under
|
|
||||||
const GATE_RELEASE_MS = 80; // then fade closed over this long (no click)
|
|
||||||
|
|
||||||
class MicCaptureProcessor extends AudioWorkletProcessor {
|
|
||||||
constructor(options) {
|
|
||||||
super();
|
|
||||||
const chunkMs = options?.processorOptions?.chunkMs ?? DEFAULT_CHUNK_MS;
|
|
||||||
this._inputRate = sampleRate;
|
|
||||||
this._ratio = this._inputRate / TARGET_RATE;
|
|
||||||
this._chunkSamples16k = Math.round((TARGET_RATE * chunkMs) / 1000);
|
|
||||||
this._scratch = new Float32Array(0);
|
|
||||||
this._decimated = new Float32Array(this._chunkSamples16k);
|
|
||||||
this._enabled = true;
|
|
||||||
|
|
||||||
// Noise gate state. Disabled by default (pure passthrough).
|
|
||||||
this._gateEnabled = false;
|
|
||||||
this._thresholdLin = 0; // linear amplitude; signal RMS must exceed this to open
|
|
||||||
this._gateGain = 1; // smoothed gain currently applied
|
|
||||||
this._holdRemaining = 0; // samples left before the gate may start closing
|
|
||||||
this._attackCoef = Math.exp(-1 / ((GATE_ATTACK_MS / 1000) * TARGET_RATE));
|
|
||||||
this._releaseCoef = Math.exp(-1 / ((GATE_RELEASE_MS / 1000) * TARGET_RATE));
|
|
||||||
this._holdSamples = Math.round((GATE_HOLD_MS / 1000) * TARGET_RATE);
|
|
||||||
|
|
||||||
this.port.onmessage = (e) => {
|
|
||||||
const data = e.data;
|
|
||||||
if (data?.kind === "enable") this._enabled = !!data.value;
|
|
||||||
else if (data?.kind === "gate") {
|
|
||||||
this._gateEnabled = !!data.enabled;
|
|
||||||
// dB -> linear amplitude. When off, threshold 0 keeps the gate open.
|
|
||||||
this._thresholdLin = data.enabled ? Math.pow(10, data.thresholdDb / 20) : 0;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Append `incoming` to the internal scratch buffer, then emit as many
|
|
||||||
* full output chunks as we have material for.
|
|
||||||
* @param {Float32Array} incoming
|
|
||||||
*/
|
|
||||||
_ingest(incoming) {
|
|
||||||
if (incoming.length === 0) return;
|
|
||||||
const next = new Float32Array(this._scratch.length + incoming.length);
|
|
||||||
next.set(this._scratch, 0);
|
|
||||||
next.set(incoming, this._scratch.length);
|
|
||||||
this._scratch = next;
|
|
||||||
this._maybeEmit();
|
|
||||||
}
|
|
||||||
|
|
||||||
_maybeEmit() {
|
|
||||||
const r = this._ratio;
|
|
||||||
const n = this._chunkSamples16k;
|
|
||||||
const needIn = Math.ceil(n * r);
|
|
||||||
const dec = this._decimated;
|
|
||||||
while (this._scratch.length >= needIn) {
|
|
||||||
// 1. Decimate to 16 kHz floats and accumulate energy for the gate/meter.
|
|
||||||
let sumSq = 0;
|
|
||||||
if (Math.abs(r - 3) < 1e-6) {
|
|
||||||
// 48 kHz -> 16 kHz fast path with boxcar lowpass.
|
|
||||||
for (let i = 0; i < n; i++) {
|
|
||||||
const idx = i * 3;
|
|
||||||
const s = (this._scratch[idx] + this._scratch[idx + 1] + this._scratch[idx + 2]) / 3;
|
|
||||||
dec[i] = s;
|
|
||||||
sumSq += s * s;
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
// Generic path: linear interpolation. Slower but works at any rate
|
|
||||||
// (e.g. some Windows boxes report sampleRate=44100).
|
|
||||||
for (let i = 0; i < n; i++) {
|
|
||||||
const srcPos = i * r;
|
|
||||||
const idx = Math.floor(srcPos);
|
|
||||||
const frac = srcPos - idx;
|
|
||||||
const a = this._scratch[idx];
|
|
||||||
const b = this._scratch[idx + 1] ?? a;
|
|
||||||
const s = a + (b - a) * frac;
|
|
||||||
dec[i] = s;
|
|
||||||
sumSq += s * s;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
const rms = Math.sqrt(sumSq / n);
|
|
||||||
|
|
||||||
// 2. Decide the gate target for this chunk, then ramp sample-by-sample.
|
|
||||||
let target = 1;
|
|
||||||
if (this._gateEnabled) {
|
|
||||||
if (rms >= this._thresholdLin) {
|
|
||||||
this._holdRemaining = this._holdSamples; // re-arm the hold
|
|
||||||
} else if (this._holdRemaining > 0) {
|
|
||||||
this._holdRemaining -= n; // coasting through the hold window
|
|
||||||
} else {
|
|
||||||
target = 0;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// 3. Apply the (smoothed) gain and pack to Int16.
|
|
||||||
const out = new Int16Array(n);
|
|
||||||
let gain = this._gateGain;
|
|
||||||
for (let i = 0; i < n; i++) {
|
|
||||||
const coef = target > gain ? this._attackCoef : this._releaseCoef;
|
|
||||||
gain = target + (gain - target) * coef;
|
|
||||||
const s = dec[i] * gain;
|
|
||||||
const clamped = s < -1 ? -1 : s > 1 ? 1 : s;
|
|
||||||
out[i] = clamped < 0 ? clamped * 0x8000 : clamped * 0x7fff;
|
|
||||||
}
|
|
||||||
this._gateGain = gain;
|
|
||||||
|
|
||||||
// Shift the scratch buffer to keep only the trailing unused samples.
|
|
||||||
const consumed = Math.floor(n * r);
|
|
||||||
this._scratch = this._scratch.slice(consumed);
|
|
||||||
|
|
||||||
// Live input level for the Settings meter (raw RMS, pre-gate).
|
|
||||||
this.port.postMessage({ kind: "level", rms });
|
|
||||||
|
|
||||||
if (this._enabled) {
|
|
||||||
this.port.postMessage(out.buffer, [out.buffer]);
|
|
||||||
}
|
|
||||||
// When disabled (mic muted) we silently consume input so the worklet
|
|
||||||
// stays alive and the buffer never grows unbounded.
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
process(inputs) {
|
|
||||||
const input = inputs[0];
|
|
||||||
if (!input || input.length === 0 || !input[0]) return true;
|
|
||||||
const mono = input[0];
|
|
||||||
if (mono.length > 0) this._ingest(mono);
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
registerProcessor("mic-capture", MicCaptureProcessor);
|
|
||||||
@@ -1,57 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* Pure, stateless helpers for the WebSocket realtime client: base64 <-> PCM
|
|
||||||
* conversion for the audio frames on the wire, transcript extraction from a
|
|
||||||
* `response.done` payload, and a tiny URL helper. Kept separate from the client
|
|
||||||
* so the protocol/state logic stays readable.
|
|
||||||
*/
|
|
||||||
|
|
||||||
/** @param {string} url */
|
|
||||||
export function trimTrailingSlash(url) {
|
|
||||||
return url.endsWith("/") ? url.slice(0, -1) : url;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Pull the assistant transcript out of a `response.done` payload. The text
|
|
||||||
* lives in `response.output[].content[].transcript` (audio) or `.text`. Used as
|
|
||||||
* the source of truth for interrupted replies, where the dedicated
|
|
||||||
* `*.transcript.done` event may never arrive.
|
|
||||||
* @param {any} response
|
|
||||||
* @returns {string}
|
|
||||||
*/
|
|
||||||
export function extractResponseTranscript(response) {
|
|
||||||
const output = response?.output;
|
|
||||||
if (!Array.isArray(output)) return "";
|
|
||||||
/** @type {string[]} */
|
|
||||||
const parts = [];
|
|
||||||
for (const item of output) {
|
|
||||||
for (const part of item?.content ?? []) {
|
|
||||||
const text = part?.transcript ?? part?.text;
|
|
||||||
if (typeof text === "string" && text.trim()) parts.push(text.trim());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return parts.join(" ").trim();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {ArrayBuffer} buf */
|
|
||||||
export function base64FromArrayBuffer(buf) {
|
|
||||||
const bytes = new Uint8Array(buf);
|
|
||||||
// Chunked encoding so we don't blow up the call stack on long buffers.
|
|
||||||
let binary = "";
|
|
||||||
const chunk = 0x8000;
|
|
||||||
for (let i = 0; i < bytes.length; i += chunk) {
|
|
||||||
binary += String.fromCharCode.apply(null, /** @type {number[]} */ (
|
|
||||||
/** @type {unknown} */ (bytes.subarray(i, i + chunk))
|
|
||||||
));
|
|
||||||
}
|
|
||||||
return btoa(binary);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {string} b64 */
|
|
||||||
export function base64ToBytes(b64) {
|
|
||||||
const binary = atob(b64);
|
|
||||||
const len = binary.length;
|
|
||||||
const out = new Uint8Array(len);
|
|
||||||
for (let i = 0; i < len; i++) out[i] = binary.charCodeAt(i);
|
|
||||||
return out;
|
|
||||||
}
|
|
||||||
@@ -1,98 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* Orb spectrum visualiser. Each animation frame it reads two AnalyserNodes (the
|
|
||||||
* mic input and the TTS output) and maps the low-frequency speech energy onto
|
|
||||||
* the orb's CSS custom properties:
|
|
||||||
* - `--bar0`..`--bar4` the 5-band level meter
|
|
||||||
* - `--ai-audio-level` the global "Reachy talks" glow / scale pulse
|
|
||||||
*
|
|
||||||
* The bottom of the FFT is where speech energy lives, so the band edges stay
|
|
||||||
* low — that keeps the bars dancing on voice rather than on noise. While the AI
|
|
||||||
* is speaking we source the bars from the OUTPUT analyser so the orb pulses with
|
|
||||||
* Reachy's voice instead of sitting dead while the user is silent.
|
|
||||||
*/
|
|
||||||
|
|
||||||
// Exported so the client can size its AnalyserNodes to match our buffer.
|
|
||||||
export const VIS_FFT_SIZE = 256;
|
|
||||||
const VIS_BAND_COUNT = 5;
|
|
||||||
const VIS_BAND_EDGES = [2, 5, 9, 16, 28, 52];
|
|
||||||
const VIS_ATTACK = 0.6; // weight for new sample on upswing (snappy)
|
|
||||||
const VIS_RELEASE = 0.18; // weight for new sample on decay (gentle fade)
|
|
||||||
|
|
||||||
export class OrbVisualiser {
|
|
||||||
/**
|
|
||||||
* @param {AnalyserNode} micAnalyser
|
|
||||||
* @param {AnalyserNode} outAnalyser
|
|
||||||
* @param {() => boolean} isAiSpeaking Source the bars from the AI output when
|
|
||||||
* true, otherwise from the mic.
|
|
||||||
*/
|
|
||||||
constructor(micAnalyser, outAnalyser, isAiSpeaking) {
|
|
||||||
this._mic = micAnalyser;
|
|
||||||
this._out = outAnalyser;
|
|
||||||
this._isAiSpeaking = isAiSpeaking;
|
|
||||||
this._buf = new Uint8Array(micAnalyser.frequencyBinCount);
|
|
||||||
this._bands = new Float32Array(VIS_BAND_COUNT);
|
|
||||||
this._aiLevel = 0;
|
|
||||||
/** @type {number | null} */
|
|
||||||
this._frame = null;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Begin the rAF loop (idempotent). */
|
|
||||||
start() {
|
|
||||||
if (this._frame !== null) return;
|
|
||||||
const root = document.documentElement;
|
|
||||||
const tick = () => {
|
|
||||||
this._frame = requestAnimationFrame(tick);
|
|
||||||
this._update(root);
|
|
||||||
};
|
|
||||||
this._frame = requestAnimationFrame(tick);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Stop the loop and clear the CSS vars so the orb returns to rest. */
|
|
||||||
stop() {
|
|
||||||
if (this._frame !== null) {
|
|
||||||
cancelAnimationFrame(this._frame);
|
|
||||||
this._frame = null;
|
|
||||||
}
|
|
||||||
const root = document.documentElement;
|
|
||||||
for (let i = 0; i < VIS_BAND_COUNT; i++) root.style.removeProperty(`--bar${i}`);
|
|
||||||
root.style.removeProperty("--ai-audio-level");
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {HTMLElement} root */
|
|
||||||
_update(root) {
|
|
||||||
// Mic bars: split FFT into 5 log-ish bands, smooth, write CSS vars.
|
|
||||||
const source = this._isAiSpeaking() ? this._out : this._mic;
|
|
||||||
source.getByteFrequencyData(this._buf);
|
|
||||||
|
|
||||||
for (let b = 0; b < VIS_BAND_COUNT; b++) {
|
|
||||||
const lo = VIS_BAND_EDGES[b];
|
|
||||||
const hi = VIS_BAND_EDGES[b + 1];
|
|
||||||
let sum = 0;
|
|
||||||
let n = 0;
|
|
||||||
for (let i = lo; i < hi && i < this._buf.length; i++) {
|
|
||||||
sum += this._buf[i];
|
|
||||||
n += 1;
|
|
||||||
}
|
|
||||||
const target = n > 0 ? sum / (n * 255) : 0;
|
|
||||||
const prev = this._bands[b];
|
|
||||||
const k = target > prev ? VIS_ATTACK : VIS_RELEASE;
|
|
||||||
const next = prev + (target - prev) * k;
|
|
||||||
this._bands[b] = next;
|
|
||||||
root.style.setProperty(`--bar${b}`, next.toFixed(3));
|
|
||||||
}
|
|
||||||
|
|
||||||
// Global AI audio level: peak of the output analyser, used by the CSS to
|
|
||||||
// make the orb's glow / scale react to Reachy's voice.
|
|
||||||
this._out.getByteFrequencyData(this._buf);
|
|
||||||
let peak = 0;
|
|
||||||
const limit = Math.min(this._buf.length, VIS_BAND_EDGES[VIS_BAND_COUNT]);
|
|
||||||
for (let i = 0; i < limit; i++) {
|
|
||||||
if (this._buf[i] > peak) peak = this._buf[i];
|
|
||||||
}
|
|
||||||
const aiTarget = peak / 255;
|
|
||||||
const k = aiTarget > this._aiLevel ? VIS_ATTACK : VIS_RELEASE;
|
|
||||||
this._aiLevel = this._aiLevel + (aiTarget - this._aiLevel) * k;
|
|
||||||
root.style.setProperty("--ai-audio-level", this._aiLevel.toFixed(3));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,12 @@
|
|||||||
|
root = true
|
||||||
|
|
||||||
|
[*]
|
||||||
|
charset = utf-8
|
||||||
|
end_of_line = lf
|
||||||
|
insert_final_newline = true
|
||||||
|
indent_style = space
|
||||||
|
indent_size = 2
|
||||||
|
trim_trailing_whitespace = true
|
||||||
|
|
||||||
|
[*.md]
|
||||||
|
trim_trailing_whitespace = false
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
# FastAPI target when running npm run dev independently. start_ui.sh sets this
|
||||||
|
# to its own backend using S2S_BACKEND_PORT (default 8421).
|
||||||
|
S2S_BACKEND_URL=http://127.0.0.1:8421
|
||||||
|
# Comma-separated hostnames for remote access, without scheme or port.
|
||||||
|
# With start_ui.sh this can also be set in the repository root .env.
|
||||||
|
S2S_DEV_ALLOWED_HOSTS=
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
# Logs
|
||||||
|
logs
|
||||||
|
*.log
|
||||||
|
npm-debug.log*
|
||||||
|
yarn-debug.log*
|
||||||
|
yarn-error.log*
|
||||||
|
pnpm-debug.log*
|
||||||
|
lerna-debug.log*
|
||||||
|
|
||||||
|
node_modules
|
||||||
|
dist
|
||||||
|
dist-ssr
|
||||||
|
*.local
|
||||||
|
|
||||||
|
# Editor directories and files
|
||||||
|
.vscode/*
|
||||||
|
!.vscode/extensions.json
|
||||||
|
.idea
|
||||||
|
.DS_Store
|
||||||
|
*.suo
|
||||||
|
*.ntvs*
|
||||||
|
*.njsproj
|
||||||
|
*.sln
|
||||||
|
*.sw?
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
dist/
|
||||||
|
coverage/
|
||||||
|
node_modules/
|
||||||
|
package-lock.json
|
||||||
@@ -0,0 +1,162 @@
|
|||||||
|
# Vocero React frontend
|
||||||
|
|
||||||
|
React 19, TypeScript, Tailwind CSS 4, and Vite 8 client for the existing
|
||||||
|
`s2s-client` FastAPI backend. This is a client-side application with React-owned
|
||||||
|
rendering and state, including the realtime WebSocket and audio integration.
|
||||||
|
|
||||||
|
## Run during development
|
||||||
|
|
||||||
|
From the repository root, run `./start_ui.sh`. Open `http://localhost:8420`.
|
||||||
|
Add `?debug=true` to access settings, tools, and custom RAG context. Existing
|
||||||
|
`phoneNumber` and `episode_id` query parameters are supported.
|
||||||
|
|
||||||
|
The launcher requires Python 3 with venv support, Node.js 22.18 or later on the
|
||||||
|
22.x line or Node.js 24+, npm, Bash 4.4+, `setsid` (util-linux), and `curl`.
|
||||||
|
It creates `.venv` and installs missing Python dependencies. It runs
|
||||||
|
`npm ci --include=dev` on first start and when either frontend package manifest
|
||||||
|
changes. Subsequent restarts reuse the installed dependencies.
|
||||||
|
|
||||||
|
| Server | Default address | Purpose |
|
||||||
|
| ------- | ---------------- | ---------------------------------------------- |
|
||||||
|
| Vite | `0.0.0.0:8420` | Public React UI, backend proxy, and hot reload |
|
||||||
|
| FastAPI | `127.0.0.1:8421` | Internal HTTP APIs and speech WebSocket relay |
|
||||||
|
|
||||||
|
`UI_PORT` preserves the existing public port override. `UI_HOST` defaults to
|
||||||
|
`0.0.0.0`; `S2S_BACKEND_PORT` changes the loopback backend port. The two ports
|
||||||
|
must differ. Occupied ports fail startup rather than silently selecting a new
|
||||||
|
endpoint. The launcher waits for FastAPI before starting Vite, stops both
|
||||||
|
process groups on Ctrl+C/SIGTERM, and exits with a failure if either server exits.
|
||||||
|
Uvicorn watches only `s2s-client`, avoiding frontend and dependency directories.
|
||||||
|
The Vite child explicitly uses `NODE_ENV=development`, including when a process
|
||||||
|
manager sets `NODE_ENV=production`, so React Fast Refresh remains enabled.
|
||||||
|
|
||||||
|
Vite serves React and Tailwind with hot updates. It proxies `/api`,
|
||||||
|
`/v1/realtime` (including WebSockets), and backend documentation routes
|
||||||
|
to FastAPI. The browser uses one origin on port `8420`, so there is no separate
|
||||||
|
API origin or development CORS setup. The proxy preserves the incoming Host
|
||||||
|
header because FastAPI uses it to construct the speech WebSocket URL.
|
||||||
|
|
||||||
|
Runtime UI flags, voices, reasoning mode, prompts, and the speech URL still come from FastAPI's
|
||||||
|
`/api/config`. A backend explicitly configured with `SPEECH_TO_SPEECH_URL`
|
||||||
|
continues to return that direct speech endpoint.
|
||||||
|
|
||||||
|
The launcher sources the root `.env`, retaining its precedence over shell
|
||||||
|
variables. The Vite configuration reads this project's environment files and
|
||||||
|
the inherited `S2S_*` settings. Do not copy the repository's backend secrets
|
||||||
|
into this directory or into `VITE_*`
|
||||||
|
variables, which are exposed to browser code.
|
||||||
|
|
||||||
|
## Staging with hot reload
|
||||||
|
|
||||||
|
PM2 can continue running the root `start_ui.sh` as its existing `s2s_ui` process.
|
||||||
|
Ensure the Node/npm versions above are on PM2's PATH and deploy `s2s-frontend`
|
||||||
|
alongside `s2s-client` and `docs`. The reverse proxy still targets port `8420`.
|
||||||
|
|
||||||
|
Set the exact public hostname in the root `.env`, without scheme or port:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
S2S_DEV_ALLOWED_HOSTS=voice-staging.example.com
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace the example with the existing staging hostname; use commas for multiple
|
||||||
|
hosts. IP addresses and localhost are accepted without this setting.
|
||||||
|
The HTTPS reverse proxy must preserve `Host`, `X-Forwarded-Host` and
|
||||||
|
`X-Forwarded-Proto`, and forward WebSocket upgrades for `/v1/realtime` and
|
||||||
|
`/__vite_hmr`. HMR uses the browser's public origin, including HTTPS/WSS, so no
|
||||||
|
second public port is needed. The browser needs HTTPS for microphone access
|
||||||
|
outside localhost. Restart `s2s_ui` after changing launcher or environment
|
||||||
|
settings; source edits use the reloaders. The speech service remains separately
|
||||||
|
managed.
|
||||||
|
|
||||||
|
## Run the servers separately
|
||||||
|
|
||||||
|
Start FastAPI on `127.0.0.1:8421` as described in the
|
||||||
|
[client usage guide](../docs/client/usage.md), then run:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd s2s-frontend
|
||||||
|
npm ci
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
Standalone Vite binds `127.0.0.1:8420`; add `-- --host 0.0.0.0` for remote
|
||||||
|
access. For a different backend, copy `.env.example` to `.env.local` and set
|
||||||
|
`S2S_BACKEND_URL`. The combined launcher overrides that setting to point at the
|
||||||
|
backend it starts. Do not start standalone Vite while the combined launcher
|
||||||
|
already owns the public port.
|
||||||
|
|
||||||
|
Component and CSS changes use Fast Refresh. Editing the session controller or
|
||||||
|
audio modules can require restarting a conversation or a page reload. Teardown
|
||||||
|
closes microphone tracks, worklets, sockets, queue tickets, timers, and pending
|
||||||
|
requests on session stop or page exit.
|
||||||
|
|
||||||
|
## Checks and commands
|
||||||
|
|
||||||
|
The scripts and code style follow `frontend-admin` conventions, with Vite and
|
||||||
|
React plugins replacing Next.js-specific tooling. Executable configuration
|
||||||
|
files use TypeScript; TypeScript's own project settings use JSON.
|
||||||
|
|
||||||
|
| Command | Purpose |
|
||||||
|
| ----------------------- | ----------------------------------------------------------- |
|
||||||
|
| `npm run dev` | Vite development server on port 8420, with a fixed port |
|
||||||
|
| `npm run check` | Formatting, strict typed ESLint, type checking, and tests |
|
||||||
|
| `npm run lint:fix` | Apply ESLint fixes |
|
||||||
|
| `npm run format` | Format source and sort Tailwind classes |
|
||||||
|
| `npm run typecheck` | Check application, tests, audio worklets, and configuration |
|
||||||
|
| `npm test` | Run Vitest and React Testing Library regression tests |
|
||||||
|
| `npm run test:watch` | Watch tests while editing |
|
||||||
|
| `npm run test:launcher` | Test launcher supervision and signal handling on Linux |
|
||||||
|
| `npm run build` | Typecheck and compile the app and worklets into `dist/` |
|
||||||
|
| `npm run preview` | Inspect the compiled output locally |
|
||||||
|
|
||||||
|
No build step is needed before `npm run dev`. `dist/` is generated and ignored
|
||||||
|
by Git. The launcher serves source through Vite; it does not use `dist/` or
|
||||||
|
`vite preview`.
|
||||||
|
|
||||||
|
Tests cover session initialization, response sequencing, cumulative transcripts,
|
||||||
|
interruptions, queue polling, admission deadlines, cancellation, microphone-permission cancellation, tool
|
||||||
|
results, external reasoning and `LOG_ONLY`, late-response cleanup, the idle
|
||||||
|
timeout, settings, RAG, profiles, and sanitized Markdown/export behavior.
|
||||||
|
Launcher tests start temporary local children to check readiness, environment
|
||||||
|
forwarding, failures, and shutdown of complete process groups.
|
||||||
|
Vitest handles Node 26's experimental web storage behavior internally.
|
||||||
|
|
||||||
|
## Source layout
|
||||||
|
|
||||||
|
- `src/components/`: voice controls, chat, dialogs, and caller profile views.
|
||||||
|
- `src/realtime/`: session lifecycle, observable application state, queue
|
||||||
|
handling, WebSocket protocol, tools, and external reasoning.
|
||||||
|
- `src/audio/`: TypeScript capture/playback worklets, audio graph, PCM codec,
|
||||||
|
and visualizer. Vite emits the worklets as separate JavaScript assets.
|
||||||
|
- `src/lib/`: validated backend contracts, settings, prompt context, profile
|
||||||
|
and report presentation, Markdown sanitization, and conversation exports.
|
||||||
|
- `config/backend-proxy.ts`: the shared HTTP/WebSocket development proxy.
|
||||||
|
- `src/styles.css`: Tailwind integration and React-specific layout adjustments.
|
||||||
|
- `src/styles/legacy-theme.css`: the existing visual styles, retained for
|
||||||
|
fidelity to both the white microphone and dark orb themes. The UI is ported
|
||||||
|
to React; the full design has not been rewritten as Tailwind utilities.
|
||||||
|
|
||||||
|
The current Spanish interface, local preference keys, session-only RAG,
|
||||||
|
Markdown replies, JSON/TXT exports, optional browser wake word, TALK/TALKFIRST
|
||||||
|
startup modes, and optional backend features are retained. Camera controls and
|
||||||
|
quick actions remain absent, matching their disabled state in the old UI.
|
||||||
|
|
||||||
|
Native speech stays at the current project's 16 kHz PCM rate. The backend
|
||||||
|
continues to own VAD, transcription, model inference, synthesis, upstream
|
||||||
|
credentials, and ApiPilot execution. Browser `SpeechSynthesis` is used only
|
||||||
|
when the configured external reasoning service returns an answer instead of
|
||||||
|
`log_only`.
|
||||||
|
|
||||||
|
Session startup waits for audio initialization and `session.created`, then sends
|
||||||
|
`session.update` before microphone audio or the initial greeting. The speech
|
||||||
|
server processes those frames in order and does not emit `session.updated` on
|
||||||
|
success. That acknowledgment is therefore optional; receiving it later does not
|
||||||
|
restart the greeting or reset an active response. The startup timeout still
|
||||||
|
closes the microphone and socket if the server never creates a session.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [Vite getting started](https://vite.dev/guide/)
|
||||||
|
- [Vite server and proxy options](https://vite.dev/config/server-options)
|
||||||
|
- [Vite features and asset handling](https://vite.dev/guide/features)
|
||||||
|
- [Tailwind's Vite integration](https://tailwindcss.com/docs/installation/using-vite)
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
import type { ProxyOptions } from "vite";
|
||||||
|
|
||||||
|
/** Preserve the public origin used by FastAPI to construct the speech WebSocket URL. */
|
||||||
|
export function createBackendProxy(target: string): Record<string, ProxyOptions> {
|
||||||
|
const http: ProxyOptions = { target, changeOrigin: false };
|
||||||
|
return {
|
||||||
|
"/api": http,
|
||||||
|
"/v1/realtime": { ...http, ws: true },
|
||||||
|
"/docs": http,
|
||||||
|
"/redoc": http,
|
||||||
|
"/openapi.json": http,
|
||||||
|
"/project": http,
|
||||||
|
"/public/docs.html": http,
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,187 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { spawn } from "node:child_process";
|
||||||
|
import { once } from "node:events";
|
||||||
|
import { copyFile, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||||
|
import { createServer } from "node:net";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { test, type TestContext } from "node:test";
|
||||||
|
import { setTimeout as delay } from "node:timers/promises";
|
||||||
|
|
||||||
|
// Exercise the real Bash supervisor with lightweight children, without installing
|
||||||
|
// packages, starting upstream services, or reading the checkout's private .env.
|
||||||
|
const childSource = `
|
||||||
|
import { createServer } from "node:http";
|
||||||
|
import { writeFileSync } from "node:fs";
|
||||||
|
import { spawn } from "node:child_process";
|
||||||
|
const role = process.argv[2] === "backend" ? "backend" : "frontend";
|
||||||
|
const root = process.env.S2S_TEST_ROOT;
|
||||||
|
const args = process.argv.slice(2);
|
||||||
|
writeFileSync(root + "/" + role + ".pid", String(process.pid));
|
||||||
|
writeFileSync(root + "/" + role + ".json", JSON.stringify({
|
||||||
|
args, target: process.env.S2S_BACKEND_URL, hosts: process.env.S2S_DEV_ALLOWED_HOSTS,
|
||||||
|
nodeEnv: process.env.NODE_ENV,
|
||||||
|
}));
|
||||||
|
if (process.env.S2S_TEST_FAIL === role) process.exit(37);
|
||||||
|
const worker = spawn(process.execPath, ["-e",
|
||||||
|
'const fs = require("node:fs"); process.on("SIGTERM", () => { fs.writeFileSync(process.argv[1], "yes"); process.exit(0); }); fs.writeFileSync(process.argv[2], "yes"); setInterval(() => {}, 1000);',
|
||||||
|
root + "/" + role + "-worker-stopped", root + "/" + role + "-worker-ready",
|
||||||
|
], { stdio: "ignore" });
|
||||||
|
process.on("SIGTERM", () => {
|
||||||
|
worker.once("exit", () => process.exit(0));
|
||||||
|
if (worker.exitCode !== null || worker.signalCode !== null) process.exit(0);
|
||||||
|
});
|
||||||
|
createServer((request, response) => {
|
||||||
|
response.setHeader("Content-Type", "application/json");
|
||||||
|
response.end("{}");
|
||||||
|
}).listen(Number(args[args.indexOf("--port") + 1]), "127.0.0.1");
|
||||||
|
`;
|
||||||
|
|
||||||
|
async function freePort() {
|
||||||
|
const server = createServer();
|
||||||
|
server.listen(0, "127.0.0.1");
|
||||||
|
await once(server, "listening");
|
||||||
|
const address = server.address();
|
||||||
|
assert.ok(address && typeof address === "object");
|
||||||
|
await new Promise<void>((resolve) =>
|
||||||
|
server.close(() => {
|
||||||
|
resolve();
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
return address.port;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function waitForFile(filename: string) {
|
||||||
|
for (let attempt = 0; attempt < 100; attempt++) {
|
||||||
|
try {
|
||||||
|
return await readFile(filename, "utf8");
|
||||||
|
} catch {
|
||||||
|
await delay(30);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assert.fail(`Timed out waiting for ${filename}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function launcher(context: TestContext, env: Record<string, string> = {}) {
|
||||||
|
const root = await mkdtemp(join(tmpdir(), "s2s-launcher-"));
|
||||||
|
const frontend = join(root, "s2s-frontend");
|
||||||
|
const modules = join(frontend, "node_modules");
|
||||||
|
await mkdir(join(root, ".venv/bin"), { recursive: true });
|
||||||
|
await mkdir(join(root, "s2s-client"));
|
||||||
|
await mkdir(join(modules, "vite/bin"), { recursive: true });
|
||||||
|
await copyFile(new URL("../../start_ui.sh", import.meta.url), join(root, "start_ui.sh"));
|
||||||
|
await writeFile(join(root, "s2s-client/requirements.txt"), "");
|
||||||
|
await writeFile(join(root, ".venv/bin/activate"), 'export PATH="$S2S_TEST_ROOT/.venv/bin:$PATH"\n');
|
||||||
|
await writeFile(
|
||||||
|
join(root, ".venv/bin/python"),
|
||||||
|
'#!/bin/bash\nif [ "$1" = "-c" ]; then exit 0; fi\nexec node "$S2S_TEST_ROOT/child.mjs" backend "$@"\n',
|
||||||
|
{ mode: 0o755 },
|
||||||
|
);
|
||||||
|
await writeFile(join(root, "child.mjs"), childSource);
|
||||||
|
await writeFile(join(modules, "vite/bin/vite.js"), 'import(process.env.S2S_TEST_ROOT + "/child.mjs");\n');
|
||||||
|
for (const manifest of ["package.json", "package-lock.json"]) {
|
||||||
|
await writeFile(join(frontend, manifest), "{}\n");
|
||||||
|
await copyFile(join(frontend, manifest), join(modules, `.s2s-${manifest}`));
|
||||||
|
}
|
||||||
|
const publicPort = await freePort();
|
||||||
|
let backendPort = await freePort();
|
||||||
|
while (backendPort === publicPort) {
|
||||||
|
backendPort = await freePort();
|
||||||
|
}
|
||||||
|
// The root .env retains the launcher's existing precedence over shell values.
|
||||||
|
await writeFile(join(root, ".env"), `S2S_DEV_ALLOWED_HOSTS=s2s.test\nS2S_BACKEND_PORT=${backendPort}\n`);
|
||||||
|
const child = spawn("bash", [join(root, "start_ui.sh")], {
|
||||||
|
cwd: tmpdir(),
|
||||||
|
env: {
|
||||||
|
...process.env,
|
||||||
|
UI_PORT: String(publicPort),
|
||||||
|
UI_HOST: "0.0.0.0",
|
||||||
|
NODE_ENV: "production",
|
||||||
|
S2S_BACKEND_PORT: "1",
|
||||||
|
S2S_BACKEND_URL: "http://wrong-target.invalid",
|
||||||
|
S2S_TEST_ROOT: root,
|
||||||
|
S2S_TEST_FAIL: "",
|
||||||
|
...env,
|
||||||
|
},
|
||||||
|
stdio: ["ignore", "pipe", "pipe"],
|
||||||
|
});
|
||||||
|
let output = "";
|
||||||
|
child.stdout.on("data", (chunk: Buffer) => {
|
||||||
|
output += chunk.toString();
|
||||||
|
});
|
||||||
|
child.stderr.on("data", (chunk: Buffer) => {
|
||||||
|
output += chunk.toString();
|
||||||
|
});
|
||||||
|
const exited = new Promise<number | null>((resolve) => child.once("close", resolve));
|
||||||
|
context.after(async () => {
|
||||||
|
child.kill("SIGTERM");
|
||||||
|
await exited;
|
||||||
|
await rm(root, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
return { root, child, exited, publicPort, backendPort, output: () => output };
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const signal of ["SIGINT", "SIGTERM"] as const) {
|
||||||
|
await test(
|
||||||
|
`keeps the public endpoint and stops both process groups on ${signal}`,
|
||||||
|
{ timeout: 10_000 },
|
||||||
|
async (context) => {
|
||||||
|
const run = await launcher(context);
|
||||||
|
for (const role of ["backend", "frontend"]) {
|
||||||
|
await waitForFile(join(run.root, `${role}-worker-ready`));
|
||||||
|
}
|
||||||
|
const metadata: unknown = JSON.parse(await readFile(join(run.root, "frontend.json"), "utf8"));
|
||||||
|
assert.deepEqual(metadata, {
|
||||||
|
args: ["--host", "0.0.0.0", "--port", String(run.publicPort), "--strictPort"],
|
||||||
|
target: `http://127.0.0.1:${run.backendPort}`,
|
||||||
|
hosts: "s2s.test",
|
||||||
|
nodeEnv: "development",
|
||||||
|
});
|
||||||
|
assert.equal(run.output().includes("Installing"), false);
|
||||||
|
run.child.kill(signal);
|
||||||
|
assert.equal(await run.exited, signal === "SIGINT" ? 130 : 143);
|
||||||
|
for (const role of ["backend", "frontend"]) {
|
||||||
|
assert.equal(await waitForFile(join(run.root, `${role}-worker-stopped`)), "yes");
|
||||||
|
}
|
||||||
|
for (const port of [run.publicPort, run.backendPort]) {
|
||||||
|
await assert.rejects(fetch(`http://127.0.0.1:${port}`));
|
||||||
|
}
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
await test(
|
||||||
|
"stops the backend if Vite exits and returns the failing exit code",
|
||||||
|
{ timeout: 10_000 },
|
||||||
|
async (context) => {
|
||||||
|
const run = await launcher(context, { S2S_TEST_FAIL: "frontend" });
|
||||||
|
assert.equal(await run.exited, 37, run.output());
|
||||||
|
assert.equal(await waitForFile(join(run.root, "backend-worker-stopped")), "yes");
|
||||||
|
await assert.rejects(fetch(`http://127.0.0.1:${run.backendPort}`));
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
await test("does not start Vite when the backend fails readiness", { timeout: 10_000 }, async (context) => {
|
||||||
|
const run = await launcher(context, { S2S_TEST_FAIL: "backend" });
|
||||||
|
assert.equal(await run.exited, 1, run.output());
|
||||||
|
await assert.rejects(readFile(join(run.root, "frontend.json")));
|
||||||
|
assert.match(run.output(), /FastAPI exited before it was ready/);
|
||||||
|
});
|
||||||
|
|
||||||
|
await test("stops Vite if the backend exits after startup", { timeout: 10_000 }, async (context) => {
|
||||||
|
const run = await launcher(context);
|
||||||
|
await waitForFile(join(run.root, "frontend-worker-ready"));
|
||||||
|
const backendPid = Number(await readFile(join(run.root, "backend.pid"), "utf8"));
|
||||||
|
assert.ok(Number.isInteger(backendPid) && backendPid > 0);
|
||||||
|
process.kill(-backendPid, "SIGTERM");
|
||||||
|
assert.equal(await run.exited, 1, run.output());
|
||||||
|
assert.equal(await waitForFile(join(run.root, "frontend-worker-stopped")), "yes");
|
||||||
|
await assert.rejects(fetch(`http://127.0.0.1:${run.publicPort}`));
|
||||||
|
});
|
||||||
|
|
||||||
|
await test("rejects an invalid public port before starting either server", { timeout: 10_000 }, async (context) => {
|
||||||
|
const run = await launcher(context, { UI_PORT: "65536" });
|
||||||
|
assert.equal(await run.exited, 1);
|
||||||
|
assert.match(run.output(), /UI_PORT must be a port/);
|
||||||
|
await assert.rejects(readFile(join(run.root, "backend.json")));
|
||||||
|
});
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
import js from "@eslint/js";
|
||||||
|
import { defineConfig, globalIgnores } from "eslint/config";
|
||||||
|
import prettier from "eslint-config-prettier/flat";
|
||||||
|
import reactHooks from "eslint-plugin-react-hooks";
|
||||||
|
import reactRefresh from "eslint-plugin-react-refresh";
|
||||||
|
import tseslint from "typescript-eslint";
|
||||||
|
|
||||||
|
export default defineConfig([
|
||||||
|
globalIgnores(["dist/**", "coverage/**", "node_modules/**"]),
|
||||||
|
{
|
||||||
|
files: ["**/*.{ts,tsx}"],
|
||||||
|
extends: [js.configs.recommended, tseslint.configs.strictTypeChecked, tseslint.configs.stylisticTypeChecked],
|
||||||
|
languageOptions: { parserOptions: { projectService: true, tsconfigRootDir: import.meta.dirname } },
|
||||||
|
rules: {
|
||||||
|
"@typescript-eslint/consistent-type-imports": ["error", { fixStyle: "inline-type-imports" }],
|
||||||
|
"@typescript-eslint/no-import-type-side-effects": "error",
|
||||||
|
"@typescript-eslint/no-unnecessary-condition": "off",
|
||||||
|
"@typescript-eslint/restrict-template-expressions": ["error", { allowNumber: true }],
|
||||||
|
"@typescript-eslint/prefer-nullish-coalescing": ["error", { ignorePrimitives: { string: true } }],
|
||||||
|
"no-console": ["error", { allow: ["warn", "error"] }],
|
||||||
|
"no-debugger": "error",
|
||||||
|
curly: ["error", "all"],
|
||||||
|
eqeqeq: ["error", "always", { null: "ignore" }],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
files: ["src/**/*.{ts,tsx}"],
|
||||||
|
plugins: { "react-hooks": { rules: reactHooks.rules }, "react-refresh": reactRefresh },
|
||||||
|
rules: {
|
||||||
|
...reactHooks.configs.recommended.rules,
|
||||||
|
"react-hooks/exhaustive-deps": "error",
|
||||||
|
"react-refresh/only-export-components": ["error", { allowConstantExport: true }],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
prettier,
|
||||||
|
]);
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="es">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
|
||||||
|
<meta name="theme-color" content="#07080d" />
|
||||||
|
<title>Vocero · Conversación de voz</title>
|
||||||
|
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
||||||
|
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
|
||||||
|
<link
|
||||||
|
href="https://fonts.googleapis.com/css2?family=Geist+Mono:wght@400;500;600&family=Inter:wght@400;500;600;700;800&display=swap"
|
||||||
|
rel="stylesheet"
|
||||||
|
/>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div id="root"></div>
|
||||||
|
<script type="module" src="/src/main.tsx"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
Generated
+4609
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,55 @@
|
|||||||
|
{
|
||||||
|
"name": "s2s-frontend",
|
||||||
|
"private": true,
|
||||||
|
"version": "0.0.0",
|
||||||
|
"type": "module",
|
||||||
|
"engines": {
|
||||||
|
"node": "^22.18.0 || >=24.0.0"
|
||||||
|
},
|
||||||
|
"scripts": {
|
||||||
|
"dev": "vite",
|
||||||
|
"build": "npm run typecheck && vite build",
|
||||||
|
"preview": "vite preview",
|
||||||
|
"lint": "eslint . --cache --cache-strategy content --max-warnings 0",
|
||||||
|
"lint:fix": "npm run lint -- --fix",
|
||||||
|
"format": "prettier --write .",
|
||||||
|
"format:check": "prettier --check .",
|
||||||
|
"typecheck": "tsc -b",
|
||||||
|
"test": "vitest run",
|
||||||
|
"test:watch": "vitest",
|
||||||
|
"test:launcher": "node --test config/launcher.integration.ts",
|
||||||
|
"check": "npm run format:check && npm run lint && npm run typecheck && npm test && npm run test:launcher"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"dompurify": "^3.4.16",
|
||||||
|
"lucide-react": "^1.52.0",
|
||||||
|
"marked": "^18.1.0",
|
||||||
|
"react": "^19.2.8",
|
||||||
|
"react-dom": "^19.2.8",
|
||||||
|
"zod": "^4.6.5"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@eslint/js": "^10.0.1",
|
||||||
|
"@tailwindcss/vite": "^4.3.3",
|
||||||
|
"@testing-library/jest-dom": "^7.0.1",
|
||||||
|
"@testing-library/react": "^16.3.3",
|
||||||
|
"@testing-library/user-event": "^14.6.7",
|
||||||
|
"@types/node": "^24.13.3",
|
||||||
|
"@types/react": "^19.2.18",
|
||||||
|
"@types/react-dom": "^19.2.7",
|
||||||
|
"@vitejs/plugin-react": "^6.1.1",
|
||||||
|
"eslint": "^10.12.0",
|
||||||
|
"eslint-config-prettier": "^10.1.8",
|
||||||
|
"eslint-plugin-react-hooks": "^7.1.1",
|
||||||
|
"eslint-plugin-react-refresh": "^0.5.7",
|
||||||
|
"jiti": "^2.7.0",
|
||||||
|
"jsdom": "^30.1.2",
|
||||||
|
"prettier": "^3.9.9",
|
||||||
|
"prettier-plugin-tailwindcss": "^0.8.1",
|
||||||
|
"tailwindcss": "^4.3.3",
|
||||||
|
"typescript": "~6.0.2",
|
||||||
|
"typescript-eslint": "^8.71.1",
|
||||||
|
"vite": "^8.3.0",
|
||||||
|
"vitest": "^5.0.3"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
import type { Config } from "prettier";
|
||||||
|
|
||||||
|
export default {
|
||||||
|
arrowParens: "always",
|
||||||
|
endOfLine: "lf",
|
||||||
|
plugins: ["prettier-plugin-tailwindcss"],
|
||||||
|
printWidth: 120,
|
||||||
|
semi: true,
|
||||||
|
singleQuote: false,
|
||||||
|
tabWidth: 2,
|
||||||
|
tailwindStylesheet: "./src/styles.css",
|
||||||
|
trailingComma: "all",
|
||||||
|
} satisfies Config & { tailwindStylesheet: string };
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
import { fireEvent, render, screen, waitFor, within } from "@testing-library/react";
|
||||||
|
import userEvent from "@testing-library/user-event";
|
||||||
|
import { StrictMode } from "react";
|
||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
|
||||||
|
import App from "./App";
|
||||||
|
|
||||||
|
const bootstrap = { theme: "white", s2sUrl: "ws://localhost:8420/v1/realtime", apipilot: true };
|
||||||
|
beforeEach(() => {
|
||||||
|
window.history.replaceState({}, "", "/?debug=true&phoneNumber=1234");
|
||||||
|
vi.spyOn(HTMLDialogElement.prototype, "showModal").mockImplementation(function (this: HTMLDialogElement) {
|
||||||
|
this.open = true;
|
||||||
|
});
|
||||||
|
vi.spyOn(HTMLDialogElement.prototype, "close").mockImplementation(function (this: HTMLDialogElement) {
|
||||||
|
this.open = false;
|
||||||
|
});
|
||||||
|
vi.stubGlobal(
|
||||||
|
"fetch",
|
||||||
|
vi.fn((input: string) => {
|
||||||
|
const path = new URL(input, window.location.origin).pathname;
|
||||||
|
const data: Record<string, unknown> = {
|
||||||
|
"/api/config": bootstrap,
|
||||||
|
"/api/user/phone": { result: { name: "Ana Vargas", contact: { emailAddress: "ana@example.test" } } },
|
||||||
|
"/api/user/reports": {
|
||||||
|
count: 1,
|
||||||
|
entries: [{ sourceScheduleLabel: null, date: null }],
|
||||||
|
hints: [],
|
||||||
|
truncated: true,
|
||||||
|
note: "Lista abreviada",
|
||||||
|
},
|
||||||
|
"/api/apipilot/tools": {
|
||||||
|
success: true,
|
||||||
|
data: { tools: [{ name: "get_reports", description: "Consulta tus reportes", parameters: {} }] },
|
||||||
|
},
|
||||||
|
};
|
||||||
|
return Promise.resolve(new Response(JSON.stringify(data[path] ?? {})));
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
afterEach(() => {
|
||||||
|
vi.unstubAllGlobals();
|
||||||
|
window.history.replaceState({}, "", "/");
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("React application", () => {
|
||||||
|
it("loads the configured theme and persists voice settings across remounts under StrictMode", async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
const view = render(
|
||||||
|
<StrictMode>
|
||||||
|
<App />
|
||||||
|
</StrictMode>,
|
||||||
|
);
|
||||||
|
await user.click(await screen.findByRole("button", { name: "Configuración" }));
|
||||||
|
expect(document.documentElement.dataset.theme).toBe("white");
|
||||||
|
await user.selectOptions(screen.getByRole("combobox"), "Serena");
|
||||||
|
expect(localStorage.getItem("s2s.ws.voice")).toBe("Serena");
|
||||||
|
await user.click(screen.getByRole("button", { name: "Guardar" }));
|
||||||
|
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
|
||||||
|
view.unmount();
|
||||||
|
render(
|
||||||
|
<StrictMode>
|
||||||
|
<App />
|
||||||
|
</StrictMode>,
|
||||||
|
);
|
||||||
|
await user.click(await screen.findByRole("button", { name: "Configuración" }));
|
||||||
|
expect(screen.getByRole("combobox")).toHaveValue("Serena");
|
||||||
|
});
|
||||||
|
it("saves and clears session-only RAG context and supports keyboard dialog dismissal", async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
render(<App />);
|
||||||
|
await user.click(await screen.findByRole("button", { name: "Contexto RAG personalizado" }));
|
||||||
|
await user.type(screen.getByRole("textbox"), "Datos de prueba para esta conversación");
|
||||||
|
await user.click(screen.getByRole("button", { name: "Enviar a la sesión" }));
|
||||||
|
expect(within(screen.getByRole("dialog")).getByRole("status")).toHaveTextContent("Contexto actualizado");
|
||||||
|
fireEvent(screen.getByRole("dialog"), new Event("cancel", { bubbles: true }));
|
||||||
|
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
|
||||||
|
await user.click(screen.getByRole("button", { name: "Contexto RAG personalizado" }));
|
||||||
|
expect(screen.getByRole("textbox")).toHaveValue("Datos de prueba para esta conversación");
|
||||||
|
await user.click(screen.getByRole("button", { name: "Vaciar" }));
|
||||||
|
expect(screen.getByRole("textbox")).toHaveValue("");
|
||||||
|
});
|
||||||
|
it("shows nested profile fields and discovers remote tools after bootstrap", async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
render(<App />);
|
||||||
|
await user.click(await screen.findByRole("button", { name: "Ver datos de Ana Vargas" }));
|
||||||
|
expect(within(screen.getByRole("dialog")).getByRole("link", { name: "ana@example.test" })).toHaveAttribute(
|
||||||
|
"href",
|
||||||
|
"mailto:ana@example.test",
|
||||||
|
);
|
||||||
|
await user.click(screen.getByRole("button", { name: "Cerrar" }));
|
||||||
|
await user.click(screen.getByRole("button", { name: "Herramientas" }));
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(screen.getByText("get_reports")).toBeInTheDocument();
|
||||||
|
});
|
||||||
|
expect(screen.getByText("Consulta tus reportes")).toBeInTheDocument();
|
||||||
|
});
|
||||||
|
it("keeps debug controls out of the normal user screen", async () => {
|
||||||
|
window.history.replaceState({}, "", "/");
|
||||||
|
render(<App />);
|
||||||
|
expect(await screen.findByRole("button", { name: "Iniciar conversación de voz" })).toBeEnabled();
|
||||||
|
expect(screen.queryByRole("button", { name: "Configuración" })).not.toBeInTheDocument();
|
||||||
|
expect(screen.getByText("Aún no hay mensajes")).toBeInTheDocument();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,294 @@
|
|||||||
|
import { BookOpen, ChartNoAxesColumn, ChevronRight, Info, PanelLeft, Settings, Wrench, X } from "lucide-react";
|
||||||
|
import { useEffect, useLayoutEffect, useState, useSyncExternalStore } from "react";
|
||||||
|
|
||||||
|
import { Bubbles } from "./components/Bubbles";
|
||||||
|
import { Conversation } from "./components/Conversation";
|
||||||
|
import { AboutDialog, ProfileDialog, RagDialog, SettingsDialog, ToolsDialog } from "./components/SettingsDialogs";
|
||||||
|
import { VoiceStage } from "./components/VoiceStage";
|
||||||
|
import { useWakeWord } from "./hooks/use-wake-word";
|
||||||
|
import { initials, profileName } from "./lib/context";
|
||||||
|
import { reportSuggestions } from "./lib/reports";
|
||||||
|
import { isDebug } from "./lib/settings";
|
||||||
|
import { ACTIVE_STATES, VoiceController } from "./realtime/controller";
|
||||||
|
|
||||||
|
type Modal = "settings" | "tools" | "rag" | "profile" | "about" | null;
|
||||||
|
export default function App() {
|
||||||
|
const [controller] = useState(() => new VoiceController());
|
||||||
|
const state = useSyncExternalStore(controller.subscribe, controller.getSnapshot);
|
||||||
|
const [modal, setModal] = useState<Modal>(null);
|
||||||
|
const [reportsOpen, setReportsOpen] = useState(false);
|
||||||
|
const debug = isDebug();
|
||||||
|
const white = state.config.theme === "white";
|
||||||
|
const name = profileName(state.profile);
|
||||||
|
const hasReports = debug && !white && !!state.reports?.count;
|
||||||
|
useEffect(() => {
|
||||||
|
const abort = new AbortController();
|
||||||
|
void controller.initialize(abort.signal);
|
||||||
|
const leave = () => {
|
||||||
|
void controller.stop();
|
||||||
|
};
|
||||||
|
window.addEventListener("pagehide", leave);
|
||||||
|
return () => {
|
||||||
|
abort.abort();
|
||||||
|
window.removeEventListener("pagehide", leave);
|
||||||
|
void controller.stop();
|
||||||
|
};
|
||||||
|
}, [controller]);
|
||||||
|
useLayoutEffect(() => {
|
||||||
|
if (!state.loading) {
|
||||||
|
document.documentElement.dataset.theme = white ? "white" : "dark";
|
||||||
|
}
|
||||||
|
}, [state.loading, white]);
|
||||||
|
useEffect(() => {
|
||||||
|
if (state.loading) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const timer = setTimeout(() => {
|
||||||
|
controller.offerAutoStart();
|
||||||
|
}, 2000);
|
||||||
|
return () => {
|
||||||
|
clearTimeout(timer);
|
||||||
|
};
|
||||||
|
}, [state.loading, controller]);
|
||||||
|
useWakeWord(!state.loading && !ACTIVE_STATES.has(state.status) && !modal && !state.unlock, controller);
|
||||||
|
useEffect(() => {
|
||||||
|
if (!reportsOpen) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const escape = (event: KeyboardEvent) => {
|
||||||
|
if (event.key === "Escape") {
|
||||||
|
setReportsOpen(false);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
window.addEventListener("keydown", escape);
|
||||||
|
return () => {
|
||||||
|
window.removeEventListener("keydown", escape);
|
||||||
|
};
|
||||||
|
}, [reportsOpen]);
|
||||||
|
const close = () => {
|
||||||
|
setModal(null);
|
||||||
|
};
|
||||||
|
if (state.loading) {
|
||||||
|
return (
|
||||||
|
<main className="loading-screen" role="status">
|
||||||
|
Preparando Vocero…
|
||||||
|
</main>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
{state.unlock && (
|
||||||
|
<button
|
||||||
|
className="talk-unlocker"
|
||||||
|
type="button"
|
||||||
|
onClick={() => {
|
||||||
|
void controller.start();
|
||||||
|
}}
|
||||||
|
aria-label="Empezar a hablar"
|
||||||
|
>
|
||||||
|
<span className="talk-unlocker-hint">
|
||||||
|
<span className="talk-unlocker-spinner" aria-hidden="true" />
|
||||||
|
<span className="talk-unlocker-title">Preparando Vocero…</span>
|
||||||
|
<span className="talk-unlocker-sub">Toca en cualquier lugar para empezar a hablar</span>
|
||||||
|
</span>
|
||||||
|
</button>
|
||||||
|
)}
|
||||||
|
<div id="app" className={hasReports ? "has-reports" : ""}>
|
||||||
|
<header className="topbar">
|
||||||
|
<div className="brand">
|
||||||
|
<div className="ident">
|
||||||
|
<div className="ident-head">
|
||||||
|
<h1 className="m-0 leading-none">
|
||||||
|
<a className="ident-title" href={window.location.pathname + window.location.search}>
|
||||||
|
Vocero
|
||||||
|
</a>
|
||||||
|
</h1>
|
||||||
|
{white && <p className="ident-subtitle">Agente conversacional</p>}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{white && name && (
|
||||||
|
<button
|
||||||
|
id="ident-user-name"
|
||||||
|
className="ident-user-name"
|
||||||
|
type="button"
|
||||||
|
aria-label={`Ver datos de ${name}`}
|
||||||
|
aria-haspopup="dialog"
|
||||||
|
onClick={() => {
|
||||||
|
setModal("profile");
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{initials(name)}
|
||||||
|
</button>
|
||||||
|
)}
|
||||||
|
{hasReports && (
|
||||||
|
<button
|
||||||
|
id="reports-sidebar-toggle"
|
||||||
|
className="icon-btn"
|
||||||
|
type="button"
|
||||||
|
aria-label="Abrir panel de reportes"
|
||||||
|
aria-expanded={reportsOpen}
|
||||||
|
aria-controls="reports-sidebar"
|
||||||
|
onClick={() => {
|
||||||
|
setReportsOpen(!reportsOpen);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<PanelLeft size={18} />
|
||||||
|
<span id="reports-sidebar-badge" className="chat-badge" data-count={state.reports?.count}>
|
||||||
|
{state.reports?.count}
|
||||||
|
</span>
|
||||||
|
</button>
|
||||||
|
)}
|
||||||
|
{debug && (
|
||||||
|
<div className="topbar-right" data-debug-enabled="true">
|
||||||
|
<button
|
||||||
|
className="icon-btn"
|
||||||
|
type="button"
|
||||||
|
aria-label="Acerca de"
|
||||||
|
onClick={() => {
|
||||||
|
setModal("about");
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Info size={18} />
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
className="icon-btn"
|
||||||
|
type="button"
|
||||||
|
aria-label="Herramientas"
|
||||||
|
onClick={() => {
|
||||||
|
setModal("tools");
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Wrench size={18} />
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
className="icon-btn"
|
||||||
|
type="button"
|
||||||
|
aria-label="Contexto RAG personalizado"
|
||||||
|
onClick={() => {
|
||||||
|
setModal("rag");
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<BookOpen size={18} />
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
className="icon-btn"
|
||||||
|
type="button"
|
||||||
|
aria-label="Configuración"
|
||||||
|
onClick={() => {
|
||||||
|
setModal("settings");
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Settings size={18} />
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</header>
|
||||||
|
<nav className="feedback-navigation" aria-label="Comentarios">
|
||||||
|
<a className="feedback-link" href="https://tally.so/r/pb8LWq" target="_blank" rel="noopener noreferrer">
|
||||||
|
Enviar comentarios
|
||||||
|
</a>
|
||||||
|
</nav>
|
||||||
|
{white && (
|
||||||
|
<div className="status-card" aria-label="Estado de la sesión">
|
||||||
|
<div className="status-card-icon" aria-hidden="true">
|
||||||
|
<ChartNoAxesColumn size={24} />
|
||||||
|
</div>
|
||||||
|
<div className="status-card-text">
|
||||||
|
<h2 className="status-card-title">Listo para conversar</h2>
|
||||||
|
<p className="status-card-sub">Informes, análisis, datos y mucho más.</p>
|
||||||
|
</div>
|
||||||
|
<button
|
||||||
|
className="status-card-arrow icon-btn"
|
||||||
|
type="button"
|
||||||
|
aria-label="Ver más detalles"
|
||||||
|
onClick={() => {
|
||||||
|
setModal("about");
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<ChevronRight size={22} />
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{state.error && (
|
||||||
|
<div className="app-error" role="alert">
|
||||||
|
<span>{state.error}</span>
|
||||||
|
<button
|
||||||
|
className="icon-btn shrink-0"
|
||||||
|
type="button"
|
||||||
|
aria-label="Cerrar aviso"
|
||||||
|
onClick={() => {
|
||||||
|
controller.dismissError();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<X size={16} />
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
<VoiceStage state={state} controller={controller} />
|
||||||
|
<Conversation messages={state.messages} userName={name || "Usuario"} status={state.status} white={white} />
|
||||||
|
</div>
|
||||||
|
{!white && ACTIVE_STATES.has(state.status) && <Bubbles messages={state.messages} userName={name || "Usuario"} />}
|
||||||
|
{hasReports && (
|
||||||
|
<aside
|
||||||
|
id="reports-sidebar"
|
||||||
|
className={`reports-sidebar ${reportsOpen ? "open" : ""}`}
|
||||||
|
aria-label="Reportes programados"
|
||||||
|
>
|
||||||
|
<button
|
||||||
|
className="reports-sidebar-backdrop"
|
||||||
|
type="button"
|
||||||
|
aria-label="Cerrar panel de reportes"
|
||||||
|
tabIndex={reportsOpen ? 0 : -1}
|
||||||
|
onClick={() => {
|
||||||
|
setReportsOpen(false);
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<div className="reports-sidebar-inner">
|
||||||
|
<header className="reports-sidebar-header">
|
||||||
|
<h2>Reportes</h2>
|
||||||
|
<button
|
||||||
|
id="reports-sidebar-close"
|
||||||
|
className="icon-btn"
|
||||||
|
type="button"
|
||||||
|
aria-label="Cerrar reportes"
|
||||||
|
onClick={() => {
|
||||||
|
setReportsOpen(false);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<X size={18} />
|
||||||
|
</button>
|
||||||
|
</header>
|
||||||
|
<div className="reports-hint">
|
||||||
|
<p>{state.reports?.count} reportes disponibles</p>
|
||||||
|
<ul>
|
||||||
|
{state.reports?.hints.map((hint, index) => (
|
||||||
|
<li key={index}>{hint}</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
{state.reports?.note && <p>{state.reports.note}</p>}
|
||||||
|
</div>
|
||||||
|
{state.reports && (
|
||||||
|
<div className="reports-suggestions">
|
||||||
|
<h3 className="reports-suggestions-title">Puedes preguntar</h3>
|
||||||
|
<ul className="reports-suggestions-list">
|
||||||
|
{reportSuggestions(state.reports).map((suggestion) => (
|
||||||
|
<li key={suggestion}>{suggestion}</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</aside>
|
||||||
|
)}
|
||||||
|
{modal === "settings" && <SettingsDialog state={state} controller={controller} onClose={close} />}
|
||||||
|
{modal === "tools" && <ToolsDialog state={state} controller={controller} onClose={close} />}
|
||||||
|
{modal === "rag" && <RagDialog state={state} controller={controller} onClose={close} />}
|
||||||
|
{modal === "profile" && <ProfileDialog state={state} onClose={close} />}
|
||||||
|
{modal === "about" && <AboutDialog onClose={close} />}
|
||||||
|
<footer className="app-footer" aria-label="Versión de la aplicación">
|
||||||
|
<span className="app-footer-text">Azurian Vocero en tiempo real ver. Beta 2.1</span>
|
||||||
|
</footer>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,108 @@
|
|||||||
|
type PlaybackMessage =
|
||||||
|
{ kind: "config"; inputRate: number } | { kind: "audio"; samples: Float32Array } | { kind: "clear" };
|
||||||
|
|
||||||
|
class AudioPlaybackProcessor extends AudioWorkletProcessor {
|
||||||
|
private rate = 16000;
|
||||||
|
private queue: Float32Array[] = [];
|
||||||
|
private offset = 0;
|
||||||
|
private fraction = 0;
|
||||||
|
private playing = false;
|
||||||
|
private fadeIn = 0;
|
||||||
|
private fadeOut = 0;
|
||||||
|
private lastSample = 0;
|
||||||
|
|
||||||
|
constructor() {
|
||||||
|
super();
|
||||||
|
this.port.onmessage = (event: MessageEvent<PlaybackMessage>) => {
|
||||||
|
const data = event.data;
|
||||||
|
switch (data.kind) {
|
||||||
|
case "config":
|
||||||
|
if (data.inputRate > 0) {
|
||||||
|
this.rate = data.inputRate;
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
case "audio":
|
||||||
|
if (data.samples.length) {
|
||||||
|
this.queue.push(data.samples);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
case "clear":
|
||||||
|
this.queue = [];
|
||||||
|
this.offset = 0;
|
||||||
|
this.fraction = 0;
|
||||||
|
this.fadeOut = 32;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
private bufferedMs() {
|
||||||
|
return (this.queue.reduce((sum, buffer) => sum + buffer.length, -this.offset) / this.rate) * 1000;
|
||||||
|
}
|
||||||
|
private read(): number | null {
|
||||||
|
const head = this.queue[0];
|
||||||
|
if (!head) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
const a = head[this.offset] ?? 0;
|
||||||
|
const b = head[this.offset + 1] ?? this.queue[1]?.[0] ?? a;
|
||||||
|
return a + (b - a) * this.fraction;
|
||||||
|
}
|
||||||
|
private advance() {
|
||||||
|
this.fraction += this.rate / sampleRate;
|
||||||
|
while (this.fraction >= 1) {
|
||||||
|
this.fraction--;
|
||||||
|
this.offset++;
|
||||||
|
}
|
||||||
|
while (this.queue[0] && this.offset >= this.queue[0].length) {
|
||||||
|
this.offset -= this.queue[0].length;
|
||||||
|
this.queue.shift();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
process(_inputs: Float32Array[][], outputs: Float32Array[][]) {
|
||||||
|
const channels = outputs[0];
|
||||||
|
const output = channels?.[0];
|
||||||
|
if (!output) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
for (let index = 0; index < output.length; index++) {
|
||||||
|
let value = 0;
|
||||||
|
if (!this.playing && this.queue.length && this.bufferedMs() >= 90) {
|
||||||
|
this.playing = true;
|
||||||
|
this.fadeIn = 32;
|
||||||
|
this.fadeOut = 0;
|
||||||
|
}
|
||||||
|
if (this.playing) {
|
||||||
|
const sample = this.read();
|
||||||
|
if (sample === null) {
|
||||||
|
value = this.lastSample * (1 - 1 / 32);
|
||||||
|
this.lastSample = value;
|
||||||
|
if (Math.abs(value) < 1e-4) {
|
||||||
|
this.playing = false;
|
||||||
|
this.lastSample = 0;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
value = sample;
|
||||||
|
this.lastSample = sample;
|
||||||
|
this.advance();
|
||||||
|
}
|
||||||
|
if (this.fadeIn > 0) {
|
||||||
|
value *= 1 - this.fadeIn-- / 32;
|
||||||
|
}
|
||||||
|
if (this.fadeOut > 0) {
|
||||||
|
value *= this.fadeOut-- / 32;
|
||||||
|
if (!this.fadeOut) {
|
||||||
|
this.playing = false;
|
||||||
|
this.lastSample = 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
output[index] = value;
|
||||||
|
const stereo = channels?.[1];
|
||||||
|
if (stereo) {
|
||||||
|
stereo[index] = value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
registerProcessor("audio-playback", AudioPlaybackProcessor);
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
export function encodePcm(buffer: ArrayBuffer) {
|
||||||
|
const bytes = new Uint8Array(buffer);
|
||||||
|
let binary = "";
|
||||||
|
for (let index = 0; index < bytes.length; index += 0x8000) {
|
||||||
|
binary += String.fromCharCode(...bytes.subarray(index, index + 0x8000));
|
||||||
|
}
|
||||||
|
return btoa(binary);
|
||||||
|
}
|
||||||
|
export function decodePcm(base64: string) {
|
||||||
|
const binary = atob(base64);
|
||||||
|
const buffer = new ArrayBuffer(binary.length);
|
||||||
|
const bytes = new Uint8Array(buffer);
|
||||||
|
for (let index = 0; index < binary.length; index++) {
|
||||||
|
bytes[index] = binary.charCodeAt(index);
|
||||||
|
}
|
||||||
|
const view = new DataView(buffer);
|
||||||
|
const samples = new Float32Array(Math.floor(binary.length / 2));
|
||||||
|
for (let index = 0; index < samples.length; index++) {
|
||||||
|
const value = view.getInt16(index * 2, true);
|
||||||
|
samples[index] = value < 0 ? value / 0x8000 : value / 0x7fff;
|
||||||
|
}
|
||||||
|
return samples;
|
||||||
|
}
|
||||||
@@ -0,0 +1,149 @@
|
|||||||
|
import type { NoiseGate } from "../lib/contracts";
|
||||||
|
import playbackUrl from "./audio-playback.worklet.ts?worker&url";
|
||||||
|
import { decodePcm } from "./codec";
|
||||||
|
import captureUrl from "./mic-capture.worklet.ts?worker&url";
|
||||||
|
|
||||||
|
const BANDS = [2, 5, 9, 16, 28, 52];
|
||||||
|
export const MIC_CONSTRAINTS: MediaStreamConstraints = {
|
||||||
|
audio: { echoCancellation: true, noiseSuppression: true, autoGainControl: true },
|
||||||
|
video: false,
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Construct inside the click handler, before awaiting microphone permission. */
|
||||||
|
export function createAudioContext() {
|
||||||
|
const context = new AudioContext({ latencyHint: "interactive" });
|
||||||
|
void context.resume().catch(() => undefined);
|
||||||
|
return context;
|
||||||
|
}
|
||||||
|
|
||||||
|
export class AudioEngine {
|
||||||
|
private readonly context: AudioContext;
|
||||||
|
private stream: MediaStream | null = null;
|
||||||
|
private capture: AudioWorkletNode | null = null;
|
||||||
|
private playback: AudioWorkletNode | null = null;
|
||||||
|
private nodes: AudioNode[] = [];
|
||||||
|
private frame = 0;
|
||||||
|
private closed = false;
|
||||||
|
private gate: NoiseGate;
|
||||||
|
private muted = false;
|
||||||
|
|
||||||
|
constructor(context: AudioContext, gate: NoiseGate) {
|
||||||
|
this.context = context;
|
||||||
|
this.gate = gate;
|
||||||
|
}
|
||||||
|
|
||||||
|
async start(onChunk: (chunk: ArrayBuffer) => void, onLevel: (rms: number) => void, speaking: () => boolean) {
|
||||||
|
const stream = await navigator.mediaDevices.getUserMedia(MIC_CONSTRAINTS);
|
||||||
|
if (this.closed) {
|
||||||
|
stream.getTracks().forEach((track) => {
|
||||||
|
track.stop();
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.stream = stream;
|
||||||
|
await Promise.all([
|
||||||
|
this.context.audioWorklet.addModule(captureUrl),
|
||||||
|
this.context.audioWorklet.addModule(playbackUrl),
|
||||||
|
]);
|
||||||
|
if (this.closed) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const capture = new AudioWorkletNode(this.context, "mic-capture", { numberOfInputs: 1, numberOfOutputs: 0 });
|
||||||
|
const playback = new AudioWorkletNode(this.context, "audio-playback", {
|
||||||
|
numberOfInputs: 0,
|
||||||
|
numberOfOutputs: 1,
|
||||||
|
outputChannelCount: [1],
|
||||||
|
});
|
||||||
|
// The checked-in speech pipeline emits native 16 kHz PCM, despite older 24 kHz comments.
|
||||||
|
playback.port.postMessage({ kind: "config", inputRate: 16000 });
|
||||||
|
this.capture = capture;
|
||||||
|
this.playback = playback;
|
||||||
|
this.setGate(this.gate);
|
||||||
|
this.setMuted(this.muted);
|
||||||
|
capture.port.onmessage = (event: MessageEvent<ArrayBuffer | { kind: "level"; rms: number }>) => {
|
||||||
|
if (this.closed) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (event.data instanceof ArrayBuffer) {
|
||||||
|
onChunk(event.data);
|
||||||
|
} else {
|
||||||
|
document.documentElement.style.setProperty("--audio-level", String(Math.min(1, event.data.rms * 5)));
|
||||||
|
onLevel(event.data.rms);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
const input = this.context.createMediaStreamSource(stream);
|
||||||
|
const micAnalyser = this.context.createAnalyser();
|
||||||
|
const outAnalyser = this.context.createAnalyser();
|
||||||
|
micAnalyser.fftSize = outAnalyser.fftSize = 256;
|
||||||
|
micAnalyser.smoothingTimeConstant = 0;
|
||||||
|
outAnalyser.smoothingTimeConstant = 0.3;
|
||||||
|
input.connect(capture);
|
||||||
|
input.connect(micAnalyser);
|
||||||
|
playback.connect(outAnalyser);
|
||||||
|
outAnalyser.connect(this.context.destination);
|
||||||
|
this.nodes = [input, capture, playback, micAnalyser, outAnalyser];
|
||||||
|
const buffer = new Uint8Array(micAnalyser.frequencyBinCount);
|
||||||
|
const levels = new Float32Array(5);
|
||||||
|
const tick = () => {
|
||||||
|
const source = speaking() ? outAnalyser : micAnalyser;
|
||||||
|
source.getByteFrequencyData(buffer);
|
||||||
|
for (let band = 0; band < 5; band++) {
|
||||||
|
const lo = BANDS[band] ?? 0;
|
||||||
|
const hi = BANDS[band + 1] ?? 52;
|
||||||
|
let sum = 0;
|
||||||
|
for (let index = lo; index < hi; index++) {
|
||||||
|
sum += buffer[index] ?? 0;
|
||||||
|
}
|
||||||
|
const target = sum / ((hi - lo) * 255);
|
||||||
|
const previous = levels[band] ?? 0;
|
||||||
|
levels[band] = previous + (target - previous) * (target > previous ? 0.6 : 0.18);
|
||||||
|
document.documentElement.style.setProperty(`--bar${band}`, String(levels[band]));
|
||||||
|
}
|
||||||
|
outAnalyser.getByteFrequencyData(buffer);
|
||||||
|
document.documentElement.style.setProperty("--ai-audio-level", String(Math.max(...buffer.subarray(0, 52)) / 255));
|
||||||
|
this.frame = requestAnimationFrame(tick);
|
||||||
|
};
|
||||||
|
this.frame = requestAnimationFrame(tick);
|
||||||
|
}
|
||||||
|
play(base64: string) {
|
||||||
|
if (!base64 || this.closed) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const samples = decodePcm(base64);
|
||||||
|
this.playback?.port.postMessage({ kind: "audio", samples }, [samples.buffer]);
|
||||||
|
}
|
||||||
|
clear() {
|
||||||
|
this.playback?.port.postMessage({ kind: "clear" });
|
||||||
|
}
|
||||||
|
setGate(gate: NoiseGate) {
|
||||||
|
this.gate = gate;
|
||||||
|
this.capture?.port.postMessage({ kind: "gate", ...gate });
|
||||||
|
}
|
||||||
|
setMuted(muted: boolean) {
|
||||||
|
this.muted = muted;
|
||||||
|
this.stream?.getAudioTracks().forEach((track) => {
|
||||||
|
track.enabled = !muted;
|
||||||
|
});
|
||||||
|
this.capture?.port.postMessage({ kind: "enable", value: !muted });
|
||||||
|
}
|
||||||
|
async close() {
|
||||||
|
this.closed = true;
|
||||||
|
cancelAnimationFrame(this.frame);
|
||||||
|
this.stream?.getTracks().forEach((track) => {
|
||||||
|
track.stop();
|
||||||
|
});
|
||||||
|
this.capture?.port.close();
|
||||||
|
this.playback?.port.close();
|
||||||
|
this.nodes.forEach((node) => {
|
||||||
|
node.disconnect();
|
||||||
|
});
|
||||||
|
if (this.context.state !== "closed") {
|
||||||
|
await this.context.close();
|
||||||
|
}
|
||||||
|
for (let index = 0; index < 5; index++) {
|
||||||
|
document.documentElement.style.removeProperty(`--bar${index}`);
|
||||||
|
}
|
||||||
|
document.documentElement.style.removeProperty("--ai-audio-level");
|
||||||
|
document.documentElement.style.removeProperty("--audio-level");
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
const TARGET_RATE = 16000;
|
||||||
|
type Control = { kind: "enable"; value: boolean } | { kind: "gate"; enabled: boolean; thresholdDb: number };
|
||||||
|
|
||||||
|
/** Preserve the existing 40 ms PCM16 frames and smoothed gate envelope. */
|
||||||
|
class MicCaptureProcessor extends AudioWorkletProcessor {
|
||||||
|
private readonly ratio = sampleRate / TARGET_RATE;
|
||||||
|
private readonly chunkSize = 640;
|
||||||
|
private scratch = new Float32Array(0);
|
||||||
|
private readonly decimated = new Float32Array(this.chunkSize);
|
||||||
|
private enabled = true;
|
||||||
|
private gateEnabled = false;
|
||||||
|
private threshold = 0;
|
||||||
|
private gain = 1;
|
||||||
|
private hold = 0;
|
||||||
|
private readonly attack = Math.exp(-1 / (0.005 * TARGET_RATE));
|
||||||
|
private readonly release = Math.exp(-1 / (0.08 * TARGET_RATE));
|
||||||
|
|
||||||
|
constructor() {
|
||||||
|
super();
|
||||||
|
this.port.onmessage = (event: MessageEvent<Control>) => {
|
||||||
|
if (event.data.kind === "enable") {
|
||||||
|
this.enabled = event.data.value;
|
||||||
|
} else {
|
||||||
|
this.gateEnabled = event.data.enabled;
|
||||||
|
this.threshold = event.data.enabled ? 10 ** (event.data.thresholdDb / 20) : 0;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
process(inputs: Float32Array[][]) {
|
||||||
|
const incoming = inputs[0]?.[0];
|
||||||
|
if (!incoming?.length) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
const next = new Float32Array(this.scratch.length + incoming.length);
|
||||||
|
next.set(this.scratch);
|
||||||
|
next.set(incoming, this.scratch.length);
|
||||||
|
this.scratch = next;
|
||||||
|
const needed = Math.ceil(this.chunkSize * this.ratio);
|
||||||
|
while (this.scratch.length >= needed) {
|
||||||
|
let sum = 0;
|
||||||
|
for (let index = 0; index < this.chunkSize; index++) {
|
||||||
|
const position = index * this.ratio;
|
||||||
|
const start = Math.floor(position);
|
||||||
|
const a = this.scratch[start] ?? 0;
|
||||||
|
const value =
|
||||||
|
Math.abs(this.ratio - 3) < 1e-6
|
||||||
|
? (a + (this.scratch[start + 1] ?? 0) + (this.scratch[start + 2] ?? 0)) / 3
|
||||||
|
: a + ((this.scratch[start + 1] ?? a) - a) * (position - start);
|
||||||
|
this.decimated[index] = value;
|
||||||
|
sum += value * value;
|
||||||
|
}
|
||||||
|
const rms = Math.sqrt(sum / this.chunkSize);
|
||||||
|
let target = 1;
|
||||||
|
if (this.gateEnabled) {
|
||||||
|
if (rms >= this.threshold) {
|
||||||
|
this.hold = TARGET_RATE * 0.25;
|
||||||
|
} else if (this.hold > 0) {
|
||||||
|
this.hold -= this.chunkSize;
|
||||||
|
} else {
|
||||||
|
target = 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const output = new Int16Array(this.chunkSize);
|
||||||
|
for (let index = 0; index < this.chunkSize; index++) {
|
||||||
|
const coefficient = target > this.gain ? this.attack : this.release;
|
||||||
|
this.gain = target + (this.gain - target) * coefficient;
|
||||||
|
const value = Math.max(-1, Math.min(1, (this.decimated[index] ?? 0) * this.gain));
|
||||||
|
output[index] = value < 0 ? value * 0x8000 : value * 0x7fff;
|
||||||
|
}
|
||||||
|
this.scratch = this.scratch.slice(Math.floor(this.chunkSize * this.ratio));
|
||||||
|
this.port.postMessage({ kind: "level", rms });
|
||||||
|
if (this.enabled) {
|
||||||
|
this.port.postMessage(output.buffer, [output.buffer]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
registerProcessor("mic-capture", MicCaptureProcessor);
|
||||||
+13
@@ -0,0 +1,13 @@
|
|||||||
|
export {};
|
||||||
|
|
||||||
|
declare global {
|
||||||
|
const sampleRate: number;
|
||||||
|
abstract class AudioWorkletProcessor {
|
||||||
|
readonly port: MessagePort;
|
||||||
|
abstract process(inputs: Float32Array[][], outputs: Float32Array[][]): boolean;
|
||||||
|
}
|
||||||
|
function registerProcessor(
|
||||||
|
name: string,
|
||||||
|
processor: new (options?: AudioWorkletNodeOptions) => AudioWorkletProcessor,
|
||||||
|
): void;
|
||||||
|
}
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
import { memo, useEffect, useState } from "react";
|
||||||
|
|
||||||
|
import { renderMarkdown, type Message } from "../lib/messages";
|
||||||
|
|
||||||
|
function Bubble({ message, userName }: { message: Message; userName: string }) {
|
||||||
|
const [expiredText, setExpiredText] = useState<string | null>(null);
|
||||||
|
useEffect(() => {
|
||||||
|
if (message.partial) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const timer = setTimeout(() => {
|
||||||
|
setExpiredText(message.text);
|
||||||
|
}, 5000);
|
||||||
|
return () => {
|
||||||
|
clearTimeout(timer);
|
||||||
|
};
|
||||||
|
}, [message.text, message.partial]);
|
||||||
|
if (expiredText === message.text) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return (
|
||||||
|
<div className={`bubble ${message.role} in`}>
|
||||||
|
<div className="bubble-role">
|
||||||
|
{message.role === "user" ? userName : message.role === "tool" ? "Herramienta" : "Vocero"}
|
||||||
|
</div>
|
||||||
|
{message.role === "assistant" ? (
|
||||||
|
<div
|
||||||
|
className="bubble-body markdown-content"
|
||||||
|
dangerouslySetInnerHTML={{ __html: renderMarkdown(message.text) }}
|
||||||
|
/>
|
||||||
|
) : (
|
||||||
|
<div className="bubble-body">{message.role === "tool" ? message.toolName : message.text}</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
export const Bubbles = memo(function Bubbles({ messages, userName }: { messages: Message[]; userName: string }) {
|
||||||
|
return (
|
||||||
|
<div className="bubble-stack" aria-hidden="true">
|
||||||
|
{messages.slice(-3).map((message) => (
|
||||||
|
<Bubble key={message.id} message={message} userName={userName} />
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
});
|
||||||
@@ -0,0 +1,140 @@
|
|||||||
|
import { Download, MessageCircle, Wrench } from "lucide-react";
|
||||||
|
import { memo, useEffect, useRef, useState } from "react";
|
||||||
|
|
||||||
|
import { initials } from "../lib/context";
|
||||||
|
import type { ConnectionStatus } from "../lib/contracts";
|
||||||
|
import { displayTime, exportConversation, renderMarkdown, type Message } from "../lib/messages";
|
||||||
|
import { LIVE_STATES } from "../realtime/controller";
|
||||||
|
|
||||||
|
const MessageRow = memo(function MessageRow({ message, userName }: { message: Message; userName: string }) {
|
||||||
|
const [expanded, setExpanded] = useState(false);
|
||||||
|
const time = displayTime(message.timestamp);
|
||||||
|
if (message.role === "tool") {
|
||||||
|
return (
|
||||||
|
<div className="hist-msg tool">
|
||||||
|
<div className="hist-role">Llamada a herramienta</div>
|
||||||
|
<button
|
||||||
|
className="hist-tool-header"
|
||||||
|
type="button"
|
||||||
|
aria-expanded={expanded}
|
||||||
|
onClick={() => {
|
||||||
|
setExpanded(!expanded);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Wrench className="hist-tool-icon" />
|
||||||
|
<span className="hist-tool-name">{message.toolName}</span>
|
||||||
|
</button>
|
||||||
|
<div className={`hist-tool-body ${expanded ? "open" : ""}`}>
|
||||||
|
<div className="hist-tool-label">Entrada</div>
|
||||||
|
<pre className="hist-tool-block">{message.toolArguments}</pre>
|
||||||
|
<div className="hist-tool-label">Salida</div>
|
||||||
|
<pre className="hist-tool-block hist-tool-output">{message.text}</pre>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const data = {
|
||||||
|
"data-user-name": userName,
|
||||||
|
"data-user-initials": initials(userName),
|
||||||
|
"data-asst-name": "Vocero",
|
||||||
|
"data-timestamp": time,
|
||||||
|
};
|
||||||
|
return (
|
||||||
|
<div className={`hist-msg ${message.role} ${message.interrupted ? "interrupted" : ""}`} {...data}>
|
||||||
|
<div className="hist-role">{message.role === "user" ? userName : "Vocero"}</div>
|
||||||
|
{message.role === "assistant" ? (
|
||||||
|
<div
|
||||||
|
className={`hist-body markdown-content ${message.partial ? "partial" : ""}`}
|
||||||
|
{...data}
|
||||||
|
dangerouslySetInnerHTML={{ __html: renderMarkdown(message.text) }}
|
||||||
|
/>
|
||||||
|
) : (
|
||||||
|
<div className={`hist-body ${message.partial ? "partial" : ""}`} {...data}>
|
||||||
|
{message.text}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{message.interrupted && <div className="hist-note">Interrumpido</div>}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
});
|
||||||
|
export const Conversation = memo(function Conversation({
|
||||||
|
messages,
|
||||||
|
userName,
|
||||||
|
status,
|
||||||
|
white,
|
||||||
|
}: {
|
||||||
|
messages: Message[];
|
||||||
|
userName: string;
|
||||||
|
status: ConnectionStatus;
|
||||||
|
white: boolean;
|
||||||
|
}) {
|
||||||
|
const history = useRef<HTMLDivElement>(null);
|
||||||
|
const atBottom = useRef(true);
|
||||||
|
useEffect(() => {
|
||||||
|
const element = history.current;
|
||||||
|
if (element && atBottom.current) {
|
||||||
|
element.scrollTop = element.scrollHeight;
|
||||||
|
}
|
||||||
|
}, [messages]);
|
||||||
|
const live = LIVE_STATES.has(status);
|
||||||
|
const waiting = ["connecting", "queued", "your-turn"].includes(status);
|
||||||
|
return (
|
||||||
|
<section id="chat-panel" className="chat-panel chat-panel-inline" aria-label="Transcripción de la conversación">
|
||||||
|
<div className="chat-panel-inner">
|
||||||
|
<header className="chat-panel-header">
|
||||||
|
<h2>Conversación</h2>
|
||||||
|
<div className="chat-header-actions" role="group" aria-label="Acciones de la conversación">
|
||||||
|
{white && (
|
||||||
|
<span
|
||||||
|
className="chat-status-pill"
|
||||||
|
data-status={live ? "online" : status === "error" ? "error" : waiting ? "waiting" : "idle"}
|
||||||
|
role="status"
|
||||||
|
>
|
||||||
|
<span className="chat-status-dot" aria-hidden="true" />
|
||||||
|
<span>
|
||||||
|
{live ? "En línea" : status === "error" ? "Sin conexión" : waiting ? "Conectando" : "Sin sesión"}
|
||||||
|
</span>
|
||||||
|
</span>
|
||||||
|
)}
|
||||||
|
<button
|
||||||
|
className="chat-export-btn icon-btn"
|
||||||
|
type="button"
|
||||||
|
aria-label="Descargar conversación"
|
||||||
|
title="Descargar conversación (JSON + TXT)"
|
||||||
|
disabled={!messages.length}
|
||||||
|
onClick={() => {
|
||||||
|
exportConversation(messages, userName);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Download size={18} />
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
<div
|
||||||
|
ref={history}
|
||||||
|
id="chat-history"
|
||||||
|
className="chat-history"
|
||||||
|
role="log"
|
||||||
|
aria-live="polite"
|
||||||
|
aria-atomic="false"
|
||||||
|
onScroll={() => {
|
||||||
|
const element = history.current;
|
||||||
|
if (element) {
|
||||||
|
atBottom.current = element.scrollHeight - element.scrollTop - element.clientHeight < 60;
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{messages.length ? (
|
||||||
|
messages.map((message) => <MessageRow key={message.id} message={message} userName={userName} />)
|
||||||
|
) : (
|
||||||
|
<div className="chat-empty">
|
||||||
|
<MessageCircle aria-hidden="true" />
|
||||||
|
<span className="chat-empty-title">Aún no hay mensajes</span>
|
||||||
|
<span className="chat-empty-hint">Toca el círculo y empieza a hablar</span>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
);
|
||||||
|
});
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
import { X } from "lucide-react";
|
||||||
|
import { useEffect, useId, useRef, type ReactNode } from "react";
|
||||||
|
|
||||||
|
interface Props {
|
||||||
|
title: string;
|
||||||
|
children: ReactNode;
|
||||||
|
onClose: () => void;
|
||||||
|
className?: string;
|
||||||
|
}
|
||||||
|
export function Dialog({ title, children, onClose, className = "" }: Props) {
|
||||||
|
const ref = useRef<HTMLDialogElement>(null);
|
||||||
|
const titleId = useId();
|
||||||
|
useEffect(() => {
|
||||||
|
const dialog = ref.current;
|
||||||
|
dialog?.showModal();
|
||||||
|
return () => {
|
||||||
|
dialog?.close();
|
||||||
|
};
|
||||||
|
}, []);
|
||||||
|
return (
|
||||||
|
<dialog
|
||||||
|
ref={ref}
|
||||||
|
className={`modal ${className}`}
|
||||||
|
aria-labelledby={titleId}
|
||||||
|
onCancel={(event) => {
|
||||||
|
event.preventDefault();
|
||||||
|
onClose();
|
||||||
|
}}
|
||||||
|
onClick={(event) => {
|
||||||
|
if (event.target === event.currentTarget) {
|
||||||
|
onClose();
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<div className="modal-content">
|
||||||
|
<header className="modal-header">
|
||||||
|
<h2 id={titleId}>{title}</h2>
|
||||||
|
<button type="button" className="icon-btn" aria-label="Cerrar" onClick={onClose}>
|
||||||
|
<X size={18} />
|
||||||
|
</button>
|
||||||
|
</header>
|
||||||
|
{children}
|
||||||
|
</div>
|
||||||
|
</dialog>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,337 @@
|
|||||||
|
import { useState } from "react";
|
||||||
|
|
||||||
|
import { initials, profileName } from "../lib/context";
|
||||||
|
import { profileRows } from "../lib/profile";
|
||||||
|
import { GATE_MAX, GATE_OFF, VOICES } from "../lib/settings";
|
||||||
|
import { LIVE_STATES, type AppState, type VoiceController } from "../realtime/controller";
|
||||||
|
import { Dialog } from "./Dialog";
|
||||||
|
|
||||||
|
interface Props {
|
||||||
|
state: AppState;
|
||||||
|
controller: VoiceController;
|
||||||
|
onClose: () => void;
|
||||||
|
}
|
||||||
|
export function SettingsDialog({ state, controller, onClose }: Props) {
|
||||||
|
const { config, settings } = state;
|
||||||
|
const meter = Math.max(
|
||||||
|
0,
|
||||||
|
Math.min(100, ((20 * Math.log10(state.level || 0.00001) - GATE_OFF) * 100) / (GATE_MAX - GATE_OFF)),
|
||||||
|
);
|
||||||
|
return (
|
||||||
|
<Dialog title="Configuración" onClose={onClose}>
|
||||||
|
<form
|
||||||
|
onSubmit={(event) => {
|
||||||
|
event.preventDefault();
|
||||||
|
onClose();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<div className="tab-panels">
|
||||||
|
<div className="tab-panel active">
|
||||||
|
{config.allowDirect && !config.lb && (
|
||||||
|
<label className="field">
|
||||||
|
<span>URL del servidor de voz</span>
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
autoComplete="off"
|
||||||
|
spellCheck={false}
|
||||||
|
placeholder="http://localhost:8080"
|
||||||
|
value={config.s2sUrl || settings.directUrl}
|
||||||
|
readOnly={!!config.s2sUrl}
|
||||||
|
onChange={(event) => {
|
||||||
|
controller.changeSettings({ directUrl: event.target.value });
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<small>
|
||||||
|
{config.s2sUrl
|
||||||
|
? "El servidor fija esta dirección para todas las conversaciones."
|
||||||
|
: "Introduce la dirección de tu servidor de voz. La aplicación añade /v1/realtime."}
|
||||||
|
</small>
|
||||||
|
</label>
|
||||||
|
)}
|
||||||
|
<div className="field-row">
|
||||||
|
<label className="field">
|
||||||
|
<span>Voz</span>
|
||||||
|
<select
|
||||||
|
value={settings.voice}
|
||||||
|
onChange={(event) => {
|
||||||
|
controller.changeSettings({ voice: event.target.value });
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{VOICES.map((voice) => (
|
||||||
|
<option key={voice}>{voice}</option>
|
||||||
|
))}
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
<div className="field">
|
||||||
|
<label className="field-head" htmlFor="noise-gate">
|
||||||
|
Puerta de ruido
|
||||||
|
<span className="field-value">
|
||||||
|
{settings.noiseGate <= GATE_OFF ? "Desactivada" : `${String(settings.noiseGate)} dB`}
|
||||||
|
</span>
|
||||||
|
</label>
|
||||||
|
<div className="gate">
|
||||||
|
<div className="gate-track">
|
||||||
|
<div className="gate-meter-fill" style={{ width: `${String(meter)}%` }} aria-hidden="true" />
|
||||||
|
<input
|
||||||
|
id="noise-gate"
|
||||||
|
aria-label="Umbral de la puerta de ruido"
|
||||||
|
type="range"
|
||||||
|
min={GATE_OFF}
|
||||||
|
max={GATE_MAX}
|
||||||
|
step="1"
|
||||||
|
value={settings.noiseGate}
|
||||||
|
onChange={(event) => {
|
||||||
|
controller.changeSettings({ noiseGate: Number(event.target.value) });
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<div className="gate-ends">
|
||||||
|
<span>Desactivada</span>
|
||||||
|
<span>−3 dB</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<small>
|
||||||
|
Silencia el micrófono por debajo del umbral para evitar enviar ruido. Desliza hasta la izquierda para
|
||||||
|
desactivarla.
|
||||||
|
</small>
|
||||||
|
</div>
|
||||||
|
<label className="field">
|
||||||
|
<span>Instrucciones</span>
|
||||||
|
<textarea
|
||||||
|
rows={5}
|
||||||
|
value={config.primaryPrompt || settings.instructions}
|
||||||
|
readOnly={!!config.primaryPrompt}
|
||||||
|
onChange={(event) => {
|
||||||
|
controller.changeSettings({ instructions: event.target.value });
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
{config.primaryPrompt && <small>Las instrucciones están definidas por el servidor.</small>}
|
||||||
|
</label>
|
||||||
|
<div className="field">
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
className="btn primary wide"
|
||||||
|
disabled={!LIVE_STATES.has(state.status)}
|
||||||
|
onClick={() => {
|
||||||
|
onClose();
|
||||||
|
void controller.restart();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
Reiniciar conversación con esta configuración
|
||||||
|
</button>
|
||||||
|
<small>La voz, las instrucciones y la puerta de ruido se actualizan en la sesión actual.</small>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<footer className="modal-footer">
|
||||||
|
<button className="btn primary" type="submit">
|
||||||
|
Guardar
|
||||||
|
</button>
|
||||||
|
</footer>
|
||||||
|
</form>
|
||||||
|
</Dialog>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function RagDialog({ state, controller, onClose }: Props) {
|
||||||
|
const [text, setText] = useState(state.rag);
|
||||||
|
const [saved, setSaved] = useState(false);
|
||||||
|
return (
|
||||||
|
<Dialog title="Contexto RAG personalizado" onClose={onClose}>
|
||||||
|
<p className="tools-intro">
|
||||||
|
Añade documentos, datos o instrucciones que Vocero debe tener en cuenta durante esta conversación. El contexto
|
||||||
|
se mantiene mientras esta página esté abierta.
|
||||||
|
</p>
|
||||||
|
<form
|
||||||
|
onSubmit={(event) => {
|
||||||
|
event.preventDefault();
|
||||||
|
controller.setRag(text);
|
||||||
|
setSaved(true);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<label className="field">
|
||||||
|
<span>Contexto RAG (texto libre)</span>
|
||||||
|
<textarea
|
||||||
|
rows={12}
|
||||||
|
spellCheck={false}
|
||||||
|
value={text}
|
||||||
|
onChange={(event) => {
|
||||||
|
setText(event.target.value);
|
||||||
|
setSaved(false);
|
||||||
|
}}
|
||||||
|
placeholder="Pega o escribe el contexto que Vocero debe usar al responder…"
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
<p className="tool-hint" role="status">
|
||||||
|
{saved
|
||||||
|
? "Contexto actualizado para esta conversación."
|
||||||
|
: state.rag
|
||||||
|
? `${String(state.rag.length)} caracteres de contexto cargados.`
|
||||||
|
: "No hay contexto RAG cargado."}
|
||||||
|
</p>
|
||||||
|
<footer className="modal-footer">
|
||||||
|
<button
|
||||||
|
className="btn secondary"
|
||||||
|
type="button"
|
||||||
|
onClick={() => {
|
||||||
|
setText("");
|
||||||
|
controller.setRag("");
|
||||||
|
setSaved(true);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
Vaciar
|
||||||
|
</button>
|
||||||
|
<div className="flex-1" />
|
||||||
|
<button className="btn primary" type="submit">
|
||||||
|
Enviar a la sesión
|
||||||
|
</button>
|
||||||
|
</footer>
|
||||||
|
</form>
|
||||||
|
</Dialog>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function ToolsDialog({ state, controller, onClose }: Props) {
|
||||||
|
return (
|
||||||
|
<Dialog title="Herramientas" onClose={onClose}>
|
||||||
|
<p className="tools-intro">
|
||||||
|
Permite que el asistente actúe durante la conversación. Los cambios se aplican de inmediato.
|
||||||
|
</p>
|
||||||
|
<div className="tool-list">
|
||||||
|
<div className="tool-row">
|
||||||
|
<div className="tool-info">
|
||||||
|
<span className="tool-name">Búsqueda web</span>
|
||||||
|
<span className="tool-desc">Busca información en Google mediante Serper.</span>
|
||||||
|
</div>
|
||||||
|
<label className="switch">
|
||||||
|
<input
|
||||||
|
type="checkbox"
|
||||||
|
aria-label="Búsqueda web"
|
||||||
|
checked={state.settings.webSearch}
|
||||||
|
onChange={(event) => {
|
||||||
|
controller.changeSettings({ webSearch: event.target.checked });
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<span className="switch-track" aria-hidden="true" />
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
<label className="field tools-key">
|
||||||
|
<span>Clave de la API de búsqueda</span>
|
||||||
|
<input
|
||||||
|
type="password"
|
||||||
|
autoComplete="off"
|
||||||
|
value={state.settings.searchKey}
|
||||||
|
disabled={state.config.search}
|
||||||
|
onChange={(event) => {
|
||||||
|
controller.changeSettings({ searchKey: event.target.value });
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<small>
|
||||||
|
{state.config.search
|
||||||
|
? "La búsqueda está configurada en el servidor."
|
||||||
|
: "Introduce tu clave de Serper para activar la búsqueda. Se guarda en este navegador."}
|
||||||
|
</small>
|
||||||
|
</label>
|
||||||
|
<div className="tool-row tool-row-sep">
|
||||||
|
<div className="tool-info">
|
||||||
|
<span className="tool-name">Herramientas MCP cargadas</span>
|
||||||
|
<span className="tool-desc">Herramientas publicadas por el perfil remoto del agente.</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<p className="tool-hint" role="status">
|
||||||
|
{state.toolsLoading
|
||||||
|
? "Cargando herramientas…"
|
||||||
|
: state.toolsError ||
|
||||||
|
(state.tools.length
|
||||||
|
? `${String(state.tools.length)} herramientas disponibles.`
|
||||||
|
: "No hay herramientas remotas disponibles.")}
|
||||||
|
</p>
|
||||||
|
<div className="agent-tools-list">
|
||||||
|
{state.tools.map((tool) => (
|
||||||
|
<article key={tool.name} className="agent-tool-item">
|
||||||
|
<strong className="block text-sm">{tool.name}</strong>
|
||||||
|
<p className="mt-1 text-sm opacity-75">{tool.description}</p>
|
||||||
|
</article>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</Dialog>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function ProfileDialog({ state, onClose }: Omit<Props, "controller">) {
|
||||||
|
const name = profileName(state.profile) || "Usuario";
|
||||||
|
const rows = profileRows(state.profile);
|
||||||
|
return (
|
||||||
|
<Dialog title="Datos del usuario" className="user-profile-modal" onClose={onClose}>
|
||||||
|
<div className="user-profile-hero">
|
||||||
|
<div className="user-profile-avatar" aria-hidden="true">
|
||||||
|
{initials(name)}
|
||||||
|
</div>
|
||||||
|
<h3 className="user-profile-title">{name}</h3>
|
||||||
|
</div>
|
||||||
|
<dl className="user-profile-metadata">
|
||||||
|
{rows.map(({ label, value, href }) => (
|
||||||
|
<div key={label} className="user-profile-row">
|
||||||
|
<dt className="user-profile-label">{label}</dt>
|
||||||
|
<dd className="user-profile-value">{href ? <a href={href}>{value}</a> : value}</dd>
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
</dl>
|
||||||
|
</Dialog>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function AboutDialog({ onClose }: { onClose: () => void }) {
|
||||||
|
return (
|
||||||
|
<Dialog title="Acerca de" className="about-modal" onClose={onClose}>
|
||||||
|
<p className="tools-intro">
|
||||||
|
Vocero convierte tu voz en respuestas habladas, en tiempo real. Puedes interrumpirlo para continuar la
|
||||||
|
conversación.
|
||||||
|
</p>
|
||||||
|
<div className="about-pipeline">
|
||||||
|
<ol className="pipeline">
|
||||||
|
<li className="pipe-endpoint">Tú hablas</li>
|
||||||
|
<li className="pipe-stage">
|
||||||
|
<span className="pipe-tag">STT</span>
|
||||||
|
<span className="pipe-job">transcribe tu voz</span>
|
||||||
|
<a
|
||||||
|
className="pipe-model"
|
||||||
|
href="https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3"
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
>
|
||||||
|
nvidia/parakeet-tdt-0.6b-v3
|
||||||
|
</a>
|
||||||
|
</li>
|
||||||
|
<li className="pipe-stage">
|
||||||
|
<span className="pipe-tag">LLM</span>
|
||||||
|
<span className="pipe-job">genera una respuesta</span>
|
||||||
|
<a
|
||||||
|
className="pipe-model"
|
||||||
|
href="https://huggingface.co/google/gemma-4-31B-it"
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
>
|
||||||
|
google/gemma-4-31B-it
|
||||||
|
</a>
|
||||||
|
</li>
|
||||||
|
<li className="pipe-stage">
|
||||||
|
<span className="pipe-tag">TTS</span>
|
||||||
|
<span className="pipe-job">la convierte en voz</span>
|
||||||
|
<a
|
||||||
|
className="pipe-model"
|
||||||
|
href="https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice"
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
>
|
||||||
|
Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
|
||||||
|
</a>
|
||||||
|
</li>
|
||||||
|
<li className="pipe-endpoint">Vocero responde</li>
|
||||||
|
</ol>
|
||||||
|
</div>
|
||||||
|
</Dialog>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,302 @@
|
|||||||
|
import { CircleAlert, Mic, MicOff, Square, Volume2 } from "lucide-react";
|
||||||
|
import { useEffect, useState, type CSSProperties, type PointerEvent } from "react";
|
||||||
|
|
||||||
|
import type { ConnectionStatus } from "../lib/contracts";
|
||||||
|
import { GATE_MAX, GATE_OFF } from "../lib/settings";
|
||||||
|
import { ACTIVE_STATES, LIVE_STATES, type AppState, type VoiceController } from "../realtime/controller";
|
||||||
|
|
||||||
|
const CAPTIONS: Record<ConnectionStatus, string> = {
|
||||||
|
idle: "Toca para empezar",
|
||||||
|
closed: "Toca para empezar",
|
||||||
|
connecting: "Conectando…",
|
||||||
|
queued: "Esperando tu turno…",
|
||||||
|
"your-turn": "¡Es tu turno!",
|
||||||
|
listening: "Te escucho…",
|
||||||
|
"user-speaking": "Te escucho…",
|
||||||
|
processing: "Pensando…",
|
||||||
|
"ai-speaking": "Vocero está hablando…",
|
||||||
|
error: "Toca para reintentar",
|
||||||
|
};
|
||||||
|
function GateArc({
|
||||||
|
threshold,
|
||||||
|
level,
|
||||||
|
onChange,
|
||||||
|
}: {
|
||||||
|
threshold: number;
|
||||||
|
level: number;
|
||||||
|
onChange: (value: number) => void;
|
||||||
|
}) {
|
||||||
|
const angle = ((135 + ((threshold - GATE_OFF) / (GATE_MAX - GATE_OFF)) * 270) * Math.PI) / 180;
|
||||||
|
const fraction = Math.max(0, Math.min(1, (20 * Math.log10(level || 0.00001) - GATE_OFF) / (GATE_MAX - GATE_OFF)));
|
||||||
|
const path = "M 20.3 79.7 A 42 42 0 1 1 79.7 79.7";
|
||||||
|
function move(event: PointerEvent<SVGSVGElement>) {
|
||||||
|
const bounds = event.currentTarget.getBoundingClientRect();
|
||||||
|
const degrees =
|
||||||
|
(Math.atan2(event.clientY - bounds.top - bounds.height / 2, event.clientX - bounds.left - bounds.width / 2) *
|
||||||
|
180) /
|
||||||
|
Math.PI;
|
||||||
|
const offset = (degrees - 135 + 360) % 360;
|
||||||
|
const progress = offset > 270 ? (offset > 315 ? 0 : 1) : offset / 270;
|
||||||
|
onChange(Math.round(GATE_OFF + progress * (GATE_MAX - GATE_OFF)));
|
||||||
|
}
|
||||||
|
return (
|
||||||
|
<svg
|
||||||
|
className="mic-gate-arc"
|
||||||
|
viewBox="0 0 100 100"
|
||||||
|
role="slider"
|
||||||
|
tabIndex={0}
|
||||||
|
aria-label="Umbral de la puerta de ruido"
|
||||||
|
aria-valuemin={GATE_OFF}
|
||||||
|
aria-valuemax={GATE_MAX}
|
||||||
|
aria-valuenow={threshold}
|
||||||
|
aria-valuetext={threshold === GATE_OFF ? "Desactivada" : `${String(threshold)} dB`}
|
||||||
|
onPointerDown={(event) => {
|
||||||
|
event.currentTarget.setPointerCapture(event.pointerId);
|
||||||
|
move(event);
|
||||||
|
}}
|
||||||
|
onPointerMove={(event) => {
|
||||||
|
if (event.currentTarget.hasPointerCapture(event.pointerId)) {
|
||||||
|
move(event);
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
onPointerUp={(event) => {
|
||||||
|
event.currentTarget.releasePointerCapture(event.pointerId);
|
||||||
|
}}
|
||||||
|
onKeyDown={(event) => {
|
||||||
|
const step =
|
||||||
|
event.key === "ArrowRight" || event.key === "ArrowUp"
|
||||||
|
? 1
|
||||||
|
: event.key === "ArrowLeft" || event.key === "ArrowDown"
|
||||||
|
? -1
|
||||||
|
: 0;
|
||||||
|
if (step || event.key === "Home" || event.key === "End") {
|
||||||
|
event.preventDefault();
|
||||||
|
onChange(
|
||||||
|
event.key === "Home"
|
||||||
|
? GATE_OFF
|
||||||
|
: event.key === "End"
|
||||||
|
? GATE_MAX
|
||||||
|
: Math.max(GATE_OFF, Math.min(GATE_MAX, threshold + step)),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<path className="mga-track" d={path} fill="none" />
|
||||||
|
<path className="mga-fill" d={path} fill="none" pathLength="1" strokeDasharray={`${String(fraction)} 1`} />
|
||||||
|
<path className="mga-hit" d={path} fill="none" />
|
||||||
|
<circle className="mga-handle" r="3" cx={50 + 42 * Math.cos(angle)} cy={50 + 42 * Math.sin(angle)} />
|
||||||
|
</svg>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
function Waveform({ side }: { side: "left" | "right" }) {
|
||||||
|
const heights = [0.4, 0.65, 0.9, 1, 0.7, 0.55, 0.3];
|
||||||
|
return (
|
||||||
|
<div className={`white-waveform white-waveform-${side}`} aria-hidden="true">
|
||||||
|
{(side === "left" ? heights : [...heights].reverse()).map((height, index) => (
|
||||||
|
<span key={index} className={`w-bar wb-${String(index + 1)}`} style={{ "--bar-h": height } as CSSProperties} />
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
function JoinCountdown({ deadline }: { deadline: number }) {
|
||||||
|
const [now, setNow] = useState(Date.now);
|
||||||
|
useEffect(() => {
|
||||||
|
const timer = setInterval(() => {
|
||||||
|
setNow(Date.now());
|
||||||
|
}, 1000);
|
||||||
|
return () => {
|
||||||
|
clearInterval(timer);
|
||||||
|
};
|
||||||
|
}, []);
|
||||||
|
return (
|
||||||
|
<p className="circle-subcaption">
|
||||||
|
Tu lugar se reserva durante {Math.max(0, Math.ceil((deadline - now) / 1000))} s.
|
||||||
|
</p>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
export function VoiceStage({ state, controller }: { state: AppState; controller: VoiceController }) {
|
||||||
|
const white = state.config.theme === "white";
|
||||||
|
const live = LIVE_STATES.has(state.status);
|
||||||
|
const active = ACTIVE_STATES.has(state.status);
|
||||||
|
const status = state.status === "closed" ? "idle" : state.status;
|
||||||
|
const caption =
|
||||||
|
state.muted && (status === "listening" || status === "user-speaking")
|
||||||
|
? "Micrófono silenciado"
|
||||||
|
: CAPTIONS[state.status];
|
||||||
|
const label = active
|
||||||
|
? caption
|
||||||
|
: status === "error"
|
||||||
|
? "Reintentar conversación de voz"
|
||||||
|
: "Iniciar conversación de voz";
|
||||||
|
const muteLabel = state.muted ? "Activar micrófono" : "Silenciar micrófono";
|
||||||
|
const start = () => {
|
||||||
|
if (status === "your-turn") {
|
||||||
|
controller.join();
|
||||||
|
} else {
|
||||||
|
void controller.start();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
const disabled = active && status !== "your-turn";
|
||||||
|
return (
|
||||||
|
<main className="stage">
|
||||||
|
{white ? (
|
||||||
|
<div className={`white-mic-stage ${live ? "live" : ""}`} role="group" aria-label="Control de voz">
|
||||||
|
<Waveform side="left" />
|
||||||
|
<button
|
||||||
|
id="white-mic-circle"
|
||||||
|
className={`white-mic-circle state-${status}`}
|
||||||
|
type="button"
|
||||||
|
aria-label={label}
|
||||||
|
disabled={disabled}
|
||||||
|
onClick={start}
|
||||||
|
>
|
||||||
|
<span className="wmc-halo" aria-hidden="true" />
|
||||||
|
<span className="wmc-ring" aria-hidden="true" />
|
||||||
|
<span className="wmc-core" aria-hidden="true" />
|
||||||
|
<Mic className="wmc-icon wmc-mic" aria-hidden="true" />
|
||||||
|
<span className="wmc-icon wmc-spinner" aria-hidden="true" />
|
||||||
|
</button>
|
||||||
|
<Waveform side="right" />
|
||||||
|
<p className="white-mic-caption" role="status">
|
||||||
|
{caption}
|
||||||
|
</p>
|
||||||
|
{state.notice && <p className="white-mic-subcaption">{state.notice}</p>}
|
||||||
|
{active && (
|
||||||
|
<div className="white-session-controls" role="group" aria-label="Controles de la conversación">
|
||||||
|
{live && (
|
||||||
|
<button
|
||||||
|
className={`white-mute-btn ${state.muted ? "muted" : ""}`}
|
||||||
|
type="button"
|
||||||
|
onClick={() => {
|
||||||
|
controller.toggleMuted();
|
||||||
|
}}
|
||||||
|
aria-pressed={state.muted}
|
||||||
|
aria-label={muteLabel}
|
||||||
|
>
|
||||||
|
{state.muted ? <MicOff /> : <Mic />}
|
||||||
|
<span>{muteLabel}</span>
|
||||||
|
</button>
|
||||||
|
)}
|
||||||
|
<button
|
||||||
|
className="white-stop-btn"
|
||||||
|
type="button"
|
||||||
|
onClick={() => {
|
||||||
|
void controller.stop();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Square fill="currentColor" />
|
||||||
|
<span>Detener</span>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
) : (
|
||||||
|
<>
|
||||||
|
<div className={`orb-wrap ${active ? "live" : ""}`}>
|
||||||
|
<div
|
||||||
|
className={`mic-gate ${20 * Math.log10(state.level || 0.00001) >= state.settings.noiseGate ? "gate-open" : ""}`}
|
||||||
|
>
|
||||||
|
{live && (
|
||||||
|
<GateArc
|
||||||
|
threshold={state.settings.noiseGate}
|
||||||
|
level={state.level}
|
||||||
|
onChange={(noiseGate) => {
|
||||||
|
controller.changeSettings({ noiseGate });
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
<button
|
||||||
|
className={`side-btn ${state.muted ? "muted" : ""}`}
|
||||||
|
type="button"
|
||||||
|
disabled={!live}
|
||||||
|
tabIndex={live ? 0 : -1}
|
||||||
|
aria-label={muteLabel}
|
||||||
|
aria-pressed={state.muted}
|
||||||
|
onClick={() => {
|
||||||
|
controller.toggleMuted();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{state.muted ? <MicOff /> : <Mic />}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
<button
|
||||||
|
id="main-circle"
|
||||||
|
className={`circle state-${status}`}
|
||||||
|
type="button"
|
||||||
|
aria-label={label}
|
||||||
|
disabled={disabled}
|
||||||
|
onClick={start}
|
||||||
|
>
|
||||||
|
<span className="circle-glow" aria-hidden="true" />
|
||||||
|
<span className="circle-ring" aria-hidden="true" />
|
||||||
|
<span className="circle-ring-outer" aria-hidden="true" />
|
||||||
|
<span className="circle-core">
|
||||||
|
<span className="circle-indicator" aria-hidden="true">
|
||||||
|
<Mic className="ind ind-mic" />
|
||||||
|
<CircleAlert className="ind ind-error" />
|
||||||
|
<span className="ind ind-spinner" />
|
||||||
|
<span className="ind ind-thinking">
|
||||||
|
<span className="dot" />
|
||||||
|
<span className="dot" />
|
||||||
|
<span className="dot" />
|
||||||
|
</span>
|
||||||
|
<span className="ind ind-bars">
|
||||||
|
{[0, 1, 2, 3, 4].map((number) => (
|
||||||
|
<span key={number} className="bar" />
|
||||||
|
))}
|
||||||
|
</span>
|
||||||
|
<Volume2 className="ind ind-voice" />
|
||||||
|
</span>
|
||||||
|
</span>
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
className="side-btn"
|
||||||
|
type="button"
|
||||||
|
disabled={!active}
|
||||||
|
tabIndex={active ? 0 : -1}
|
||||||
|
aria-label="Finalizar"
|
||||||
|
onClick={() => {
|
||||||
|
void controller.stop();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Square fill="currentColor" />
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
<p className="circle-caption" role="status">
|
||||||
|
{caption}
|
||||||
|
</p>
|
||||||
|
{state.notice && <p className="circle-subcaption">{state.notice}</p>}
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
{(status === "queued" || status === "your-turn") && (
|
||||||
|
<div className="queue-actions">
|
||||||
|
{status === "queued" ? (
|
||||||
|
<p className="circle-subcaption">Posición en la cola: {state.queuePosition}</p>
|
||||||
|
) : (
|
||||||
|
<>
|
||||||
|
<JoinCountdown deadline={state.joinDeadline} />
|
||||||
|
<button
|
||||||
|
className="join-queue-btn"
|
||||||
|
type="button"
|
||||||
|
onClick={() => {
|
||||||
|
controller.join();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
Entrar ahora
|
||||||
|
</button>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
<button
|
||||||
|
className="leave-queue-btn"
|
||||||
|
type="button"
|
||||||
|
onClick={() => {
|
||||||
|
void controller.stop();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
Salir de la cola
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</main>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
import { useEffect } from "react";
|
||||||
|
|
||||||
|
import type { VoiceController } from "../realtime/controller";
|
||||||
|
|
||||||
|
interface Recognition {
|
||||||
|
lang: string;
|
||||||
|
continuous: boolean;
|
||||||
|
interimResults: boolean;
|
||||||
|
onresult: ((event: { resultIndex: number; results: ArrayLike<ArrayLike<{ transcript: string }>> }) => void) | null;
|
||||||
|
onerror: ((event: { error: string }) => void) | null;
|
||||||
|
onend: (() => void) | null;
|
||||||
|
start: () => void;
|
||||||
|
abort: () => void;
|
||||||
|
}
|
||||||
|
declare global {
|
||||||
|
interface Window {
|
||||||
|
SpeechRecognition?: new () => Recognition;
|
||||||
|
webkitSpeechRecognition?: new () => Recognition;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/** Optional browser wake word; never owns the microphone during a voice session. */
|
||||||
|
export function useWakeWord(enabled: boolean, controller: VoiceController) {
|
||||||
|
useEffect(() => {
|
||||||
|
const Constructor = window.SpeechRecognition ?? window.webkitSpeechRecognition;
|
||||||
|
if (!enabled || !Constructor) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const recognition = new Constructor();
|
||||||
|
let stopped = false;
|
||||||
|
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||||
|
recognition.lang = "es-ES";
|
||||||
|
recognition.continuous = true;
|
||||||
|
recognition.interimResults = true;
|
||||||
|
const start = () => {
|
||||||
|
if (!stopped) {
|
||||||
|
try {
|
||||||
|
recognition.start();
|
||||||
|
} catch {
|
||||||
|
/* A browser may require a click first. */
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
recognition.onresult = (event) => {
|
||||||
|
for (let index = event.resultIndex; index < event.results.length; index++) {
|
||||||
|
const text =
|
||||||
|
event.results[index]?.[0]?.transcript
|
||||||
|
.toLowerCase()
|
||||||
|
.normalize("NFD")
|
||||||
|
.replace(/[\u0300-\u036f]/g, "") ?? "";
|
||||||
|
if (/\b(?:hola|ola)\s+(?:vocero|bocero|bosero|vocerro)\b/.test(text)) {
|
||||||
|
stopped = true;
|
||||||
|
recognition.abort();
|
||||||
|
void controller.start(false, undefined, "hola vocero");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
recognition.onerror = (event) => {
|
||||||
|
if (["not-allowed", "service-not-allowed", "audio-capture"].includes(event.error)) {
|
||||||
|
stopped = true;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
recognition.onend = () => {
|
||||||
|
if (!stopped) {
|
||||||
|
timer = setTimeout(start, 1000);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
start();
|
||||||
|
return () => {
|
||||||
|
stopped = true;
|
||||||
|
clearTimeout(timer);
|
||||||
|
recognition.onresult = null;
|
||||||
|
recognition.onerror = null;
|
||||||
|
recognition.onend = null;
|
||||||
|
recognition.abort();
|
||||||
|
};
|
||||||
|
}, [enabled, controller]);
|
||||||
|
}
|
||||||
@@ -0,0 +1,122 @@
|
|||||||
|
import { z } from "zod";
|
||||||
|
|
||||||
|
import { configSchema, recordSchema, reportsSchema, toolsEnvelopeSchema } from "./contracts";
|
||||||
|
|
||||||
|
export class ApiError extends Error {
|
||||||
|
readonly status: number;
|
||||||
|
readonly body: unknown;
|
||||||
|
|
||||||
|
constructor(status: number, body: unknown) {
|
||||||
|
const data = recordSchema.safeParse(body);
|
||||||
|
super(data.success && typeof data.data.detail === "string" ? data.data.detail : `Error del servidor (${status}).`);
|
||||||
|
this.name = "ApiError";
|
||||||
|
this.status = status;
|
||||||
|
this.body = body;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function request<T>(path: string, schema: z.ZodType<T>, init?: RequestInit): Promise<T> {
|
||||||
|
const response = await fetch(path, init);
|
||||||
|
const body: unknown = await response.json().catch(() => null);
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new ApiError(response.status, body);
|
||||||
|
}
|
||||||
|
return schema.parse(body);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function post<T>(path: string, schema: z.ZodType<T>, body: unknown, signal?: AbortSignal) {
|
||||||
|
return request(path, schema, {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify(body),
|
||||||
|
signal,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function optionalRequest<T>(path: string, schema: z.ZodType<T>, signal: AbortSignal) {
|
||||||
|
try {
|
||||||
|
return await request(path, schema, { signal });
|
||||||
|
} catch (error) {
|
||||||
|
if (signal.aborted) {
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
// Profile and reports are optional in generic deployments.
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function getConfig(signal: AbortSignal) {
|
||||||
|
return request("/api/config", configSchema, { signal });
|
||||||
|
}
|
||||||
|
export async function getProfile(phone: string, signal: AbortSignal) {
|
||||||
|
if (!phone) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
const data = await optionalRequest(
|
||||||
|
`/api/user/phone?${new URLSearchParams({ phoneNumber: phone })}`,
|
||||||
|
recordSchema,
|
||||||
|
signal,
|
||||||
|
);
|
||||||
|
if (!data) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
const result = recordSchema.safeParse(data.result);
|
||||||
|
return result.success ? result.data : data;
|
||||||
|
}
|
||||||
|
export function getReports(phone: string, signal: AbortSignal) {
|
||||||
|
const query = phone ? `?${new URLSearchParams({ phoneNumber: phone })}` : "";
|
||||||
|
return optionalRequest(`/api/user/reports${query}`, reportsSchema, signal);
|
||||||
|
}
|
||||||
|
export async function getTools(signal: AbortSignal, refresh = false) {
|
||||||
|
const result = await request(
|
||||||
|
`/api/apipilot/tools?include_unauthorized=true&refresh=${String(refresh)}`,
|
||||||
|
toolsEnvelopeSchema,
|
||||||
|
{ signal },
|
||||||
|
);
|
||||||
|
if (!result.success || !result.data) {
|
||||||
|
throw new Error("No se pudieron cargar las herramientas.");
|
||||||
|
}
|
||||||
|
return result.data.tools;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const reasoningSchema = z.object({ answer: z.string().default(""), log_only: z.boolean().default(false) });
|
||||||
|
export const searchSchema = z.object({
|
||||||
|
answer: z.string().nullish(),
|
||||||
|
results: z.array(z.object({ title: z.string(), snippet: z.string().default(""), url: z.string() })).default([]),
|
||||||
|
});
|
||||||
|
|
||||||
|
/** Teardown must survive pagehide and must not hold up a new session. */
|
||||||
|
export function cancelQueue(queueId: string) {
|
||||||
|
const path = "/api/queue/end";
|
||||||
|
const payload = JSON.stringify({ queueId });
|
||||||
|
if (
|
||||||
|
typeof navigator.sendBeacon === "function" &&
|
||||||
|
navigator.sendBeacon(path, new Blob([payload], { type: "application/json" }))
|
||||||
|
) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
void fetch(path, {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: payload,
|
||||||
|
keepalive: true,
|
||||||
|
}).catch(() => undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function errorMessage(error: unknown) {
|
||||||
|
if (error instanceof z.ZodError) {
|
||||||
|
return "El servidor devolvió una respuesta inesperada.";
|
||||||
|
}
|
||||||
|
if (error instanceof DOMException) {
|
||||||
|
if (error.name === "NotAllowedError") {
|
||||||
|
return "No se pudo acceder al micrófono. Revisa el permiso del navegador y vuelve a intentarlo.";
|
||||||
|
}
|
||||||
|
if (error.name === "NotFoundError") {
|
||||||
|
return "No se encontró un micrófono. Conecta uno y vuelve a intentarlo.";
|
||||||
|
}
|
||||||
|
if (error.name === "NotReadableError") {
|
||||||
|
return "El micrófono está ocupado o no está disponible.";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return error instanceof Error ? error.message : "No se pudo completar la operación.";
|
||||||
|
}
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
import { recordSchema, type DynamicTool, type FunctionTool, type Profile, type ServerConfig } from "./contracts";
|
||||||
|
import type { Settings } from "./settings";
|
||||||
|
|
||||||
|
export function profileName(profile: Profile | null): string {
|
||||||
|
if (!profile) {
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
for (const key of ["name", "fullName", "full_name", "displayName", "display_name", "username"]) {
|
||||||
|
const value = profile[key];
|
||||||
|
if (typeof value === "string" && value.trim()) {
|
||||||
|
return value.trim();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const full = [profile.firstName, profile.lastName]
|
||||||
|
.filter((part): part is string => typeof part === "string")
|
||||||
|
.join(" ")
|
||||||
|
.trim();
|
||||||
|
if (full) {
|
||||||
|
return full;
|
||||||
|
}
|
||||||
|
for (const key of ["user", "customer", "profile", "contact"]) {
|
||||||
|
const nested = recordSchema.safeParse(profile[key]);
|
||||||
|
if (nested.success) {
|
||||||
|
const name = profileName(nested.data);
|
||||||
|
if (name) {
|
||||||
|
return name;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
export function initials(name: string) {
|
||||||
|
const words = name.trim().split(/\s+/);
|
||||||
|
return (words.length > 1 ? `${words[0]?.[0] ?? ""}${words.at(-1)?.[0] ?? ""}` : name.slice(0, 2)).toUpperCase();
|
||||||
|
}
|
||||||
|
export function profileContext(profile: Profile | null) {
|
||||||
|
if (!profile) {
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
const serialized = JSON.stringify(profile);
|
||||||
|
return `Caller profile resolved from the page phoneNumber: ${serialized.length > 1600 ? `${serialized.slice(0, 1600)}...` : serialized}. Use this as trusted caller context when relevant.`;
|
||||||
|
}
|
||||||
|
export function ragContext(text: string) {
|
||||||
|
return text.trim() ? `<<<CUSTOM_RAG_BEGIN>>>\n${text.trim()}\n<<<CUSTOM_RAG_END>>>` : "";
|
||||||
|
}
|
||||||
|
export const TOOL_MARKER = "<<<APIPILOT_TOOLS_LIST_BEGIN>>>";
|
||||||
|
export function toolContext(tools: DynamicTool[]) {
|
||||||
|
if (!tools.length) {
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
const entries = tools.slice(0, 100).map((tool) => {
|
||||||
|
const required = Array.isArray(tool.parameters.required)
|
||||||
|
? tool.parameters.required.filter((key): key is string => typeof key === "string")
|
||||||
|
: [];
|
||||||
|
return `- name: ${tool.name}\n requires_auth: ${String(tool.requires_auth)}${required.length ? `\n required_parameters: [${required.slice(0, 40).join(", ")}]` : ""}\n description: ${tool.description.slice(0, 500)}`;
|
||||||
|
});
|
||||||
|
return `${TOOL_MARKER}\n### Available dynamic tools (ApiPilot AquaPortal)\nEach tool name is a unique identifier: DO NOT change, camelCase or translate it.\nMANDATORY EXECUTION RULES:\n1. If the user request matches a tool description, output ONLY one line as the FIRST line of your response:\n<<<APIPILOT_CALL:NAME=tool_name,ARGUMENTS={"parameter1":"value1"}>>>\n2. ARGUMENTS must be valid JSON. Include ONLY fields listed in required_parameters, if any.\n3. Do NOT add text after the marker. The engine executes the tool and supplies its result in a second call.\n4. If no description matches, respond normally in Spanish without a marker.\n5. If multiple descriptions match, choose the most specific.\n${entries.join("\n")}\n<<<APIPILOT_TOOLS_LIST_END>>>`;
|
||||||
|
}
|
||||||
|
export function buildInstructions(
|
||||||
|
config: ServerConfig,
|
||||||
|
settings: Settings,
|
||||||
|
profile: Profile | null,
|
||||||
|
rag: string,
|
||||||
|
tools: DynamicTool[],
|
||||||
|
) {
|
||||||
|
const base = config.primaryPrompt || settings.instructions;
|
||||||
|
return [
|
||||||
|
base,
|
||||||
|
config.injectProfile ? profileContext(profile) : "",
|
||||||
|
ragContext(rag),
|
||||||
|
base.includes(TOOL_MARKER) ? "" : toolContext(tools),
|
||||||
|
]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join("\n\n");
|
||||||
|
}
|
||||||
|
export function speechInstructions(instructions: string, config: ServerConfig) {
|
||||||
|
return config.reasoningType === "OPENAI"
|
||||||
|
? `${instructions}\nDo not answer ordinary user speech directly. Stay silent unless the latest user message starts with [VOCERO_RENDER_REPLY]. When it does, speak only the content after the marker naturally. Do not call tools while rendering that message.`
|
||||||
|
: instructions;
|
||||||
|
}
|
||||||
|
export const WEB_SEARCH: FunctionTool = {
|
||||||
|
type: "function",
|
||||||
|
name: "web_search",
|
||||||
|
description: "Search the web for current or factual information. Returns results with titles, snippets and URLs.",
|
||||||
|
parameters: {
|
||||||
|
type: "object",
|
||||||
|
properties: { query: { type: "string", description: "The search query." } },
|
||||||
|
required: ["query"],
|
||||||
|
},
|
||||||
|
};
|
||||||
@@ -0,0 +1,133 @@
|
|||||||
|
import { z } from "zod";
|
||||||
|
|
||||||
|
export const recordSchema = z.record(z.string(), z.unknown());
|
||||||
|
export type Profile = z.infer<typeof recordSchema>;
|
||||||
|
|
||||||
|
export const configSchema = z.object({
|
||||||
|
search: z.boolean().default(false),
|
||||||
|
lb: z.boolean().default(false),
|
||||||
|
allowDirect: z.boolean().default(true),
|
||||||
|
s2sUrl: z.string().default(""),
|
||||||
|
primaryPrompt: z.string().default(""),
|
||||||
|
reasoningType: z.enum(["PROXY", "OPENAI", "NODE_RED"]).catch("PROXY"),
|
||||||
|
injectProfile: z.boolean().default(false),
|
||||||
|
debugPrompt: z.boolean().default(false),
|
||||||
|
startSessionMode: z.string().default(""),
|
||||||
|
theme: z.string().default(""),
|
||||||
|
apipilot: z.boolean().default(false),
|
||||||
|
});
|
||||||
|
export type ServerConfig = z.infer<typeof configSchema>;
|
||||||
|
export const defaultConfig = configSchema.parse({});
|
||||||
|
|
||||||
|
export const reportsSchema = z.object({
|
||||||
|
count: z.number().default(0),
|
||||||
|
hints: z.array(z.string()).default([]),
|
||||||
|
entries: z
|
||||||
|
.array(
|
||||||
|
z.object({
|
||||||
|
sourceScheduleLabel: z.string().nullish(),
|
||||||
|
date: z.union([z.string(), z.number()]).nullish(),
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.default([]),
|
||||||
|
reason: z.string().optional(),
|
||||||
|
truncated: z.boolean().optional(),
|
||||||
|
note: z.string().optional(),
|
||||||
|
});
|
||||||
|
export type Reports = z.infer<typeof reportsSchema>;
|
||||||
|
|
||||||
|
export const toolSchema = z.object({
|
||||||
|
name: z.string(),
|
||||||
|
description: z.string().default(""),
|
||||||
|
parameters: recordSchema.default({}),
|
||||||
|
requires_auth: z.boolean().default(false),
|
||||||
|
});
|
||||||
|
export type DynamicTool = z.infer<typeof toolSchema>;
|
||||||
|
export interface FunctionTool {
|
||||||
|
type: "function";
|
||||||
|
name: string;
|
||||||
|
description: string;
|
||||||
|
parameters: Record<string, unknown>;
|
||||||
|
}
|
||||||
|
export const toolsEnvelopeSchema = z.object({
|
||||||
|
success: z.boolean(),
|
||||||
|
data: z.object({ tools: z.array(toolSchema), count: z.number().optional() }).nullish(),
|
||||||
|
error: z.unknown().optional(),
|
||||||
|
});
|
||||||
|
|
||||||
|
export const grantSchema = z.object({
|
||||||
|
session_id: z.string(),
|
||||||
|
connect_url: z.string(),
|
||||||
|
pending_timeout_s: z.number().default(60),
|
||||||
|
});
|
||||||
|
export type SessionGrant = z.infer<typeof grantSchema>;
|
||||||
|
export const queueSchema = z.object({
|
||||||
|
state: z.literal("queued"),
|
||||||
|
queue_id: z.string(),
|
||||||
|
position: z.number().default(0),
|
||||||
|
poll_interval_s: z.number().default(2),
|
||||||
|
});
|
||||||
|
export const sessionSchema = z.union([queueSchema, grantSchema]);
|
||||||
|
|
||||||
|
export type ConnectionStatus =
|
||||||
|
| "idle"
|
||||||
|
| "connecting"
|
||||||
|
| "queued"
|
||||||
|
| "your-turn"
|
||||||
|
| "listening"
|
||||||
|
| "user-speaking"
|
||||||
|
| "processing"
|
||||||
|
| "ai-speaking"
|
||||||
|
| "closed"
|
||||||
|
| "error";
|
||||||
|
export interface Transcript {
|
||||||
|
role: "user" | "assistant";
|
||||||
|
text: string;
|
||||||
|
partial: boolean;
|
||||||
|
id: string;
|
||||||
|
}
|
||||||
|
export interface FinishedResponse {
|
||||||
|
responseId: string;
|
||||||
|
status: string;
|
||||||
|
transcript: string;
|
||||||
|
audible: boolean;
|
||||||
|
}
|
||||||
|
export interface ToolCall {
|
||||||
|
name: string;
|
||||||
|
arguments: string;
|
||||||
|
callId: string;
|
||||||
|
}
|
||||||
|
export interface NoiseGate {
|
||||||
|
enabled: boolean;
|
||||||
|
thresholdDb: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
const contentSchema = z.object({
|
||||||
|
type: z.string().optional(),
|
||||||
|
text: z.string().optional(),
|
||||||
|
transcript: z.string().optional(),
|
||||||
|
});
|
||||||
|
// The wire protocol is additive; unknown event fields are ignored, while consumed fields are validated.
|
||||||
|
export const realtimeEventSchema = z.object({
|
||||||
|
type: z.string(),
|
||||||
|
response_id: z.string().optional(),
|
||||||
|
item_id: z.string().optional(),
|
||||||
|
delta: z.string().optional(),
|
||||||
|
transcript: z.string().optional(),
|
||||||
|
text: z.string().optional(),
|
||||||
|
name: z.string().optional(),
|
||||||
|
arguments: z.string().optional(),
|
||||||
|
call_id: z.string().optional(),
|
||||||
|
response: z
|
||||||
|
.object({
|
||||||
|
id: z.string().optional(),
|
||||||
|
status: z.string().optional(),
|
||||||
|
output: z.array(z.object({ content: z.array(contentSchema).optional() })).optional(),
|
||||||
|
})
|
||||||
|
.optional(),
|
||||||
|
part: contentSchema.optional(),
|
||||||
|
error: z
|
||||||
|
.object({ type: z.string().optional(), code: z.string().optional(), message: z.string().optional() })
|
||||||
|
.optional(),
|
||||||
|
});
|
||||||
|
export type RealtimeEvent = z.infer<typeof realtimeEventSchema>;
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
|
||||||
|
import { conversationText, renderMarkdown, serializeConversation } from "./messages";
|
||||||
|
|
||||||
|
describe("assistant Markdown and export", () => {
|
||||||
|
it("renders formatting and accessible tables while keeping embedded HTML passive", () => {
|
||||||
|
const html = renderMarkdown(
|
||||||
|
"## Resumen\n\n**Bien**\n\n| A | B |\n|---|---|\n|1|2|\n\n<script>alert(1)</script>\n\n[malicioso](javascript:alert%281%29)\n\n[reporte](https://example.test)",
|
||||||
|
);
|
||||||
|
const container = document.createElement("div");
|
||||||
|
container.innerHTML = html;
|
||||||
|
expect(container.querySelector("h2")?.textContent).toBe("Resumen");
|
||||||
|
expect(container.querySelector("strong")?.textContent).toBe("Bien");
|
||||||
|
expect(container.querySelector("[role='region']")?.getAttribute("tabindex")).toBe("0");
|
||||||
|
expect(container.querySelector("script")).toBeNull();
|
||||||
|
expect(container.querySelector("[href^='javascript:']")).toBeNull();
|
||||||
|
expect(container.querySelector("a")?.getAttribute("rel")).toBe("noopener noreferrer");
|
||||||
|
expect(container.textContent).toContain("<script>alert(1)</script>");
|
||||||
|
});
|
||||||
|
it("exports the original Markdown and interrupted/tool records without scraping the UI", () => {
|
||||||
|
const conversation = serializeConversation(
|
||||||
|
[
|
||||||
|
{ id: "1", role: "user", text: "Hola", partial: false, timestamp: "2026-10-06T15:00:00Z" },
|
||||||
|
{
|
||||||
|
id: "2",
|
||||||
|
role: "assistant",
|
||||||
|
text: "**Respuesta** parcial",
|
||||||
|
partial: false,
|
||||||
|
interrupted: true,
|
||||||
|
timestamp: "2026-10-06T15:00:01Z",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "3",
|
||||||
|
role: "tool",
|
||||||
|
text: "Resultado",
|
||||||
|
toolName: "web_search",
|
||||||
|
toolArguments: '{"query":"hoy"}',
|
||||||
|
partial: false,
|
||||||
|
timestamp: "2026-10-06T15:00:02Z",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
"Ana Vargas",
|
||||||
|
);
|
||||||
|
expect(conversation.messageCount).toBe(3);
|
||||||
|
expect(conversation.messages[0]).toMatchObject({ name: "Ana Vargas", initials: "AV" });
|
||||||
|
expect(conversation.messages[1]).toMatchObject({ text: "**Respuesta** parcial", interrupted: true });
|
||||||
|
expect(conversationText(conversation)).toContain("[interrumpida]");
|
||||||
|
expect(conversationText(conversation)).toContain('Entrada:\n{"query":"hoy"}');
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,174 @@
|
|||||||
|
import DOMPurify from "dompurify";
|
||||||
|
import { Marked } from "marked";
|
||||||
|
|
||||||
|
import { initials } from "./context";
|
||||||
|
|
||||||
|
export interface Message {
|
||||||
|
id: string;
|
||||||
|
role: "user" | "assistant" | "tool";
|
||||||
|
text: string;
|
||||||
|
timestamp: string;
|
||||||
|
partial: boolean;
|
||||||
|
interrupted?: boolean;
|
||||||
|
toolName?: string;
|
||||||
|
toolArguments?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
function escapeHtml(text: string) {
|
||||||
|
return text.replace(
|
||||||
|
/[&<>"']/g,
|
||||||
|
(character) => ({ "&": "&", "<": "<", ">": ">", '"': """, "'": "'" })[character] ?? character,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const markdown = new Marked({
|
||||||
|
gfm: true,
|
||||||
|
breaks: true,
|
||||||
|
renderer: { html: ({ text }) => escapeHtml(text), checkbox: ({ checked }) => (checked ? "☑ " : "☐ ") },
|
||||||
|
});
|
||||||
|
|
||||||
|
/** Only passive, sanitized Markdown is allowed inside an assistant message. */
|
||||||
|
export function renderMarkdown(text: string): string {
|
||||||
|
if (!DOMPurify.isSupported) {
|
||||||
|
return escapeHtml(text);
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const fragment = DOMPurify.sanitize(markdown.parse(text, { async: false }), {
|
||||||
|
ALLOWED_TAGS: [
|
||||||
|
"p",
|
||||||
|
"br",
|
||||||
|
"strong",
|
||||||
|
"em",
|
||||||
|
"del",
|
||||||
|
"a",
|
||||||
|
"img",
|
||||||
|
"ul",
|
||||||
|
"ol",
|
||||||
|
"li",
|
||||||
|
"h1",
|
||||||
|
"h2",
|
||||||
|
"h3",
|
||||||
|
"h4",
|
||||||
|
"h5",
|
||||||
|
"h6",
|
||||||
|
"blockquote",
|
||||||
|
"pre",
|
||||||
|
"code",
|
||||||
|
"hr",
|
||||||
|
"table",
|
||||||
|
"thead",
|
||||||
|
"tbody",
|
||||||
|
"tr",
|
||||||
|
"th",
|
||||||
|
"td",
|
||||||
|
],
|
||||||
|
ALLOWED_ATTR: ["href", "title", "src", "alt", "start", "align"],
|
||||||
|
ALLOW_DATA_ATTR: false,
|
||||||
|
ALLOW_ARIA_ATTR: false,
|
||||||
|
RETURN_DOM_FRAGMENT: true,
|
||||||
|
});
|
||||||
|
for (const link of fragment.querySelectorAll("a")) {
|
||||||
|
if (!link.hasAttribute("href")) {
|
||||||
|
link.replaceWith(...link.childNodes);
|
||||||
|
} else {
|
||||||
|
link.target = "_blank";
|
||||||
|
link.rel = "noopener noreferrer";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const image of fragment.querySelectorAll("img")) {
|
||||||
|
image.loading = "lazy";
|
||||||
|
}
|
||||||
|
for (const table of fragment.querySelectorAll("table")) {
|
||||||
|
const wrapper = document.createElement("div");
|
||||||
|
wrapper.className = "markdown-table";
|
||||||
|
wrapper.tabIndex = 0;
|
||||||
|
wrapper.setAttribute("role", "region");
|
||||||
|
wrapper.setAttribute("aria-label", "Tabla de la respuesta");
|
||||||
|
table.replaceWith(wrapper);
|
||||||
|
wrapper.append(table);
|
||||||
|
}
|
||||||
|
for (const code of fragment.querySelectorAll("pre")) {
|
||||||
|
code.tabIndex = 0;
|
||||||
|
}
|
||||||
|
const container = document.createElement("div");
|
||||||
|
container.append(fragment);
|
||||||
|
return container.innerHTML;
|
||||||
|
} catch {
|
||||||
|
return escapeHtml(text);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function displayTime(timestamp: string) {
|
||||||
|
return new Intl.DateTimeFormat("es-CL", { timeZone: "America/Santiago", hour: "2-digit", minute: "2-digit" }).format(
|
||||||
|
new Date(timestamp),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
export function serializeConversation(messages: Message[], userName: string) {
|
||||||
|
return {
|
||||||
|
exportedAt: new Date().toISOString(),
|
||||||
|
messageCount: messages.length,
|
||||||
|
messages: messages.map((message) =>
|
||||||
|
message.role === "tool"
|
||||||
|
? {
|
||||||
|
type: "tool" as const,
|
||||||
|
timestamp: message.timestamp,
|
||||||
|
name: message.toolName ?? "",
|
||||||
|
args: message.toolArguments ?? "",
|
||||||
|
output: message.text,
|
||||||
|
}
|
||||||
|
: {
|
||||||
|
type: message.role,
|
||||||
|
timestamp: message.timestamp,
|
||||||
|
text: message.text,
|
||||||
|
...(message.role === "user"
|
||||||
|
? { name: userName || "Usuario", initials: initials(userName || "Usuario") }
|
||||||
|
: {}),
|
||||||
|
...(message.interrupted ? { interrupted: true } : {}),
|
||||||
|
},
|
||||||
|
),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
export function conversationText(conversation: ReturnType<typeof serializeConversation>) {
|
||||||
|
const lines = [
|
||||||
|
`Conversación Vocero — ${conversation.exportedAt}`,
|
||||||
|
`Mensajes totales: ${String(conversation.messageCount)}`,
|
||||||
|
"—".repeat(52),
|
||||||
|
"",
|
||||||
|
];
|
||||||
|
for (const message of conversation.messages) {
|
||||||
|
if (message.type === "tool") {
|
||||||
|
lines.push(
|
||||||
|
`${message.timestamp} HERRAMIENTA: ${message.name ?? ""}`,
|
||||||
|
`Entrada:\n${message.args ?? ""}`,
|
||||||
|
`Salida:\n${message.output ?? ""}`,
|
||||||
|
"",
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
lines.push(
|
||||||
|
`${message.timestamp} ${message.type === "user" ? (message.name ?? "Usuario") : "Vocero"}${message.interrupted ? " [interrumpida]" : ""}:`,
|
||||||
|
message.text ?? "",
|
||||||
|
"",
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return lines.join("\n");
|
||||||
|
}
|
||||||
|
export function exportConversation(messages: Message[], userName: string) {
|
||||||
|
const conversation = serializeConversation(messages, userName);
|
||||||
|
const name = `vocero-conversacion-${conversation.exportedAt.replace(/[:.]/g, "-")}`;
|
||||||
|
const downloads = [
|
||||||
|
{ content: JSON.stringify(conversation, null, 2), extension: "json", mime: "application/json" },
|
||||||
|
{ content: conversationText(conversation), extension: "txt", mime: "text/plain;charset=utf-8" },
|
||||||
|
];
|
||||||
|
for (const { content, extension, mime } of downloads) {
|
||||||
|
const url = URL.createObjectURL(new Blob([content], { type: mime }));
|
||||||
|
const link = document.createElement("a");
|
||||||
|
link.href = url;
|
||||||
|
link.download = `${name}.${extension}`;
|
||||||
|
document.body.append(link);
|
||||||
|
link.click();
|
||||||
|
link.remove();
|
||||||
|
setTimeout(() => {
|
||||||
|
URL.revokeObjectURL(url);
|
||||||
|
}, 4000);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,160 @@
|
|||||||
|
import { recordSchema, type Profile } from "./contracts";
|
||||||
|
|
||||||
|
function field(profile: Profile | null, keys: readonly string[], depth = 0): string {
|
||||||
|
if (!profile || depth > 3) {
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
for (const key of keys) {
|
||||||
|
const value = profile[key];
|
||||||
|
if (typeof value === "string" && value.trim()) {
|
||||||
|
return value.trim();
|
||||||
|
}
|
||||||
|
if (typeof value === "number" && Number.isFinite(value)) {
|
||||||
|
return String(value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const key of ["user", "customer", "profile", "contact"]) {
|
||||||
|
const nested = recordSchema.safeParse(profile[key]);
|
||||||
|
if (nested.success) {
|
||||||
|
const value = field(nested.data, keys, depth + 1);
|
||||||
|
if (value) {
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
const FIELDS = [
|
||||||
|
{
|
||||||
|
label: "Correo electrónico",
|
||||||
|
keys: [
|
||||||
|
"email",
|
||||||
|
"emailAddress",
|
||||||
|
"email_address",
|
||||||
|
"eMail",
|
||||||
|
"mail",
|
||||||
|
"contactEmail",
|
||||||
|
"user_email",
|
||||||
|
"customer_email",
|
||||||
|
"loginEmail",
|
||||||
|
"username_email",
|
||||||
|
],
|
||||||
|
kind: "email",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Teléfono",
|
||||||
|
keys: [
|
||||||
|
"phone",
|
||||||
|
"phoneNumber",
|
||||||
|
"phone_number",
|
||||||
|
"mobile",
|
||||||
|
"mobileNumber",
|
||||||
|
"telephone",
|
||||||
|
"tel",
|
||||||
|
"cellphone",
|
||||||
|
"contactPhone",
|
||||||
|
"user_phone",
|
||||||
|
"whatsapp",
|
||||||
|
],
|
||||||
|
kind: "phone",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Nombre de usuario",
|
||||||
|
keys: ["username", "userName", "user_name", "login", "handle", "nickname", "screenName", "accountName"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Rol / Plan",
|
||||||
|
keys: ["role", "roleName", "userRole", "plan", "planName", "tier", "subscription", "membership", "accessLevel"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Empresa",
|
||||||
|
keys: ["company", "companyName", "company_name", "organization", "org", "organizationName", "business"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "ID de usuario",
|
||||||
|
keys: [
|
||||||
|
"userId",
|
||||||
|
"user_id",
|
||||||
|
"uid",
|
||||||
|
"accountId",
|
||||||
|
"account_id",
|
||||||
|
"sub",
|
||||||
|
"customerId",
|
||||||
|
"customer_id",
|
||||||
|
"id",
|
||||||
|
"userPk",
|
||||||
|
"profileId",
|
||||||
|
"profile_id",
|
||||||
|
"identityId",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "País / Idioma",
|
||||||
|
keys: [
|
||||||
|
"country",
|
||||||
|
"countryName",
|
||||||
|
"country_code",
|
||||||
|
"countryCode",
|
||||||
|
"locale",
|
||||||
|
"language",
|
||||||
|
"lang",
|
||||||
|
"region",
|
||||||
|
"timezone",
|
||||||
|
"tz",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Fecha de registro",
|
||||||
|
keys: [
|
||||||
|
"createdAt",
|
||||||
|
"created_at",
|
||||||
|
"created",
|
||||||
|
"createdDate",
|
||||||
|
"created_date",
|
||||||
|
"signUpDate",
|
||||||
|
"signup_date",
|
||||||
|
"joined",
|
||||||
|
"joinDate",
|
||||||
|
"join_date",
|
||||||
|
"registrationDate",
|
||||||
|
],
|
||||||
|
kind: "date",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Último acceso",
|
||||||
|
keys: [
|
||||||
|
"lastLogin",
|
||||||
|
"last_login",
|
||||||
|
"lastLoginAt",
|
||||||
|
"lastActive",
|
||||||
|
"last_active",
|
||||||
|
"lastActiveAt",
|
||||||
|
"updatedAt",
|
||||||
|
"updated_at",
|
||||||
|
],
|
||||||
|
kind: "date",
|
||||||
|
},
|
||||||
|
] satisfies { label: string; keys: string[]; kind?: string }[];
|
||||||
|
|
||||||
|
export function profileRows(profile: Profile | null) {
|
||||||
|
return FIELDS.flatMap(({ label, keys, kind }) => {
|
||||||
|
let value = field(profile, keys);
|
||||||
|
if (!value) {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
if (kind === "date") {
|
||||||
|
const timestamp = /^\d{10,13}$/.test(value) ? Number(value) * (value.length <= 10 ? 1000 : 1) : value;
|
||||||
|
const date = new Date(timestamp);
|
||||||
|
if (!Number.isNaN(date.getTime())) {
|
||||||
|
value = new Intl.DateTimeFormat("es-CL", {
|
||||||
|
timeZone: "America/Santiago",
|
||||||
|
dateStyle: "medium",
|
||||||
|
timeStyle: "short",
|
||||||
|
}).format(date);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const href =
|
||||||
|
kind === "email" ? `mailto:${value}` : kind === "phone" ? `tel:${value.replace(/[^\d+]/g, "")}` : undefined;
|
||||||
|
return [{ label, value, href }];
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
import type { Reports } from "./contracts";
|
||||||
|
|
||||||
|
/** Keep the original report-based conversation suggestions in the debug sidebar. */
|
||||||
|
export function reportSuggestions(reports: Reports) {
|
||||||
|
const labels = reports.entries
|
||||||
|
.map((entry) => entry.sourceScheduleLabel?.trim())
|
||||||
|
.filter((label): label is string => !!label);
|
||||||
|
const first = labels[0];
|
||||||
|
if (!first) {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
const second = labels[1] ?? first;
|
||||||
|
const third = labels[2] ?? first;
|
||||||
|
const templates: [RegExp, (label: string) => string][] = [
|
||||||
|
[
|
||||||
|
/riesgo|mortalidad|porcentaje|kpi/i,
|
||||||
|
(label) => `Dame un resumen ejecutivo de los riesgos operativos y KPIs del reporte de ${label}.`,
|
||||||
|
],
|
||||||
|
[/diari[oa]|informe/i, (label) => `Cuéntame los puntos clave del informe diario más reciente (${label}).`],
|
||||||
|
[/métrica|metrica|medici[oó]n/i, (label) => `Compara las métricas de la última semana del reporte ${label}.`],
|
||||||
|
[/producto|servicio/i, (label) => `¿Qué hay de nuevo en productos y servicios según el reporte ${label}?`],
|
||||||
|
[
|
||||||
|
/supply|cadena|log[ií]stica|env[ií]o|chile|latin/i,
|
||||||
|
(label) => `¿Hay novedades en la cadena de suministro o logística en el reporte ${label}?`,
|
||||||
|
],
|
||||||
|
[
|
||||||
|
/synapse|ejecuci[oó]n|calendario/i,
|
||||||
|
(label) => `¿Qué ejecuciones del calendario Synapse TI están reportadas? (${label}).`,
|
||||||
|
],
|
||||||
|
[/podcast|briefing|mañana|matutino|morning/i, (label) => `Lee el podcast briefing matutino de hoy (${label}).`],
|
||||||
|
[
|
||||||
|
/financier|finanza|costo|presupuesto|venta/i,
|
||||||
|
(label) => `Análisis financiero: costos, presupuesto y ventas del reporte ${label}.`,
|
||||||
|
],
|
||||||
|
[
|
||||||
|
/ejecutivo|ceo|reuni[oó]n|semanal/i,
|
||||||
|
(label) => `Puntos de agenda para la reunión ejecutiva semanal según el reporte ${label}.`,
|
||||||
|
],
|
||||||
|
];
|
||||||
|
const contextual = templates.flatMap(([pattern, text]) => {
|
||||||
|
const label = labels.find((label) => pattern.test(label));
|
||||||
|
return label ? [text(label)] : [];
|
||||||
|
});
|
||||||
|
return [
|
||||||
|
...contextual,
|
||||||
|
`¿Qué highlights hay en el reporte más reciente «${first}»?`,
|
||||||
|
`Resume los 3 hallazgos principales de los reportes de hoy (${String(reports.count)} disponibles).`,
|
||||||
|
...(reports.count >= 2 ? [`Compara ${first} vs ${second} y dime cuál tiene mayor prioridad.`] : []),
|
||||||
|
...(reports.count >= 3 ? [`Plan de acción sugerido con base en ${first}, ${second} y ${third}.`] : []),
|
||||||
|
].slice(0, 4);
|
||||||
|
}
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
import { recordSchema, type ServerConfig } from "./contracts";
|
||||||
|
|
||||||
|
export const VOICES = ["Aiden", "Ryan", "Dylan", "Eric", "Ono_Anna", "Serena", "Sohee", "Uncle_Fu", "Vivian"];
|
||||||
|
export const DEFAULT_INSTRUCTIONS =
|
||||||
|
"You are Vocero, a friendly voice assistant. Keep replies short, warm, and spoken. Avoid long monologues. Respond ALWAYS and ONLY in Spanish, regardless of the language used by the user or any other instruction. No responses in Italian, English, Portuguese, or any other language are allowed. Every spoken sentence, greeting, confirmation, clarification, and error message must be in Spanish. The comma sign ',' in numbers is used to separate thousands. The point sign '.' in numbers is used to separate decimals. If you don't know the answer, say so in Spanish instead of guessing.";
|
||||||
|
export const GATE_OFF = -66;
|
||||||
|
export const GATE_MAX = -3;
|
||||||
|
export interface Settings {
|
||||||
|
directUrl: string;
|
||||||
|
voice: string;
|
||||||
|
instructions: string;
|
||||||
|
noiseGate: number;
|
||||||
|
webSearch: boolean;
|
||||||
|
searchKey: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
function read(key: string) {
|
||||||
|
try {
|
||||||
|
return localStorage.getItem(`s2s.ws.${key}`);
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export function loadSettings(): Settings {
|
||||||
|
const threshold = read("noiseGate");
|
||||||
|
const number = threshold === null || threshold === "" ? -50 : Number(threshold);
|
||||||
|
let webSearch = true;
|
||||||
|
try {
|
||||||
|
const data = recordSchema.parse(JSON.parse(read("tools") ?? "{}"));
|
||||||
|
webSearch = typeof data.web_search === "boolean" ? data.web_search : true;
|
||||||
|
} catch {
|
||||||
|
/* Ignore unavailable or older storage. */
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
directUrl: read("directUrl") ?? "",
|
||||||
|
voice: read("voice") || "Aiden",
|
||||||
|
instructions: read("instructions") || DEFAULT_INSTRUCTIONS,
|
||||||
|
noiseGate: Number.isFinite(number) ? Math.min(GATE_MAX, Math.max(GATE_OFF, Math.round(number))) : -50,
|
||||||
|
webSearch,
|
||||||
|
searchKey: read("searchKey") ?? "",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
export function saveSettings(settings: Settings) {
|
||||||
|
try {
|
||||||
|
for (const key of ["directUrl", "voice", "instructions", "noiseGate", "searchKey"] as const) {
|
||||||
|
localStorage.setItem(`s2s.ws.${key}`, String(settings[key]));
|
||||||
|
}
|
||||||
|
localStorage.setItem("s2s.ws.tools", JSON.stringify({ web_search: settings.webSearch }));
|
||||||
|
} catch {
|
||||||
|
/* The current session still works when browser storage is disabled. */
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface Caller {
|
||||||
|
phoneNumber: string;
|
||||||
|
episodeId: string;
|
||||||
|
}
|
||||||
|
export function callerFromLocation(search = window.location.search): Caller {
|
||||||
|
const query = new URLSearchParams(search);
|
||||||
|
return { phoneNumber: (query.get("phoneNumber") ?? "").trim(), episodeId: (query.get("episode_id") ?? "").trim() };
|
||||||
|
}
|
||||||
|
export function realtimeUrl(raw: string, caller: Caller): string {
|
||||||
|
let value = raw.trim();
|
||||||
|
if (!value) {
|
||||||
|
throw new Error("Introduce la URL del servidor de voz en Configuración.");
|
||||||
|
}
|
||||||
|
if (/^https?:\/\//i.test(value)) {
|
||||||
|
value = value.replace(/^http/i, "ws");
|
||||||
|
}
|
||||||
|
if (!/^wss?:\/\//i.test(value)) {
|
||||||
|
value = (/^(localhost|127\.0\.0\.1|\[::1\])(:|\/|$)/i.test(value) ? "ws://" : "wss://") + value;
|
||||||
|
}
|
||||||
|
const url = new URL(value);
|
||||||
|
if (url.pathname === "/") {
|
||||||
|
url.pathname = "/v1/realtime";
|
||||||
|
}
|
||||||
|
if (caller.phoneNumber) {
|
||||||
|
url.searchParams.set("phoneNumber", caller.phoneNumber);
|
||||||
|
}
|
||||||
|
if (caller.episodeId) {
|
||||||
|
url.searchParams.set("episode_id", caller.episodeId);
|
||||||
|
}
|
||||||
|
return url.toString();
|
||||||
|
}
|
||||||
|
export function connectionTarget(config: ServerConfig, settings: Settings, caller: Caller) {
|
||||||
|
return config.lb || !config.allowDirect
|
||||||
|
? { sessionUrl: "/api/session" }
|
||||||
|
: { directUrl: realtimeUrl(config.s2sUrl || settings.directUrl, caller) };
|
||||||
|
}
|
||||||
|
export function isDebug(search = window.location.search) {
|
||||||
|
const value = new URLSearchParams(search).get("debug")?.toLowerCase();
|
||||||
|
return value === "true" || value === "1";
|
||||||
|
}
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
import { StrictMode } from "react";
|
||||||
|
import { createRoot } from "react-dom/client";
|
||||||
|
import "./styles.css";
|
||||||
|
import App from "./App.tsx";
|
||||||
|
|
||||||
|
const root = document.getElementById("root");
|
||||||
|
if (!root) {
|
||||||
|
throw new Error("The application root was not found.");
|
||||||
|
}
|
||||||
|
createRoot(root).render(
|
||||||
|
<StrictMode>
|
||||||
|
<App />
|
||||||
|
</StrictMode>,
|
||||||
|
);
|
||||||
@@ -0,0 +1,437 @@
|
|||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
|
||||||
|
import type { AudioEngine } from "../audio/engine";
|
||||||
|
|
||||||
|
import { RealtimeClient, SessionError, type ClientOptions } from "./client";
|
||||||
|
|
||||||
|
const audio = vi.hoisted(() => ({
|
||||||
|
start: vi.fn<AudioEngine["start"]>(),
|
||||||
|
close: vi.fn<() => Promise<void>>(),
|
||||||
|
play: vi.fn(),
|
||||||
|
clear: vi.fn(),
|
||||||
|
setMuted: vi.fn(),
|
||||||
|
setGate: vi.fn(),
|
||||||
|
}));
|
||||||
|
vi.mock("../audio/engine", () => ({
|
||||||
|
MIC_CONSTRAINTS: { audio: true },
|
||||||
|
AudioEngine: class {
|
||||||
|
start = audio.start;
|
||||||
|
close = audio.close;
|
||||||
|
play = audio.play;
|
||||||
|
clear = audio.clear;
|
||||||
|
setMuted = audio.setMuted;
|
||||||
|
setGate = audio.setGate;
|
||||||
|
},
|
||||||
|
}));
|
||||||
|
|
||||||
|
class Socket extends EventTarget {
|
||||||
|
static OPEN = 1;
|
||||||
|
static instances: Socket[] = [];
|
||||||
|
readyState = 1;
|
||||||
|
binaryType = "";
|
||||||
|
sent: Record<string, unknown>[] = [];
|
||||||
|
readonly url: string;
|
||||||
|
constructor(url: string) {
|
||||||
|
super();
|
||||||
|
this.url = url;
|
||||||
|
Socket.instances.push(this);
|
||||||
|
}
|
||||||
|
send(text: string) {
|
||||||
|
this.sent.push(JSON.parse(text) as Record<string, unknown>);
|
||||||
|
}
|
||||||
|
close = vi.fn(() => {
|
||||||
|
this.readyState = 3;
|
||||||
|
});
|
||||||
|
receive(value: unknown) {
|
||||||
|
this.dispatchEvent(new MessageEvent("message", { data: JSON.stringify(value) }));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const stopTrack = vi.fn();
|
||||||
|
const getUserMedia = vi.fn();
|
||||||
|
const sendBeacon = vi.fn(() => true);
|
||||||
|
const options = (): ClientOptions => ({
|
||||||
|
directUrl: "ws://localhost:8420/v1/realtime",
|
||||||
|
context: { resume: vi.fn().mockResolvedValue(undefined) } as unknown as AudioContext,
|
||||||
|
voice: "Aiden",
|
||||||
|
instructions: "Habla español",
|
||||||
|
tools: [],
|
||||||
|
gate: { enabled: false, thresholdDb: -66 },
|
||||||
|
});
|
||||||
|
|
||||||
|
async function connect(client: RealtimeClient, pending = client.connect()) {
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(Socket.instances).toHaveLength(1);
|
||||||
|
});
|
||||||
|
const socket = Socket.instances[0];
|
||||||
|
if (!socket) {
|
||||||
|
throw new Error("Missing socket");
|
||||||
|
}
|
||||||
|
socket.receive({ type: "session.created" });
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(socket.sent[0]?.type).toBe("session.update");
|
||||||
|
});
|
||||||
|
await pending;
|
||||||
|
return socket;
|
||||||
|
}
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.useFakeTimers();
|
||||||
|
vi.clearAllMocks();
|
||||||
|
Socket.instances = [];
|
||||||
|
audio.start.mockResolvedValue(undefined);
|
||||||
|
audio.close.mockResolvedValue(undefined);
|
||||||
|
getUserMedia.mockResolvedValue({ getTracks: () => [{ stop: stopTrack }] });
|
||||||
|
vi.stubGlobal("navigator", { mediaDevices: { getUserMedia }, sendBeacon });
|
||||||
|
vi.stubGlobal("WebSocket", Socket);
|
||||||
|
});
|
||||||
|
afterEach(() => {
|
||||||
|
vi.useRealTimers();
|
||||||
|
vi.unstubAllGlobals();
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("realtime session", () => {
|
||||||
|
it("starts microphone streaming without a session.updated acknowledgment", async () => {
|
||||||
|
const client = new RealtimeClient(options());
|
||||||
|
const pending = client.connect();
|
||||||
|
const settled = pending.catch(() => undefined);
|
||||||
|
try {
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(audio.start).toHaveBeenCalledOnce();
|
||||||
|
});
|
||||||
|
const socket = Socket.instances[0];
|
||||||
|
const capture = audio.start.mock.calls[0]?.[0];
|
||||||
|
if (!socket || !capture) {
|
||||||
|
throw new Error("Missing socket or microphone callback");
|
||||||
|
}
|
||||||
|
const pcm = new ArrayBuffer(2);
|
||||||
|
capture(pcm);
|
||||||
|
expect(socket.sent).toHaveLength(0);
|
||||||
|
|
||||||
|
socket.receive({ type: "session.created" });
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(client.status).toBe("listening");
|
||||||
|
});
|
||||||
|
await expect(pending).resolves.toBeUndefined();
|
||||||
|
capture(pcm);
|
||||||
|
expect(socket.sent).toEqual([
|
||||||
|
expect.objectContaining({ type: "session.update" }),
|
||||||
|
{ type: "input_audio_buffer.append", audio: "AAA=" },
|
||||||
|
]);
|
||||||
|
await vi.advanceTimersByTimeAsync(21000);
|
||||||
|
expect(client.status).toBe("listening");
|
||||||
|
expect(socket.close).not.toHaveBeenCalled();
|
||||||
|
} finally {
|
||||||
|
await client.close();
|
||||||
|
await settled;
|
||||||
|
}
|
||||||
|
expect(audio.close).toHaveBeenCalledOnce();
|
||||||
|
expect(vi.getTimerCount()).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("times out and releases the microphone when the server never creates a session", async () => {
|
||||||
|
const client = new RealtimeClient(options());
|
||||||
|
const rejected = expect(client.connect()).rejects.toThrow("El servidor de voz no respondió a tiempo.");
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(audio.start).toHaveBeenCalledOnce();
|
||||||
|
});
|
||||||
|
await vi.advanceTimersByTimeAsync(20000);
|
||||||
|
await rejected;
|
||||||
|
expect(Socket.instances[0]?.sent).toHaveLength(0);
|
||||||
|
expect(Socket.instances[0]?.close).toHaveBeenCalledExactlyOnceWith(1000, "client closed");
|
||||||
|
expect(audio.close).toHaveBeenCalledOnce();
|
||||||
|
expect(client.status).toBe("closed");
|
||||||
|
expect(vi.getTimerCount()).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("connects with a plain load-balancer grant and closes without further HTTP requests", async () => {
|
||||||
|
const fetchMock = vi.fn().mockResolvedValueOnce(
|
||||||
|
new Response(
|
||||||
|
JSON.stringify({
|
||||||
|
session_id: "local-session",
|
||||||
|
connect_url: "ws://localhost/v1/realtime",
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
const client = new RealtimeClient({ ...options(), directUrl: undefined, sessionUrl: "/api/session" });
|
||||||
|
const error = vi.fn();
|
||||||
|
client.on("error", error);
|
||||||
|
const socket = await connect(client);
|
||||||
|
await vi.advanceTimersByTimeAsync(60000);
|
||||||
|
expect(client.status).toBe("listening");
|
||||||
|
expect(error).not.toHaveBeenCalled();
|
||||||
|
expect(socket.url).toBe("ws://localhost/v1/realtime");
|
||||||
|
await client.close();
|
||||||
|
expect(socket.close).toHaveBeenCalledOnce();
|
||||||
|
expect(audio.close).toHaveBeenCalledOnce();
|
||||||
|
expect(fetchMock).toHaveBeenCalledOnce();
|
||||||
|
expect(fetchMock).toHaveBeenCalledWith("/api/session", expect.objectContaining({ method: "POST" }));
|
||||||
|
expect(sendBeacon).not.toHaveBeenCalled();
|
||||||
|
expect(client.status).toBe("closed");
|
||||||
|
expect(vi.getTimerCount()).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(["ws://voice.test/v1/realtime", "ws://localhost:8420/v1/realtime"])(
|
||||||
|
"starts and cleans up the direct or proxied socket at %s",
|
||||||
|
async (directUrl) => {
|
||||||
|
const fetchMock = vi.fn();
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
const client = new RealtimeClient({ ...options(), directUrl });
|
||||||
|
const socket = await connect(client);
|
||||||
|
expect(socket.url).toBe(directUrl);
|
||||||
|
expect(audio.start).toHaveBeenCalledOnce();
|
||||||
|
await client.close();
|
||||||
|
await client.close();
|
||||||
|
expect(socket.close).toHaveBeenCalledExactlyOnceWith(1000, "client closed");
|
||||||
|
expect(audio.close).toHaveBeenCalledOnce();
|
||||||
|
expect(fetchMock).not.toHaveBeenCalled();
|
||||||
|
expect(sendBeacon).not.toHaveBeenCalled();
|
||||||
|
expect(vi.getTimerCount()).toBe(0);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
it("polls a queue ticket and joins the granted slot before its deadline", async () => {
|
||||||
|
const fetchMock = vi
|
||||||
|
.fn<(path: string, init?: RequestInit) => Promise<Response>>()
|
||||||
|
.mockResolvedValueOnce(new Response(JSON.stringify({ state: "queued", queue_id: "q1", position: 2 })))
|
||||||
|
.mockResolvedValueOnce(new Response(JSON.stringify({ state: "queued", queue_id: "q1", position: 1 })))
|
||||||
|
.mockResolvedValueOnce(
|
||||||
|
new Response(JSON.stringify({ session_id: "s1", connect_url: "ws://voice.test", pending_timeout_s: 10 })),
|
||||||
|
);
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
const client = new RealtimeClient({ ...options(), directUrl: undefined });
|
||||||
|
const readyToJoin = vi.fn();
|
||||||
|
client.on("readyToJoin", readyToJoin);
|
||||||
|
const pending = client.connect();
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(client.status).toBe("queued");
|
||||||
|
});
|
||||||
|
await vi.advanceTimersByTimeAsync(4000);
|
||||||
|
expect(fetchMock.mock.calls.map(([path]) => path)).toEqual(["/api/session", "/api/queue/q1", "/api/queue/q1"]);
|
||||||
|
expect(client.status).toBe("your-turn");
|
||||||
|
expect(readyToJoin).toHaveBeenCalledWith(expect.objectContaining({ expiresSec: 7 }));
|
||||||
|
expect(Socket.instances).toHaveLength(0);
|
||||||
|
client.join();
|
||||||
|
await connect(client, pending);
|
||||||
|
await client.close();
|
||||||
|
expect(sendBeacon).not.toHaveBeenCalled();
|
||||||
|
expect(vi.getTimerCount()).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("expires an unclaimed join offer after releasing the permission probe's microphone", async () => {
|
||||||
|
vi.stubGlobal(
|
||||||
|
"fetch",
|
||||||
|
vi
|
||||||
|
.fn()
|
||||||
|
.mockResolvedValueOnce(new Response(JSON.stringify({ state: "queued", queue_id: "q1" })))
|
||||||
|
.mockResolvedValueOnce(
|
||||||
|
new Response(JSON.stringify({ session_id: "s1", connect_url: "ws://voice.test", pending_timeout_s: 10 })),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
const client = new RealtimeClient({ ...options(), directUrl: undefined });
|
||||||
|
const rejected = expect(client.connect()).rejects.toMatchObject({ code: "join-expired" });
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(client.status).toBe("queued");
|
||||||
|
});
|
||||||
|
await vi.advanceTimersByTimeAsync(9000);
|
||||||
|
await rejected;
|
||||||
|
expect(stopTrack).toHaveBeenCalledOnce();
|
||||||
|
expect(audio.start).not.toHaveBeenCalled();
|
||||||
|
expect(Socket.instances).toHaveLength(0);
|
||||||
|
expect(client.status).toBe("closed");
|
||||||
|
expect(sendBeacon).not.toHaveBeenCalled();
|
||||||
|
expect(vi.getTimerCount()).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("stops polling when the load balancer expires a queue ticket", async () => {
|
||||||
|
const fetchMock = vi
|
||||||
|
.fn()
|
||||||
|
.mockResolvedValueOnce(new Response(JSON.stringify({ state: "queued", queue_id: "q1" })))
|
||||||
|
.mockResolvedValueOnce(new Response(JSON.stringify({ state: "expired" }), { status: 404 }));
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
const client = new RealtimeClient({ ...options(), directUrl: undefined });
|
||||||
|
const rejected = expect(client.connect()).rejects.toMatchObject({ code: "queue-expired" });
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(client.status).toBe("queued");
|
||||||
|
});
|
||||||
|
await vi.advanceTimersByTimeAsync(2000);
|
||||||
|
await rejected;
|
||||||
|
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||||
|
expect(Socket.instances).toHaveLength(0);
|
||||||
|
expect(vi.getTimerCount()).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("uses a keepalive request to cancel a queue ticket when sendBeacon declines it", async () => {
|
||||||
|
sendBeacon.mockReturnValueOnce(false);
|
||||||
|
const fetchMock = vi
|
||||||
|
.fn()
|
||||||
|
.mockResolvedValueOnce(new Response(JSON.stringify({ state: "queued", queue_id: "q1" })))
|
||||||
|
.mockResolvedValueOnce(new Response(JSON.stringify({ ok: true })));
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
const client = new RealtimeClient({ ...options(), directUrl: undefined });
|
||||||
|
const rejected = expect(client.connect()).rejects.toMatchObject({ code: "aborted" });
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(client.status).toBe("queued");
|
||||||
|
});
|
||||||
|
await client.close();
|
||||||
|
await rejected;
|
||||||
|
expect(fetchMock).toHaveBeenLastCalledWith(
|
||||||
|
"/api/queue/end",
|
||||||
|
expect.objectContaining({ method: "POST", body: JSON.stringify({ queueId: "q1" }), keepalive: true }),
|
||||||
|
);
|
||||||
|
expect(vi.getTimerCount()).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("replaces cumulative user transcript deltas instead of duplicating words", async () => {
|
||||||
|
const client = new RealtimeClient(options());
|
||||||
|
const transcript = vi.fn();
|
||||||
|
client.on("transcript", transcript);
|
||||||
|
const socket = await connect(client);
|
||||||
|
socket.receive({ type: "conversation.item.input_audio_transcription.delta", item_id: "user1", delta: "Hola" });
|
||||||
|
socket.receive({
|
||||||
|
type: "conversation.item.input_audio_transcription.delta",
|
||||||
|
item_id: "user1",
|
||||||
|
delta: "Hola Vocero",
|
||||||
|
});
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(transcript).toHaveBeenCalledTimes(2);
|
||||||
|
});
|
||||||
|
expect(transcript).toHaveBeenLastCalledWith({ role: "user", text: "Hola Vocero", partial: true, id: "user1" });
|
||||||
|
await client.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("suppresses the next native reply when external reasoning wins before response.created", async () => {
|
||||||
|
const client = new RealtimeClient(options());
|
||||||
|
const transcript = vi.fn();
|
||||||
|
client.on("transcript", transcript);
|
||||||
|
const socket = await connect(client);
|
||||||
|
client.renderExternalAnswer("Respuesta externa");
|
||||||
|
socket.receive({ type: "response.created", response: { id: "native" } });
|
||||||
|
socket.receive({ type: "response.audio.delta", response_id: "native", delta: "AAAA" });
|
||||||
|
socket.receive({ type: "response.audio_transcript.delta", response_id: "native", delta: "Respuesta duplicada" });
|
||||||
|
socket.receive({ type: "response.done", response: { id: "native", status: "completed" } });
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(client.status).toBe("listening");
|
||||||
|
});
|
||||||
|
expect(audio.play).not.toHaveBeenCalled();
|
||||||
|
expect(transcript).not.toHaveBeenCalled();
|
||||||
|
expect(socket.sent.at(-1)).toMatchObject({ item: { role: "assistant", content: [{ text: "Respuesta externa" }] } });
|
||||||
|
await client.close();
|
||||||
|
});
|
||||||
|
it("sends configuration before greeting without an acknowledgment and serializes response requests", async () => {
|
||||||
|
const client = new RealtimeClient({ ...options(), greeting: "hola vocero" });
|
||||||
|
const socket = await connect(client);
|
||||||
|
expect(socket.sent.map((event) => event.type)).toEqual([
|
||||||
|
"session.update",
|
||||||
|
"conversation.item.create",
|
||||||
|
"response.create",
|
||||||
|
]);
|
||||||
|
socket.receive({ type: "response.created", response: { id: "r1" } });
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(client.status).toBe("processing");
|
||||||
|
});
|
||||||
|
const sentCount = socket.sent.length;
|
||||||
|
socket.receive({ type: "session.updated" });
|
||||||
|
socket.receive({ type: "session.updated" });
|
||||||
|
await vi.advanceTimersByTimeAsync(0);
|
||||||
|
expect(client.status).toBe("processing");
|
||||||
|
expect(socket.sent).toHaveLength(sentCount);
|
||||||
|
client.sendUserText("Otra pregunta");
|
||||||
|
client.requestResponse();
|
||||||
|
expect(socket.sent.filter((event) => event.type === "response.create")).toHaveLength(1);
|
||||||
|
socket.receive({ type: "response.done", response: { id: "r1", status: "completed" } });
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(socket.sent.filter((event) => event.type === "response.create")).toHaveLength(2);
|
||||||
|
});
|
||||||
|
expect(socket.sent.at(-2)).toMatchObject({
|
||||||
|
type: "conversation.item.create",
|
||||||
|
item: { role: "user", content: [{ text: "Otra pregunta" }] },
|
||||||
|
});
|
||||||
|
await client.close();
|
||||||
|
expect(audio.close).toHaveBeenCalledOnce();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("assembles streaming transcripts and preserves an interrupted response", async () => {
|
||||||
|
const client = new RealtimeClient(options());
|
||||||
|
const transcripts = vi.fn();
|
||||||
|
const finished = vi.fn();
|
||||||
|
client.on("transcript", transcripts);
|
||||||
|
client.on("finished", finished);
|
||||||
|
const socket = await connect(client);
|
||||||
|
socket.receive({ type: "response.created", response: { id: "reply" } });
|
||||||
|
socket.receive({ type: "response.audio_transcript.delta", response_id: "reply", delta: "Hola " });
|
||||||
|
socket.receive({ type: "response.audio_transcript.delta", response_id: "reply", delta: "mundo" });
|
||||||
|
socket.receive({ type: "response.audio.delta", response_id: "reply", delta: "AAAA" });
|
||||||
|
socket.receive({ type: "input_audio_buffer.speech_started" });
|
||||||
|
socket.receive({ type: "response.done", response: { id: "reply", status: "cancelled" } });
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(finished).toHaveBeenCalledOnce();
|
||||||
|
});
|
||||||
|
expect(transcripts).toHaveBeenLastCalledWith({ role: "assistant", id: "reply", text: "Hola mundo", partial: true });
|
||||||
|
expect(finished).toHaveBeenCalledWith({
|
||||||
|
responseId: "reply",
|
||||||
|
transcript: "Hola mundo",
|
||||||
|
status: "cancelled",
|
||||||
|
audible: true,
|
||||||
|
});
|
||||||
|
expect(audio.clear).toHaveBeenCalled();
|
||||||
|
await client.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not open a socket after cancellation during microphone permission", async () => {
|
||||||
|
let allow: ((value: unknown) => void) | undefined;
|
||||||
|
getUserMedia.mockReturnValue(
|
||||||
|
new Promise((resolve) => {
|
||||||
|
allow = resolve;
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
const client = new RealtimeClient(options());
|
||||||
|
const pending = client.connect();
|
||||||
|
const rejected = expect(pending).rejects.toMatchObject({ code: "aborted" });
|
||||||
|
await client.close();
|
||||||
|
allow?.({ getTracks: () => [{ stop: stopTrack }] });
|
||||||
|
await rejected;
|
||||||
|
expect(stopTrack).toHaveBeenCalledOnce();
|
||||||
|
expect(Socket.instances).toHaveLength(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("releases a queue ticket when leaving before admission", async () => {
|
||||||
|
const fetchMock = vi
|
||||||
|
.fn()
|
||||||
|
.mockResolvedValue(new Response(JSON.stringify({ state: "queued", queue_id: "q1", position: 2 })));
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
const client = new RealtimeClient({ ...options(), directUrl: undefined, sessionUrl: "/api/session" });
|
||||||
|
const pending = client.connect();
|
||||||
|
const rejected = expect(pending).rejects.toBeInstanceOf(SessionError);
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(client.status).toBe("queued");
|
||||||
|
});
|
||||||
|
await client.close();
|
||||||
|
await rejected;
|
||||||
|
expect(sendBeacon).toHaveBeenCalledWith("/api/queue/end", expect.any(Blob));
|
||||||
|
expect(fetchMock).toHaveBeenCalledOnce();
|
||||||
|
expect(Socket.instances).toHaveLength(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("sends each tool call once and returns its result using the realtime protocol", async () => {
|
||||||
|
const client = new RealtimeClient(options());
|
||||||
|
const tool = vi.fn();
|
||||||
|
client.on("tool", tool);
|
||||||
|
const socket = await connect(client);
|
||||||
|
const call = {
|
||||||
|
type: "response.function_call_arguments.done",
|
||||||
|
name: "web_search",
|
||||||
|
arguments: '{"query":"hoy"}',
|
||||||
|
call_id: "call1",
|
||||||
|
};
|
||||||
|
socket.receive(call);
|
||||||
|
socket.receive(call);
|
||||||
|
await vi.waitFor(() => {
|
||||||
|
expect(tool).toHaveBeenCalledOnce();
|
||||||
|
});
|
||||||
|
client.sendToolOutput("call1", "Resultado");
|
||||||
|
expect(socket.sent.at(-2)).toMatchObject({
|
||||||
|
item: { type: "function_call_output", call_id: "call1", output: "Resultado" },
|
||||||
|
});
|
||||||
|
await client.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,569 @@
|
|||||||
|
import { z } from "zod";
|
||||||
|
|
||||||
|
import { encodePcm } from "../audio/codec";
|
||||||
|
import { AudioEngine, MIC_CONSTRAINTS } from "../audio/engine";
|
||||||
|
import { ApiError, cancelQueue, post, request } from "../lib/api";
|
||||||
|
import {
|
||||||
|
grantSchema,
|
||||||
|
realtimeEventSchema,
|
||||||
|
sessionSchema,
|
||||||
|
type ConnectionStatus,
|
||||||
|
type FinishedResponse,
|
||||||
|
type FunctionTool,
|
||||||
|
type NoiseGate,
|
||||||
|
type RealtimeEvent,
|
||||||
|
type SessionGrant,
|
||||||
|
type ToolCall,
|
||||||
|
type Transcript,
|
||||||
|
} from "../lib/contracts";
|
||||||
|
|
||||||
|
interface Events {
|
||||||
|
status: ConnectionStatus;
|
||||||
|
transcript: Transcript;
|
||||||
|
finished: FinishedResponse;
|
||||||
|
tool: ToolCall;
|
||||||
|
level: number;
|
||||||
|
queue: { position: number; queueId: string };
|
||||||
|
grant: SessionGrant;
|
||||||
|
readyToJoin: { grant: SessionGrant; expiresSec: number };
|
||||||
|
error: Error;
|
||||||
|
serverError: Error;
|
||||||
|
}
|
||||||
|
export interface ClientOptions {
|
||||||
|
directUrl?: string;
|
||||||
|
sessionUrl?: string;
|
||||||
|
voice: string;
|
||||||
|
instructions: string;
|
||||||
|
tools: FunctionTool[];
|
||||||
|
gate: NoiseGate;
|
||||||
|
context: AudioContext;
|
||||||
|
greeting?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export class SessionError extends Error {
|
||||||
|
readonly code: "queue-full" | "queue-expired" | "join-expired" | "aborted";
|
||||||
|
constructor(code: SessionError["code"], message: string) {
|
||||||
|
super(message);
|
||||||
|
this.code = code;
|
||||||
|
this.name = "SessionError";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Owns one complete session, including pending acquisition, audio, queue tickets and teardown. */
|
||||||
|
export class RealtimeClient {
|
||||||
|
private readonly options: ClientOptions;
|
||||||
|
private readonly events = new EventTarget();
|
||||||
|
private readonly abort = new AbortController();
|
||||||
|
private readonly audio: AudioEngine;
|
||||||
|
private socket: WebSocket | null = null;
|
||||||
|
private currentStatus: ConnectionStatus = "idle";
|
||||||
|
private configured = false;
|
||||||
|
private closed = false;
|
||||||
|
private closing: Promise<void> | null = null;
|
||||||
|
private queueId = "";
|
||||||
|
private joinResolve: (() => void) | null = null;
|
||||||
|
private joinReject: ((reason: Error) => void) | null = null;
|
||||||
|
private joinTimer: ReturnType<typeof setTimeout> | undefined;
|
||||||
|
private incoming: Promise<void> = Promise.resolve();
|
||||||
|
private activeResponses = new Set<string>();
|
||||||
|
private activeResponse = "";
|
||||||
|
private createInFlight = false;
|
||||||
|
private pendingCreates = 0;
|
||||||
|
private deferredTexts: string[] = [];
|
||||||
|
private suppressed = new Set<string>();
|
||||||
|
private suppressNext = false;
|
||||||
|
private audible = new Set<string>();
|
||||||
|
private segments = new Map<string, string>();
|
||||||
|
private completed = new Map<string, string>();
|
||||||
|
private userSegments = new Map<string, string>();
|
||||||
|
private toolCalls = new Set<string>();
|
||||||
|
private resolveReady: (() => void) | null = null;
|
||||||
|
private rejectReady: ((reason: Error) => void) | null = null;
|
||||||
|
private readyTimer: ReturnType<typeof setTimeout> | undefined;
|
||||||
|
|
||||||
|
constructor(options: ClientOptions) {
|
||||||
|
this.options = options;
|
||||||
|
this.audio = new AudioEngine(options.context, options.gate);
|
||||||
|
}
|
||||||
|
get status() {
|
||||||
|
return this.currentStatus;
|
||||||
|
}
|
||||||
|
on<K extends keyof Events>(name: K, listener: (detail: Events[K]) => void) {
|
||||||
|
const handler = (event: Event) => {
|
||||||
|
listener((event as CustomEvent<Events[K]>).detail);
|
||||||
|
};
|
||||||
|
this.events.addEventListener(name, handler);
|
||||||
|
return () => {
|
||||||
|
this.events.removeEventListener(name, handler);
|
||||||
|
};
|
||||||
|
}
|
||||||
|
private emit<K extends keyof Events>(name: K, detail: Events[K]) {
|
||||||
|
if (!this.closed || name === "status") {
|
||||||
|
this.events.dispatchEvent(new CustomEvent(name, { detail }));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
private setStatus(status: ConnectionStatus) {
|
||||||
|
if (this.currentStatus !== status) {
|
||||||
|
this.currentStatus = status;
|
||||||
|
this.emit("status", status);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
private ensureOpen() {
|
||||||
|
if (this.closed) {
|
||||||
|
throw new SessionError("aborted", "La conexión se canceló.");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async connect() {
|
||||||
|
this.setStatus("connecting");
|
||||||
|
try {
|
||||||
|
// Ask for permission while the click-created AudioContext is still resumed.
|
||||||
|
const permission = await navigator.mediaDevices.getUserMedia(MIC_CONSTRAINTS);
|
||||||
|
permission.getTracks().forEach((track) => {
|
||||||
|
track.stop();
|
||||||
|
});
|
||||||
|
this.ensureOpen();
|
||||||
|
let url = this.options.directUrl;
|
||||||
|
if (!url) {
|
||||||
|
const grant = await this.acquireSession();
|
||||||
|
this.ensureOpen();
|
||||||
|
url = grant.connect_url;
|
||||||
|
}
|
||||||
|
this.ensureOpen();
|
||||||
|
this.socket = new WebSocket(url);
|
||||||
|
this.socket.binaryType = "arraybuffer";
|
||||||
|
const ready = new Promise<void>((resolve, reject) => {
|
||||||
|
this.resolveReady = resolve;
|
||||||
|
this.rejectReady = reject;
|
||||||
|
this.readyTimer = setTimeout(() => {
|
||||||
|
reject(new Error("El servidor de voz no respondió a tiempo."));
|
||||||
|
}, 20000);
|
||||||
|
});
|
||||||
|
const audioReady = this.audio.start(
|
||||||
|
(buffer) => {
|
||||||
|
if (this.configured) {
|
||||||
|
this.send({ type: "input_audio_buffer.append", audio: encodePcm(buffer) });
|
||||||
|
}
|
||||||
|
},
|
||||||
|
(rms) => {
|
||||||
|
this.emit("level", rms);
|
||||||
|
},
|
||||||
|
() => this.currentStatus === "ai-speaking",
|
||||||
|
);
|
||||||
|
this.socket.addEventListener("message", (event: MessageEvent<unknown>) => {
|
||||||
|
this.incoming = this.incoming
|
||||||
|
.then(async () => {
|
||||||
|
await audioReady;
|
||||||
|
if (!this.closed) {
|
||||||
|
await this.receive(event.data);
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.catch((error: unknown) => {
|
||||||
|
if (!this.closed) {
|
||||||
|
this.fail(error);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
this.socket.addEventListener("error", () => {
|
||||||
|
this.fail(new Error("No se pudo conectar con el servidor de voz."));
|
||||||
|
});
|
||||||
|
this.socket.addEventListener("close", (event) => {
|
||||||
|
if (this.closed) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (event.code === 1000 && this.configured) {
|
||||||
|
void this.close();
|
||||||
|
} else {
|
||||||
|
this.fail(new Error(`Se cerró la conexión de voz (${event.code}). ${event.reason}`.trim()));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
await Promise.all([audioReady, ready]);
|
||||||
|
this.ensureOpen();
|
||||||
|
} catch (error) {
|
||||||
|
const failure = this.closed ? new SessionError("aborted", "La conexión se canceló.") : error;
|
||||||
|
await this.close();
|
||||||
|
throw failure;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async acquireSession() {
|
||||||
|
let result: z.infer<typeof sessionSchema>;
|
||||||
|
try {
|
||||||
|
result = await post(this.options.sessionUrl ?? "/api/session", sessionSchema, {}, this.abort.signal);
|
||||||
|
} catch (error) {
|
||||||
|
throw this.sessionError(error);
|
||||||
|
}
|
||||||
|
let waited = false;
|
||||||
|
if ("state" in result) {
|
||||||
|
waited = true;
|
||||||
|
const ticket = result;
|
||||||
|
this.queueId = ticket.queue_id;
|
||||||
|
this.setStatus("queued");
|
||||||
|
this.emit("queue", { position: ticket.position, queueId: this.queueId });
|
||||||
|
while ("state" in result) {
|
||||||
|
await this.delay(Math.max(1, ticket.poll_interval_s) * 1000);
|
||||||
|
this.ensureOpen();
|
||||||
|
try {
|
||||||
|
result = await request(`/api/queue/${encodeURIComponent(this.queueId)}`, sessionSchema, {
|
||||||
|
signal: this.abort.signal,
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
this.ensureOpen();
|
||||||
|
if (error instanceof ApiError && error.status === 404) {
|
||||||
|
throw this.sessionError(error);
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if ("state" in result) {
|
||||||
|
this.emit("queue", { position: result.position, queueId: this.queueId });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const grant = grantSchema.parse(result);
|
||||||
|
this.queueId = "";
|
||||||
|
this.ensureOpen();
|
||||||
|
if (waited) {
|
||||||
|
const seconds = Math.max(3, grant.pending_timeout_s - 3);
|
||||||
|
this.setStatus("your-turn");
|
||||||
|
const joined = new Promise<void>((resolve, reject) => {
|
||||||
|
this.joinResolve = resolve;
|
||||||
|
this.joinReject = reject;
|
||||||
|
this.joinTimer = setTimeout(() => {
|
||||||
|
reject(new SessionError("join-expired", "Tu turno ha caducado. Vuelve a intentarlo."));
|
||||||
|
}, seconds * 1000);
|
||||||
|
});
|
||||||
|
this.emit("readyToJoin", { grant, expiresSec: seconds });
|
||||||
|
await joined;
|
||||||
|
}
|
||||||
|
this.emit("grant", grant);
|
||||||
|
this.setStatus("connecting");
|
||||||
|
return grant;
|
||||||
|
}
|
||||||
|
private sessionError(error: unknown): unknown {
|
||||||
|
if (error instanceof ApiError) {
|
||||||
|
if (error.status === 404) {
|
||||||
|
return new SessionError("queue-expired", "La espera ha caducado. Vuelve a intentarlo.");
|
||||||
|
}
|
||||||
|
const data = z.object({ state: z.string() }).safeParse(error.body);
|
||||||
|
if (error.status === 503 && data.success && data.data.state === "at_capacity") {
|
||||||
|
return new SessionError("queue-full", "Todos los lugares y la cola están ocupados. Vuelve en un minuto.");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return error;
|
||||||
|
}
|
||||||
|
private delay(ms: number) {
|
||||||
|
return new Promise<void>((resolve) => {
|
||||||
|
const finish = () => {
|
||||||
|
clearTimeout(timer);
|
||||||
|
this.abort.signal.removeEventListener("abort", finish);
|
||||||
|
resolve();
|
||||||
|
};
|
||||||
|
const timer = setTimeout(finish, ms);
|
||||||
|
this.abort.signal.addEventListener("abort", finish, { once: true });
|
||||||
|
if (this.abort.signal.aborted) {
|
||||||
|
finish();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
join() {
|
||||||
|
clearTimeout(this.joinTimer);
|
||||||
|
void this.options.context.resume().catch(() => undefined);
|
||||||
|
this.joinResolve?.();
|
||||||
|
this.joinResolve = null;
|
||||||
|
this.joinReject = null;
|
||||||
|
}
|
||||||
|
private fail(error: unknown) {
|
||||||
|
if (this.closed) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const failure = error instanceof Error ? error : new Error("La sesión de voz falló.");
|
||||||
|
this.rejectReady?.(failure);
|
||||||
|
this.emit("error", failure);
|
||||||
|
this.setStatus("error");
|
||||||
|
void this.close();
|
||||||
|
}
|
||||||
|
private async receive(raw: unknown) {
|
||||||
|
const text =
|
||||||
|
typeof raw === "string"
|
||||||
|
? raw
|
||||||
|
: raw instanceof ArrayBuffer
|
||||||
|
? new TextDecoder().decode(raw)
|
||||||
|
: raw instanceof Blob
|
||||||
|
? await raw.text()
|
||||||
|
: "";
|
||||||
|
let parsed: unknown;
|
||||||
|
try {
|
||||||
|
parsed = JSON.parse(text);
|
||||||
|
} catch {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const result = realtimeEventSchema.safeParse(parsed);
|
||||||
|
if (!result.success) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.handle(result.data);
|
||||||
|
}
|
||||||
|
private assistantText(id: string) {
|
||||||
|
return [this.completed.get(id), this.segments.get(id)].filter(Boolean).join(" ");
|
||||||
|
}
|
||||||
|
private handle(event: RealtimeEvent) {
|
||||||
|
const id = event.response_id ?? event.response?.id ?? this.activeResponse;
|
||||||
|
switch (event.type) {
|
||||||
|
case "session.created":
|
||||||
|
this.send({
|
||||||
|
type: "session.update",
|
||||||
|
session: {
|
||||||
|
type: "realtime",
|
||||||
|
instructions: this.options.instructions,
|
||||||
|
audio: { output: { voice: this.options.voice } },
|
||||||
|
tools: this.options.tools,
|
||||||
|
tool_choice: this.options.tools.length ? "auto" : "none",
|
||||||
|
},
|
||||||
|
});
|
||||||
|
// The speech server applies frames in order without acknowledging session.update.
|
||||||
|
// Send configuration before microphone audio or the greeting, then consider it ready.
|
||||||
|
if (!this.configured) {
|
||||||
|
this.configured = true;
|
||||||
|
clearTimeout(this.readyTimer);
|
||||||
|
this.resolveReady?.();
|
||||||
|
this.resolveReady = null;
|
||||||
|
this.rejectReady = null;
|
||||||
|
this.setStatus("listening");
|
||||||
|
if (this.options.greeting) {
|
||||||
|
this.sendUserText(this.options.greeting);
|
||||||
|
this.requestResponse();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
case "input_audio_buffer.speech_started":
|
||||||
|
this.audio.clear();
|
||||||
|
this.setStatus("user-speaking");
|
||||||
|
break;
|
||||||
|
case "input_audio_buffer.speech_stopped":
|
||||||
|
this.setStatus("processing");
|
||||||
|
break;
|
||||||
|
case "response.created":
|
||||||
|
this.activeResponse = id;
|
||||||
|
this.activeResponses.add(id);
|
||||||
|
this.createInFlight = false;
|
||||||
|
if (this.suppressNext) {
|
||||||
|
this.suppressed.add(id);
|
||||||
|
this.suppressNext = false;
|
||||||
|
}
|
||||||
|
this.setStatus("processing");
|
||||||
|
break;
|
||||||
|
case "response.audio.delta":
|
||||||
|
case "response.output_audio.delta":
|
||||||
|
if (event.delta && !this.suppressed.has(id)) {
|
||||||
|
this.audio.play(event.delta);
|
||||||
|
this.audible.add(id);
|
||||||
|
this.setStatus("ai-speaking");
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
case "conversation.item.input_audio_transcription.delta": {
|
||||||
|
const itemId = event.item_id ?? "user-current";
|
||||||
|
// This server sends the complete speculative transcription in each delta.
|
||||||
|
const text = event.delta ?? "";
|
||||||
|
this.userSegments.set(itemId, text);
|
||||||
|
this.emit("transcript", { role: "user", text, partial: true, id: itemId });
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
case "conversation.item.input_audio_transcription.completed": {
|
||||||
|
const itemId = event.item_id ?? "user-current";
|
||||||
|
this.emit("transcript", {
|
||||||
|
role: "user",
|
||||||
|
text: event.transcript ?? this.userSegments.get(itemId) ?? "",
|
||||||
|
partial: false,
|
||||||
|
id: itemId,
|
||||||
|
});
|
||||||
|
this.userSegments.delete(itemId);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
case "response.audio_transcript.delta":
|
||||||
|
case "response.output_audio_transcript.delta":
|
||||||
|
case "response.text.delta":
|
||||||
|
case "response.output_text.delta":
|
||||||
|
if (!this.suppressed.has(id)) {
|
||||||
|
this.segments.set(id, (this.segments.get(id) ?? "") + (event.delta ?? ""));
|
||||||
|
this.emit("transcript", { role: "assistant", text: this.assistantText(id), partial: true, id });
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
case "response.audio_transcript.done":
|
||||||
|
case "response.output_audio_transcript.done":
|
||||||
|
case "response.text.done":
|
||||||
|
case "response.output_text.done":
|
||||||
|
if (!this.suppressed.has(id)) {
|
||||||
|
const segment = event.transcript ?? event.text ?? this.segments.get(id) ?? "";
|
||||||
|
this.completed.set(id, [this.completed.get(id), segment].filter(Boolean).join(" "));
|
||||||
|
this.segments.delete(id);
|
||||||
|
this.emit("transcript", { role: "assistant", text: this.assistantText(id), partial: false, id });
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
case "response.done": {
|
||||||
|
const transcript =
|
||||||
|
event.response?.output
|
||||||
|
?.flatMap((item) => item.content ?? [])
|
||||||
|
.map((part) => part.transcript ?? part.text ?? "")
|
||||||
|
.filter(Boolean)
|
||||||
|
.join(" ") || this.assistantText(id);
|
||||||
|
if (!this.suppressed.has(id)) {
|
||||||
|
this.emit("finished", {
|
||||||
|
responseId: id,
|
||||||
|
status: event.response?.status ?? "completed",
|
||||||
|
transcript,
|
||||||
|
audible: this.audible.has(id),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
this.activeResponses.delete(id);
|
||||||
|
this.audible.delete(id);
|
||||||
|
this.suppressed.delete(id);
|
||||||
|
this.segments.delete(id);
|
||||||
|
this.completed.delete(id);
|
||||||
|
if (this.activeResponse === id) {
|
||||||
|
this.activeResponse = "";
|
||||||
|
}
|
||||||
|
if (this.currentStatus === "ai-speaking" || this.currentStatus === "processing") {
|
||||||
|
this.setStatus("listening");
|
||||||
|
}
|
||||||
|
this.flush();
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
case "response.function_call_arguments.done":
|
||||||
|
if (event.name && event.call_id && !this.toolCalls.has(event.call_id)) {
|
||||||
|
this.toolCalls.add(event.call_id);
|
||||||
|
this.emit("tool", { name: event.name, arguments: event.arguments ?? "{}", callId: event.call_id });
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
case "error":
|
||||||
|
if (
|
||||||
|
event.error?.code === "conversation_already_has_active_response" ||
|
||||||
|
event.error?.type === "conversation_already_has_active_response"
|
||||||
|
) {
|
||||||
|
if (this.createInFlight) {
|
||||||
|
this.createInFlight = false;
|
||||||
|
this.pendingCreates++;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
this.emit("serverError", new Error(event.error?.message ?? "Vocero no pudo completar la respuesta."));
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
private busy() {
|
||||||
|
return this.activeResponses.size > 0 || this.createInFlight;
|
||||||
|
}
|
||||||
|
private send(event: Record<string, unknown>) {
|
||||||
|
if (!this.closed && this.socket?.readyState === WebSocket.OPEN) {
|
||||||
|
this.socket.send(JSON.stringify(event));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
updateSession(instructions: string, voice: string, tools: FunctionTool[]) {
|
||||||
|
this.options.instructions = instructions;
|
||||||
|
this.options.voice = voice;
|
||||||
|
this.options.tools = tools;
|
||||||
|
if (this.configured) {
|
||||||
|
this.send({
|
||||||
|
type: "session.update",
|
||||||
|
session: {
|
||||||
|
type: "realtime",
|
||||||
|
instructions,
|
||||||
|
audio: { output: { voice } },
|
||||||
|
tools,
|
||||||
|
tool_choice: tools.length ? "auto" : "none",
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sendUserText(text: string) {
|
||||||
|
if (!text.trim()) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (this.busy()) {
|
||||||
|
this.deferredTexts.push(text.trim());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.send({
|
||||||
|
type: "conversation.item.create",
|
||||||
|
item: { type: "message", role: "user", content: [{ type: "input_text", text: text.trim() }] },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
requestResponse() {
|
||||||
|
if (this.closed || !this.configured) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (this.busy()) {
|
||||||
|
this.pendingCreates++;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.createInFlight = true;
|
||||||
|
this.send({ type: "response.create" });
|
||||||
|
}
|
||||||
|
private flush() {
|
||||||
|
if (this.busy()) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const text = this.deferredTexts.shift();
|
||||||
|
if (text) {
|
||||||
|
this.sendUserText(text);
|
||||||
|
}
|
||||||
|
if (this.pendingCreates) {
|
||||||
|
this.pendingCreates--;
|
||||||
|
this.requestResponse();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sendToolOutput(callId: string, output: string) {
|
||||||
|
this.send({ type: "conversation.item.create", item: { type: "function_call_output", call_id: callId, output } });
|
||||||
|
this.requestResponse();
|
||||||
|
}
|
||||||
|
renderExternalAnswer(text: string) {
|
||||||
|
if (this.activeResponse) {
|
||||||
|
this.suppressed.add(this.activeResponse);
|
||||||
|
} else {
|
||||||
|
this.suppressNext = true;
|
||||||
|
}
|
||||||
|
this.audio.clear();
|
||||||
|
this.pendingCreates = 0;
|
||||||
|
this.deferredTexts = [];
|
||||||
|
this.send({
|
||||||
|
type: "conversation.item.create",
|
||||||
|
item: { type: "message", role: "assistant", content: [{ type: "text", text }] },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
setMuted(muted: boolean) {
|
||||||
|
this.audio.setMuted(muted);
|
||||||
|
}
|
||||||
|
setGate(gate: NoiseGate) {
|
||||||
|
this.audio.setGate(gate);
|
||||||
|
}
|
||||||
|
close(): Promise<void> {
|
||||||
|
if (this.closing) {
|
||||||
|
return this.closing;
|
||||||
|
}
|
||||||
|
this.closed = true;
|
||||||
|
this.abort.abort();
|
||||||
|
clearTimeout(this.joinTimer);
|
||||||
|
clearTimeout(this.readyTimer);
|
||||||
|
const cancelled = new SessionError("aborted", "La conexión se canceló.");
|
||||||
|
this.joinReject?.(cancelled);
|
||||||
|
this.rejectReady?.(cancelled);
|
||||||
|
this.joinResolve = null;
|
||||||
|
this.joinReject = null;
|
||||||
|
this.resolveReady = null;
|
||||||
|
this.rejectReady = null;
|
||||||
|
if (this.queueId) {
|
||||||
|
cancelQueue(this.queueId);
|
||||||
|
this.queueId = "";
|
||||||
|
}
|
||||||
|
if (this.socket && this.socket.readyState <= WebSocket.OPEN) {
|
||||||
|
this.socket.close(1000, "client closed");
|
||||||
|
}
|
||||||
|
this.socket = null;
|
||||||
|
this.closing = this.audio
|
||||||
|
.close()
|
||||||
|
.catch(() => undefined)
|
||||||
|
.then(() => {
|
||||||
|
this.setStatus("closed");
|
||||||
|
});
|
||||||
|
return this.closing;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,139 @@
|
|||||||
|
import { afterEach, beforeEach, describe, expect, it, vi, type Mock } from "vitest";
|
||||||
|
|
||||||
|
import type { ClientOptions } from "./client";
|
||||||
|
import { VoiceController } from "./controller";
|
||||||
|
|
||||||
|
const { MockClient, closeContext } = vi.hoisted(() => {
|
||||||
|
const closeContext = vi.fn().mockResolvedValue(undefined);
|
||||||
|
class MockClient {
|
||||||
|
static instances: MockClient[] = [];
|
||||||
|
readonly options: ClientOptions;
|
||||||
|
private listeners = new Map<string, Set<(value: unknown) => void>>();
|
||||||
|
constructor(options: ClientOptions) {
|
||||||
|
this.options = options;
|
||||||
|
MockClient.instances.push(this);
|
||||||
|
}
|
||||||
|
on(name: string, listener: (value: unknown) => void) {
|
||||||
|
const listeners = this.listeners.get(name) ?? new Set();
|
||||||
|
listeners.add(listener);
|
||||||
|
this.listeners.set(name, listeners);
|
||||||
|
return () => {
|
||||||
|
listeners.delete(listener);
|
||||||
|
};
|
||||||
|
}
|
||||||
|
emit(name: string, value: unknown) {
|
||||||
|
this.listeners.get(name)?.forEach((listener) => {
|
||||||
|
listener(value);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
connect = vi.fn(() => {
|
||||||
|
this.emit("status", "listening");
|
||||||
|
return Promise.resolve();
|
||||||
|
});
|
||||||
|
close = vi.fn(() => {
|
||||||
|
this.emit("status", "closed");
|
||||||
|
return Promise.resolve();
|
||||||
|
});
|
||||||
|
updateSession = vi.fn();
|
||||||
|
setGate = vi.fn();
|
||||||
|
setMuted = vi.fn();
|
||||||
|
renderExternalAnswer = vi.fn();
|
||||||
|
sendToolOutput = vi.fn();
|
||||||
|
join = vi.fn();
|
||||||
|
}
|
||||||
|
return { MockClient, closeContext };
|
||||||
|
});
|
||||||
|
vi.mock("./client", async (importOriginal) => ({ ...(await importOriginal()), RealtimeClient: MockClient }));
|
||||||
|
vi.mock("../audio/engine", () => ({ createAudioContext: () => ({ state: "running", close: closeContext }) }));
|
||||||
|
let pendingReasoning: ((value: Response) => void) | undefined;
|
||||||
|
let controller: VoiceController;
|
||||||
|
let fetchMock: Mock<(path: string, init?: RequestInit) => Promise<Response>>;
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.useFakeTimers();
|
||||||
|
vi.clearAllMocks();
|
||||||
|
MockClient.instances = [];
|
||||||
|
pendingReasoning = undefined;
|
||||||
|
vi.stubGlobal("AudioContext", vi.fn());
|
||||||
|
vi.stubGlobal("navigator", { mediaDevices: { getUserMedia: vi.fn() } });
|
||||||
|
fetchMock = vi.fn((path: string) => {
|
||||||
|
if (path === "/api/reasoning") {
|
||||||
|
return new Promise<Response>((resolve) => {
|
||||||
|
pendingReasoning = resolve;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
const pathname = new URL(path, "http://localhost").pathname;
|
||||||
|
const responses: Record<string, unknown> = {
|
||||||
|
"/api/config": { reasoningType: "NODE_RED", injectProfile: false, s2sUrl: "ws://localhost/v1/realtime" },
|
||||||
|
"/api/user/phone": { result: { name: "Ana" } },
|
||||||
|
"/api/user/reports": { count: 0 },
|
||||||
|
};
|
||||||
|
return Promise.resolve(new Response(JSON.stringify(responses[pathname] ?? {})));
|
||||||
|
});
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
controller = new VoiceController({ phoneNumber: "1234", episodeId: "episode1" });
|
||||||
|
});
|
||||||
|
afterEach(async () => {
|
||||||
|
await controller.stop();
|
||||||
|
vi.useRealTimers();
|
||||||
|
vi.unstubAllGlobals();
|
||||||
|
});
|
||||||
|
async function start() {
|
||||||
|
await controller.initialize(new AbortController().signal);
|
||||||
|
await controller.start();
|
||||||
|
const client = MockClient.instances[0];
|
||||||
|
if (!client) {
|
||||||
|
throw new Error("No client created");
|
||||||
|
}
|
||||||
|
return client;
|
||||||
|
}
|
||||||
|
describe("voice application lifecycle", () => {
|
||||||
|
it("loads configuration and caller context without an account request on startup or stop", async () => {
|
||||||
|
const client = await start();
|
||||||
|
await controller.stop();
|
||||||
|
expect(client.close).toHaveBeenCalledOnce();
|
||||||
|
expect(fetchMock.mock.calls.map(([path]) => new URL(path, "http://localhost").pathname)).toEqual([
|
||||||
|
"/api/config",
|
||||||
|
"/api/user/phone",
|
||||||
|
"/api/user/reports",
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps LOG_ONLY reasoning on the native voice path and preserves separate profile context", async () => {
|
||||||
|
const client = await start();
|
||||||
|
client.emit("transcript", { id: "u1", role: "user", text: "Hola", partial: false });
|
||||||
|
pendingReasoning?.(new Response(JSON.stringify({ log_only: true, answer: "" })));
|
||||||
|
await vi.advanceTimersByTimeAsync(1);
|
||||||
|
const request = fetchMock.mock.calls.find(([path]) => path === "/api/reasoning");
|
||||||
|
expect(request?.[1]?.body).toContain('"phoneNumber":"1234"');
|
||||||
|
expect(request?.[1]?.body).toContain("Caller profile resolved");
|
||||||
|
expect(client.options.instructions).not.toContain("Caller profile resolved");
|
||||||
|
expect(client.renderExternalAnswer).not.toHaveBeenCalled();
|
||||||
|
expect(controller.getSnapshot().messages).toHaveLength(1);
|
||||||
|
});
|
||||||
|
it("discards external reasoning that finishes after the session was stopped", async () => {
|
||||||
|
const client = await start();
|
||||||
|
client.emit("transcript", { id: "u1", role: "user", text: "Consulta", partial: false });
|
||||||
|
await controller.stop();
|
||||||
|
pendingReasoning?.(new Response(JSON.stringify({ answer: "Respuesta tardía" })));
|
||||||
|
await vi.advanceTimersByTimeAsync(1);
|
||||||
|
expect(client.renderExternalAnswer).not.toHaveBeenCalled();
|
||||||
|
expect(controller.getSnapshot().messages.map((message) => message.text)).toEqual(["Consulta"]);
|
||||||
|
});
|
||||||
|
it("applies and removes RAG context in the live session", async () => {
|
||||||
|
const client = await start();
|
||||||
|
controller.setRag("Datos importantes");
|
||||||
|
expect(client.updateSession).toHaveBeenLastCalledWith(expect.stringContaining("Datos importantes"), "Aiden", []);
|
||||||
|
controller.setRag("");
|
||||||
|
expect(client.updateSession).toHaveBeenLastCalledWith(expect.not.stringContaining("CUSTOM_RAG_BEGIN"), "Aiden", []);
|
||||||
|
});
|
||||||
|
it("closes after silence but does not cut off the assistant while speaking", async () => {
|
||||||
|
const client = await start();
|
||||||
|
client.emit("status", "ai-speaking");
|
||||||
|
await vi.advanceTimersByTimeAsync(60000);
|
||||||
|
expect(client.close).not.toHaveBeenCalled();
|
||||||
|
client.emit("status", "listening");
|
||||||
|
await vi.advanceTimersByTimeAsync(30000);
|
||||||
|
expect(client.close).toHaveBeenCalledOnce();
|
||||||
|
expect(controller.getSnapshot().status).toBe("closed");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,507 @@
|
|||||||
|
import { z } from "zod";
|
||||||
|
|
||||||
|
import { createAudioContext } from "../audio/engine";
|
||||||
|
import {
|
||||||
|
errorMessage,
|
||||||
|
getConfig,
|
||||||
|
getProfile,
|
||||||
|
getReports,
|
||||||
|
getTools,
|
||||||
|
post,
|
||||||
|
reasoningSchema,
|
||||||
|
searchSchema,
|
||||||
|
} from "../lib/api";
|
||||||
|
import { buildInstructions, profileContext, ragContext, speechInstructions, WEB_SEARCH } from "../lib/context";
|
||||||
|
import {
|
||||||
|
defaultConfig,
|
||||||
|
type ConnectionStatus,
|
||||||
|
type DynamicTool,
|
||||||
|
type Profile,
|
||||||
|
type Reports,
|
||||||
|
type ServerConfig,
|
||||||
|
type ToolCall,
|
||||||
|
type Transcript,
|
||||||
|
} from "../lib/contracts";
|
||||||
|
import type { Message } from "../lib/messages";
|
||||||
|
import {
|
||||||
|
callerFromLocation,
|
||||||
|
connectionTarget,
|
||||||
|
GATE_OFF,
|
||||||
|
loadSettings,
|
||||||
|
saveSettings,
|
||||||
|
type Caller,
|
||||||
|
type Settings,
|
||||||
|
} from "../lib/settings";
|
||||||
|
import { RealtimeClient, SessionError } from "./client";
|
||||||
|
|
||||||
|
export const LIVE_STATES = new Set<ConnectionStatus>(["listening", "user-speaking", "processing", "ai-speaking"]);
|
||||||
|
export const ACTIVE_STATES = new Set<ConnectionStatus>([...LIVE_STATES, "connecting", "queued", "your-turn"]);
|
||||||
|
export interface AppState {
|
||||||
|
loading: boolean;
|
||||||
|
config: ServerConfig;
|
||||||
|
settings: Settings;
|
||||||
|
profile: Profile | null;
|
||||||
|
reports: Reports | null;
|
||||||
|
tools: DynamicTool[];
|
||||||
|
toolsLoading: boolean;
|
||||||
|
toolsError: string;
|
||||||
|
status: ConnectionStatus;
|
||||||
|
error: string;
|
||||||
|
notice: string;
|
||||||
|
messages: Message[];
|
||||||
|
muted: boolean;
|
||||||
|
level: number;
|
||||||
|
rag: string;
|
||||||
|
queuePosition: number;
|
||||||
|
joinDeadline: number;
|
||||||
|
unlock: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Observable application state; React owns rendering, this object owns session lifetime. */
|
||||||
|
export class VoiceController {
|
||||||
|
readonly caller: Caller;
|
||||||
|
private state: AppState;
|
||||||
|
private readonly listeners = new Set<() => void>();
|
||||||
|
private client: RealtimeClient | null = null;
|
||||||
|
private subscriptions: (() => void)[] = [];
|
||||||
|
private requests = new AbortController();
|
||||||
|
private idleTimer: ReturnType<typeof setInterval> | undefined;
|
||||||
|
private lastActivity = Date.now();
|
||||||
|
private lastMeter = 0;
|
||||||
|
private sessionNumber = 0;
|
||||||
|
private reasoned = new Set<string>();
|
||||||
|
private autoStarted = false;
|
||||||
|
|
||||||
|
constructor(caller = callerFromLocation()) {
|
||||||
|
this.caller = caller;
|
||||||
|
this.state = {
|
||||||
|
loading: true,
|
||||||
|
config: defaultConfig,
|
||||||
|
settings: loadSettings(),
|
||||||
|
profile: null,
|
||||||
|
reports: null,
|
||||||
|
tools: [],
|
||||||
|
toolsLoading: false,
|
||||||
|
toolsError: "",
|
||||||
|
status: "idle",
|
||||||
|
error: "",
|
||||||
|
notice: "",
|
||||||
|
messages: [],
|
||||||
|
muted: false,
|
||||||
|
level: 0,
|
||||||
|
rag: "",
|
||||||
|
queuePosition: 0,
|
||||||
|
joinDeadline: 0,
|
||||||
|
unlock: false,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
getSnapshot = () => this.state;
|
||||||
|
subscribe = (listener: () => void) => {
|
||||||
|
this.listeners.add(listener);
|
||||||
|
return () => {
|
||||||
|
this.listeners.delete(listener);
|
||||||
|
};
|
||||||
|
};
|
||||||
|
private update(patch: Partial<AppState>) {
|
||||||
|
this.state = { ...this.state, ...patch };
|
||||||
|
this.listeners.forEach((listener) => {
|
||||||
|
listener();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async initialize(signal: AbortSignal) {
|
||||||
|
try {
|
||||||
|
const [config, profile, reports] = await Promise.all([
|
||||||
|
getConfig(signal),
|
||||||
|
getProfile(this.caller.phoneNumber, signal),
|
||||||
|
getReports(this.caller.phoneNumber, signal),
|
||||||
|
]);
|
||||||
|
if (signal.aborted) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.update({ config, profile, reports, loading: false, error: "" });
|
||||||
|
if (config.apipilot) {
|
||||||
|
await this.refreshTools(signal, !!profile);
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
if (!signal.aborted) {
|
||||||
|
this.update({ loading: false, error: `No se pudo cargar la configuración. ${errorMessage(error)}` });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async refreshTools(signal: AbortSignal, refresh = false) {
|
||||||
|
this.update({ toolsLoading: true, toolsError: "" });
|
||||||
|
try {
|
||||||
|
const tools = await getTools(signal, refresh);
|
||||||
|
if (!signal.aborted) {
|
||||||
|
this.update({ tools, toolsLoading: false });
|
||||||
|
this.pushSettings();
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
if (!signal.aborted) {
|
||||||
|
this.update({ toolsError: errorMessage(error), toolsLoading: false });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
private instructions() {
|
||||||
|
return buildInstructions(
|
||||||
|
this.state.config,
|
||||||
|
this.state.settings,
|
||||||
|
this.state.profile,
|
||||||
|
this.state.rag,
|
||||||
|
this.state.tools,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
private tools() {
|
||||||
|
return this.state.settings.webSearch && (this.state.config.search || this.state.settings.searchKey)
|
||||||
|
? [WEB_SEARCH]
|
||||||
|
: [];
|
||||||
|
}
|
||||||
|
private gate() {
|
||||||
|
return { enabled: this.state.settings.noiseGate > GATE_OFF, thresholdDb: this.state.settings.noiseGate };
|
||||||
|
}
|
||||||
|
private pushSettings() {
|
||||||
|
this.client?.updateSession(
|
||||||
|
speechInstructions(this.instructions(), this.state.config),
|
||||||
|
this.state.settings.voice,
|
||||||
|
this.tools(),
|
||||||
|
);
|
||||||
|
this.client?.setGate(this.gate());
|
||||||
|
}
|
||||||
|
changeSettings(patch: Partial<Settings>) {
|
||||||
|
const settings = { ...this.state.settings, ...patch };
|
||||||
|
this.update({ settings });
|
||||||
|
saveSettings(settings);
|
||||||
|
this.pushSettings();
|
||||||
|
}
|
||||||
|
setRag(rag: string) {
|
||||||
|
this.update({ rag: rag.trim() });
|
||||||
|
this.pushSettings();
|
||||||
|
}
|
||||||
|
dismissError() {
|
||||||
|
this.update({ error: "" });
|
||||||
|
}
|
||||||
|
offerAutoStart() {
|
||||||
|
if (this.autoStarted || this.client || this.state.loading) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.autoStarted = true;
|
||||||
|
const mode = this.state.config.startSessionMode.trim().toUpperCase();
|
||||||
|
if (mode === "TALK") {
|
||||||
|
this.update({ unlock: true });
|
||||||
|
} else if (mode === "TALKFIRST") {
|
||||||
|
void this.start(true);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async start(automatic = false, preparedContext?: AudioContext, greeting?: string) {
|
||||||
|
if (this.client || this.state.loading) {
|
||||||
|
if (preparedContext) {
|
||||||
|
void preparedContext.close().catch(() => undefined);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.update({
|
||||||
|
unlock: false,
|
||||||
|
error: "",
|
||||||
|
notice: "",
|
||||||
|
status: "connecting",
|
||||||
|
muted: false,
|
||||||
|
joinDeadline: 0,
|
||||||
|
});
|
||||||
|
let context: AudioContext | undefined;
|
||||||
|
try {
|
||||||
|
if (!navigator.mediaDevices?.getUserMedia || typeof AudioContext === "undefined") {
|
||||||
|
throw new Error("El micrófono necesita HTTPS o localhost y un navegador compatible.");
|
||||||
|
}
|
||||||
|
const target = connectionTarget(this.state.config, this.state.settings, this.caller);
|
||||||
|
context = preparedContext ?? createAudioContext();
|
||||||
|
if (automatic && context.state !== "running") {
|
||||||
|
await context.close();
|
||||||
|
this.update({ status: "idle", unlock: true });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.requests = new AbortController();
|
||||||
|
this.reasoned.clear();
|
||||||
|
this.sessionNumber++;
|
||||||
|
const client = new RealtimeClient({
|
||||||
|
...target,
|
||||||
|
context,
|
||||||
|
voice: this.state.settings.voice,
|
||||||
|
instructions: speechInstructions(this.instructions(), this.state.config),
|
||||||
|
tools: this.tools(),
|
||||||
|
gate: this.gate(),
|
||||||
|
greeting:
|
||||||
|
greeting ??
|
||||||
|
(["TALK", "TALKFIRST"].includes(this.state.config.startSessionMode.toUpperCase())
|
||||||
|
? "hola vocero"
|
||||||
|
: undefined),
|
||||||
|
});
|
||||||
|
this.client = client;
|
||||||
|
this.subscriptions = [
|
||||||
|
client.on("status", (status) => {
|
||||||
|
if (client !== this.client) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (status === "closed") {
|
||||||
|
void this.stop(this.state.error ? "error" : "closed");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (status === "listening" || status === "user-speaking" || status === "ai-speaking") {
|
||||||
|
this.lastActivity = Date.now();
|
||||||
|
}
|
||||||
|
if (status === "user-speaking") {
|
||||||
|
window.speechSynthesis?.cancel();
|
||||||
|
this.update({
|
||||||
|
messages: this.state.messages.map((message) =>
|
||||||
|
message.role === "assistant" && message.partial
|
||||||
|
? { ...message, partial: false, interrupted: true }
|
||||||
|
: message,
|
||||||
|
),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
this.update({ status });
|
||||||
|
}),
|
||||||
|
client.on("transcript", (transcript) => {
|
||||||
|
this.lastActivity = Date.now();
|
||||||
|
this.transcript(transcript);
|
||||||
|
if (transcript.role === "user" && !transcript.partial && !this.reasoned.has(transcript.id)) {
|
||||||
|
this.reasoned.add(transcript.id);
|
||||||
|
void this.reason(transcript, client);
|
||||||
|
}
|
||||||
|
}),
|
||||||
|
client.on("finished", (response) => {
|
||||||
|
if (response.transcript) {
|
||||||
|
this.transcript(
|
||||||
|
{ id: response.responseId, role: "assistant", text: response.transcript, partial: false },
|
||||||
|
response.status !== "completed",
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}),
|
||||||
|
client.on("level", (level) => {
|
||||||
|
if (!this.state.muted && level > 0.0025) {
|
||||||
|
this.lastActivity = Date.now();
|
||||||
|
}
|
||||||
|
if (Date.now() - this.lastMeter > 150) {
|
||||||
|
this.lastMeter = Date.now();
|
||||||
|
this.update({ level });
|
||||||
|
}
|
||||||
|
}),
|
||||||
|
client.on("queue", ({ position }) => {
|
||||||
|
this.update({ queuePosition: position });
|
||||||
|
}),
|
||||||
|
client.on("readyToJoin", ({ expiresSec }) => {
|
||||||
|
this.update({ joinDeadline: Date.now() + expiresSec * 1000 });
|
||||||
|
}),
|
||||||
|
client.on("grant", () => {
|
||||||
|
this.update({ joinDeadline: 0 });
|
||||||
|
}),
|
||||||
|
client.on("error", (error) => {
|
||||||
|
this.reportError(error);
|
||||||
|
}),
|
||||||
|
client.on("serverError", (error) => {
|
||||||
|
this.update({ error: error.message });
|
||||||
|
}),
|
||||||
|
client.on("tool", (call) => {
|
||||||
|
void this.runTool(call, client);
|
||||||
|
}),
|
||||||
|
];
|
||||||
|
this.lastActivity = Date.now();
|
||||||
|
this.idleTimer = setInterval(() => {
|
||||||
|
if (
|
||||||
|
["listening", "user-speaking", "processing"].includes(this.state.status) &&
|
||||||
|
Date.now() - this.lastActivity >= 30000
|
||||||
|
) {
|
||||||
|
void this.stop();
|
||||||
|
this.update({ notice: "Sesión finalizada por inactividad. Toca para volver a conversar." });
|
||||||
|
}
|
||||||
|
}, 1000);
|
||||||
|
try {
|
||||||
|
await client.connect();
|
||||||
|
} catch (error) {
|
||||||
|
// A closed event may already have cleared the client; explicit cancellation is silent.
|
||||||
|
if (!(error instanceof SessionError && error.code === "aborted") && (!this.client || this.client === client)) {
|
||||||
|
this.reportError(error);
|
||||||
|
if (automatic) {
|
||||||
|
this.update({ unlock: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
if (context) {
|
||||||
|
void context.close().catch(() => undefined);
|
||||||
|
}
|
||||||
|
this.reportError(error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
private reportError(error: unknown) {
|
||||||
|
this.update({
|
||||||
|
error: errorMessage(error),
|
||||||
|
status: "error",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
private transcript(transcript: Transcript, interrupted = false) {
|
||||||
|
if (!transcript.text.trim()) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const id = `${String(this.sessionNumber)}:${transcript.role}:${transcript.id}`;
|
||||||
|
const existing = this.state.messages.find((message) => message.id === id);
|
||||||
|
const message: Message = {
|
||||||
|
...existing,
|
||||||
|
id,
|
||||||
|
role: transcript.role,
|
||||||
|
text: transcript.text,
|
||||||
|
partial: transcript.partial,
|
||||||
|
timestamp: existing?.timestamp ?? new Date().toISOString(),
|
||||||
|
interrupted: interrupted || existing?.interrupted,
|
||||||
|
};
|
||||||
|
this.update({
|
||||||
|
messages: existing
|
||||||
|
? this.state.messages.map((item) => (item.id === id ? message : item))
|
||||||
|
: [...this.state.messages, message],
|
||||||
|
});
|
||||||
|
}
|
||||||
|
private async reason(transcript: Transcript, client: RealtimeClient) {
|
||||||
|
const { config, profile, rag } = this.state;
|
||||||
|
if (config.reasoningType === "PROXY" || !transcript.text.trim()) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const result = await post(
|
||||||
|
"/api/reasoning",
|
||||||
|
reasoningSchema,
|
||||||
|
{
|
||||||
|
transcript: transcript.text.trim(),
|
||||||
|
instructions: this.instructions(),
|
||||||
|
language: "es-ES",
|
||||||
|
user_context: [profileContext(profile), ragContext(rag)].filter(Boolean).join("\n\n"),
|
||||||
|
phoneNumber: this.caller.phoneNumber || null,
|
||||||
|
episode_id: this.caller.episodeId || null,
|
||||||
|
},
|
||||||
|
this.requests.signal,
|
||||||
|
);
|
||||||
|
if (this.client !== client || result.log_only) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const answer = result.answer.trim();
|
||||||
|
if (!answer) {
|
||||||
|
throw new Error("El servicio de razonamiento no devolvió una respuesta.");
|
||||||
|
}
|
||||||
|
client.renderExternalAnswer(answer);
|
||||||
|
this.transcript({ role: "assistant", id: `reasoning:${transcript.id}`, text: answer, partial: false });
|
||||||
|
if ("speechSynthesis" in window) {
|
||||||
|
const utterance = new SpeechSynthesisUtterance(answer);
|
||||||
|
utterance.lang = "es-ES";
|
||||||
|
const voice = window.speechSynthesis.getVoices().find((voice) => voice.lang.startsWith("es"));
|
||||||
|
if (voice) {
|
||||||
|
utterance.voice = voice;
|
||||||
|
}
|
||||||
|
utterance.onstart = () => {
|
||||||
|
if (this.client === client) {
|
||||||
|
this.update({ status: "ai-speaking" });
|
||||||
|
}
|
||||||
|
};
|
||||||
|
utterance.onend = utterance.onerror = () => {
|
||||||
|
if (this.client === client) {
|
||||||
|
this.lastActivity = Date.now();
|
||||||
|
this.update({ status: "listening" });
|
||||||
|
}
|
||||||
|
};
|
||||||
|
window.speechSynthesis.cancel();
|
||||||
|
window.speechSynthesis.speak(utterance);
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
if (this.client === client && !this.requests.signal.aborted) {
|
||||||
|
this.update({ error: errorMessage(error) });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
private async runTool(call: ToolCall, client: RealtimeClient) {
|
||||||
|
const id = `${String(this.sessionNumber)}:tool:${call.callId}`;
|
||||||
|
let args = call.arguments;
|
||||||
|
let output: string;
|
||||||
|
try {
|
||||||
|
const parsed: unknown = JSON.parse(call.arguments || "{}");
|
||||||
|
args = JSON.stringify(parsed, null, 2);
|
||||||
|
if (call.name !== "web_search") {
|
||||||
|
throw new Error(`Herramienta desconocida: ${call.name}`);
|
||||||
|
}
|
||||||
|
const query = z.object({ query: z.string().trim().min(1) }).parse(parsed).query;
|
||||||
|
const result = await post(
|
||||||
|
"/api/search",
|
||||||
|
searchSchema,
|
||||||
|
{
|
||||||
|
query,
|
||||||
|
...(!this.state.config.search && this.state.settings.searchKey ? { key: this.state.settings.searchKey } : {}),
|
||||||
|
},
|
||||||
|
this.requests.signal,
|
||||||
|
);
|
||||||
|
output = [
|
||||||
|
`Google search result from ${new Date().toISOString().slice(0, 10)}:`,
|
||||||
|
result.answer ? `Answer: ${result.answer}` : "",
|
||||||
|
...result.results.map((item) => `- ${item.title}: ${item.snippet} (${item.url})`),
|
||||||
|
]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join("\n");
|
||||||
|
if (!result.answer && !result.results.length) {
|
||||||
|
output += "\nNo results found.";
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
output = `Tool error: ${errorMessage(error)}`;
|
||||||
|
}
|
||||||
|
if (client !== this.client) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.update({
|
||||||
|
messages: [
|
||||||
|
...this.state.messages,
|
||||||
|
{
|
||||||
|
id,
|
||||||
|
role: "tool",
|
||||||
|
text: output,
|
||||||
|
toolName: call.name,
|
||||||
|
toolArguments: args,
|
||||||
|
partial: false,
|
||||||
|
timestamp: new Date().toISOString(),
|
||||||
|
},
|
||||||
|
],
|
||||||
|
});
|
||||||
|
client.sendToolOutput(call.callId, output);
|
||||||
|
}
|
||||||
|
join() {
|
||||||
|
this.client?.join();
|
||||||
|
this.update({ joinDeadline: 0 });
|
||||||
|
}
|
||||||
|
toggleMuted() {
|
||||||
|
const muted = !this.state.muted;
|
||||||
|
this.client?.setMuted(muted);
|
||||||
|
this.update({ muted });
|
||||||
|
}
|
||||||
|
async restart() {
|
||||||
|
// Resume before awaiting teardown so browsers retain the click's audio permission.
|
||||||
|
const context = createAudioContext();
|
||||||
|
await this.stop();
|
||||||
|
await this.start(false, context);
|
||||||
|
}
|
||||||
|
async stop(status: ConnectionStatus = "closed") {
|
||||||
|
const client = this.client;
|
||||||
|
this.client = null;
|
||||||
|
this.subscriptions.forEach((unsubscribe) => {
|
||||||
|
unsubscribe();
|
||||||
|
});
|
||||||
|
this.subscriptions = [];
|
||||||
|
clearInterval(this.idleTimer);
|
||||||
|
this.requests.abort();
|
||||||
|
window.speechSynthesis?.cancel();
|
||||||
|
this.update({
|
||||||
|
status,
|
||||||
|
muted: false,
|
||||||
|
level: 0,
|
||||||
|
joinDeadline: 0,
|
||||||
|
unlock: false,
|
||||||
|
messages: this.state.messages.map((message) =>
|
||||||
|
message.partial ? { ...message, partial: false, interrupted: message.role === "assistant" } : message,
|
||||||
|
),
|
||||||
|
});
|
||||||
|
await client?.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
@layer theme, base, components, utilities;
|
||||||
|
@import "tailwindcss/theme.css" layer(theme);
|
||||||
|
@import "./styles/legacy-theme.css" layer(components);
|
||||||
|
@import "tailwindcss/utilities.css" layer(utilities);
|
||||||
|
|
||||||
|
@layer components {
|
||||||
|
.chat-empty,
|
||||||
|
.chat-empty-hint {
|
||||||
|
opacity: 1;
|
||||||
|
color: var(--text-dim);
|
||||||
|
}
|
||||||
|
html[data-theme="white"] .chat-empty,
|
||||||
|
html[data-theme="white"] .chat-empty-hint {
|
||||||
|
color: #64748b;
|
||||||
|
}
|
||||||
|
.feedback-navigation {
|
||||||
|
align-self: flex-end;
|
||||||
|
display: flex;
|
||||||
|
}
|
||||||
|
#root {
|
||||||
|
display: contents;
|
||||||
|
}
|
||||||
|
.state-idle .ind-mic {
|
||||||
|
opacity: 1;
|
||||||
|
transform: scale(1);
|
||||||
|
}
|
||||||
|
html[data-theme="white"] .topbar-right[data-debug-enabled="true"] {
|
||||||
|
display: flex;
|
||||||
|
grid-column: 1 / -1;
|
||||||
|
grid-row: 2;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
justify-content: center;
|
||||||
|
margin-top: 12px;
|
||||||
|
}
|
||||||
|
html[data-theme="white"] .white-mic-circle svg.wmc-mic {
|
||||||
|
fill: none;
|
||||||
|
}
|
||||||
|
#app:not(.has-reports) {
|
||||||
|
padding-left: 0;
|
||||||
|
}
|
||||||
|
.loading-screen {
|
||||||
|
position: fixed;
|
||||||
|
inset: 0;
|
||||||
|
display: grid;
|
||||||
|
place-items: center;
|
||||||
|
background: white;
|
||||||
|
color: #334155;
|
||||||
|
font: 500 16px system-ui;
|
||||||
|
z-index: 1000;
|
||||||
|
}
|
||||||
|
.app-error {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 12px;
|
||||||
|
margin: 12px auto;
|
||||||
|
padding: 10px 16px;
|
||||||
|
max-width: 680px;
|
||||||
|
border: 1px solid var(--error);
|
||||||
|
border-radius: 12px;
|
||||||
|
font-size: 13px;
|
||||||
|
background: var(--bg-elev);
|
||||||
|
}
|
||||||
|
.talk-unlocker {
|
||||||
|
border: 0;
|
||||||
|
width: 100%;
|
||||||
|
color: inherit;
|
||||||
|
font: inherit;
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
.talk-unlocker-hint > span {
|
||||||
|
display: block;
|
||||||
|
}
|
||||||
|
.agent-tool-item {
|
||||||
|
padding: 12px 0;
|
||||||
|
border-bottom: 1px solid var(--border);
|
||||||
|
overflow-wrap: anywhere;
|
||||||
|
}
|
||||||
|
.queue-actions {
|
||||||
|
align-items: center;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
justify-content: center;
|
||||||
|
}
|
||||||
|
.queue-actions .circle-subcaption {
|
||||||
|
flex-basis: 100%;
|
||||||
|
}
|
||||||
|
.reports-sidebar-backdrop {
|
||||||
|
border: 0;
|
||||||
|
cursor: pointer;
|
||||||
|
visibility: hidden;
|
||||||
|
}
|
||||||
|
.reports-sidebar-inner {
|
||||||
|
visibility: hidden;
|
||||||
|
}
|
||||||
|
.reports-sidebar {
|
||||||
|
pointer-events: none;
|
||||||
|
}
|
||||||
|
.reports-sidebar.open .reports-sidebar-inner,
|
||||||
|
.reports-sidebar.open .reports-sidebar-backdrop {
|
||||||
|
visibility: visible;
|
||||||
|
}
|
||||||
|
@media (min-width: 1024px) {
|
||||||
|
#reports-sidebar-close {
|
||||||
|
display: none;
|
||||||
|
}
|
||||||
|
.reports-sidebar-inner {
|
||||||
|
visibility: visible;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.bubble-body {
|
||||||
|
max-height: 240px;
|
||||||
|
overflow: auto;
|
||||||
|
}
|
||||||
|
.white-session-controls svg {
|
||||||
|
width: 18px;
|
||||||
|
height: 18px;
|
||||||
|
}
|
||||||
|
.mic-gate-arc:focus-visible {
|
||||||
|
outline: 2px solid var(--accent-2);
|
||||||
|
outline-offset: 4px;
|
||||||
|
border-radius: 50%;
|
||||||
|
}
|
||||||
|
.user-profile-row {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: 90px 1fr;
|
||||||
|
gap: 16px;
|
||||||
|
padding: 12px 0;
|
||||||
|
}
|
||||||
|
.user-profile-value {
|
||||||
|
margin: 0;
|
||||||
|
overflow-wrap: anywhere;
|
||||||
|
}
|
||||||
|
.user-profile-label {
|
||||||
|
color: var(--text-dim);
|
||||||
|
}
|
||||||
|
.hist-tool-block {
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,13 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
|
||||||
|
import { createBackendProxy } from "../../config/backend-proxy";
|
||||||
|
|
||||||
|
describe("development backend proxy", () => {
|
||||||
|
it("keeps the public host and original paths for the FastAPI runtime URL", () => {
|
||||||
|
const proxy = createBackendProxy("http://127.0.0.1:8421");
|
||||||
|
expect(proxy["/api"]).toEqual({ target: "http://127.0.0.1:8421", changeOrigin: false });
|
||||||
|
expect(proxy["/v1/realtime"]).toEqual({ target: "http://127.0.0.1:8421", changeOrigin: false, ws: true });
|
||||||
|
expect(proxy["/openapi.json"]).toEqual(proxy["/api"]);
|
||||||
|
expect(proxy["/docs"]).toEqual(proxy["/api"]);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
import "@testing-library/jest-dom/vitest";
|
||||||
|
|
||||||
|
import { cleanup } from "@testing-library/react";
|
||||||
|
import { afterEach } from "vitest";
|
||||||
|
|
||||||
|
// jsdom has no native modal top layer; real focus/inert behavior is checked in Chromium.
|
||||||
|
Object.defineProperties(HTMLDialogElement.prototype, {
|
||||||
|
showModal: {
|
||||||
|
configurable: true,
|
||||||
|
writable: true,
|
||||||
|
value: function (this: HTMLDialogElement) {
|
||||||
|
this.open = true;
|
||||||
|
},
|
||||||
|
},
|
||||||
|
close: {
|
||||||
|
configurable: true,
|
||||||
|
writable: true,
|
||||||
|
value: function (this: HTMLDialogElement) {
|
||||||
|
this.open = false;
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
cleanup();
|
||||||
|
window.localStorage.clear();
|
||||||
|
window.sessionStorage.clear();
|
||||||
|
});
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.app.tsbuildinfo",
|
||||||
|
"target": "es2023",
|
||||||
|
"lib": ["ES2023", "DOM", "DOM.Iterable"],
|
||||||
|
"module": "esnext",
|
||||||
|
"types": ["vite/client"],
|
||||||
|
"allowArbitraryExtensions": true,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
|
||||||
|
/* Bundler mode */
|
||||||
|
"moduleResolution": "bundler",
|
||||||
|
"allowImportingTsExtensions": true,
|
||||||
|
"verbatimModuleSyntax": true,
|
||||||
|
"moduleDetection": "force",
|
||||||
|
"noEmit": true,
|
||||||
|
"jsx": "react-jsx",
|
||||||
|
|
||||||
|
/* Linting */
|
||||||
|
"strict": true,
|
||||||
|
"noUncheckedIndexedAccess": true,
|
||||||
|
"noUnusedLocals": true,
|
||||||
|
"noUnusedParameters": true,
|
||||||
|
"erasableSyntaxOnly": true,
|
||||||
|
"noFallthroughCasesInSwitch": true
|
||||||
|
},
|
||||||
|
"include": ["src"]
|
||||||
|
}
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
{
|
||||||
|
"files": [],
|
||||||
|
"references": [{ "path": "./tsconfig.app.json" }, { "path": "./tsconfig.node.json" }]
|
||||||
|
}
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.node.tsbuildinfo",
|
||||||
|
"target": "es2023",
|
||||||
|
"lib": ["ES2023"],
|
||||||
|
"types": ["node"],
|
||||||
|
"skipLibCheck": true,
|
||||||
|
|
||||||
|
/* Bundler mode */
|
||||||
|
"module": "esnext",
|
||||||
|
"moduleResolution": "bundler",
|
||||||
|
"allowImportingTsExtensions": true,
|
||||||
|
"verbatimModuleSyntax": true,
|
||||||
|
"moduleDetection": "force",
|
||||||
|
"noEmit": true,
|
||||||
|
|
||||||
|
/* Linting */
|
||||||
|
"strict": true,
|
||||||
|
"noUnusedLocals": true,
|
||||||
|
"noUnusedParameters": true,
|
||||||
|
"erasableSyntaxOnly": true,
|
||||||
|
"noFallthroughCasesInSwitch": true
|
||||||
|
},
|
||||||
|
"include": ["*.config.ts", "config/**/*.ts"]
|
||||||
|
}
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
import { fileURLToPath } from "node:url";
|
||||||
|
|
||||||
|
import tailwindcss from "@tailwindcss/vite";
|
||||||
|
import react from "@vitejs/plugin-react";
|
||||||
|
import { defineConfig, loadEnv } from "vite";
|
||||||
|
|
||||||
|
import { createBackendProxy } from "./config/backend-proxy.ts";
|
||||||
|
|
||||||
|
export default defineConfig(({ mode }) => {
|
||||||
|
// Read this project's configuration; the repository .env contains server secrets.
|
||||||
|
const env = loadEnv(mode, process.cwd(), "S2S_");
|
||||||
|
return {
|
||||||
|
plugins: [react(), tailwindcss()],
|
||||||
|
resolve: { alias: { "@": fileURLToPath(new URL("./src", import.meta.url)) } },
|
||||||
|
server: {
|
||||||
|
host: "127.0.0.1",
|
||||||
|
port: 8420,
|
||||||
|
strictPort: true,
|
||||||
|
ws: { path: "/__vite_hmr" },
|
||||||
|
allowedHosts: (env.S2S_DEV_ALLOWED_HOSTS ?? "")
|
||||||
|
.split(",")
|
||||||
|
.map((host) => host.trim())
|
||||||
|
.filter(Boolean),
|
||||||
|
proxy: createBackendProxy(env.S2S_BACKEND_URL ?? "http://127.0.0.1:8421"),
|
||||||
|
},
|
||||||
|
build: { target: "es2022" },
|
||||||
|
};
|
||||||
|
});
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
import { defineConfig } from "vitest/config";
|
||||||
|
|
||||||
|
export default defineConfig({
|
||||||
|
test: {
|
||||||
|
environment: "jsdom",
|
||||||
|
execArgv: process.allowedNodeEnvironmentFlags.has("--no-experimental-webstorage")
|
||||||
|
? ["--no-experimental-webstorage"]
|
||||||
|
: [],
|
||||||
|
setupFiles: ["./src/test/setup.ts"],
|
||||||
|
restoreMocks: true,
|
||||||
|
},
|
||||||
|
});
|
||||||
+126
-7
@@ -4,16 +4,45 @@ set -euo pipefail
|
|||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
VENV_DIR="$ROOT_DIR/.venv"
|
VENV_DIR="$ROOT_DIR/.venv"
|
||||||
REQ_FILE="$ROOT_DIR/s2s-client/requirements.txt"
|
REQ_FILE="$ROOT_DIR/s2s-client/requirements.txt"
|
||||||
UI_PORT="${UI_PORT:-8420}"
|
FRONTEND_DIR="$ROOT_DIR/s2s-frontend"
|
||||||
if [ -f "$ROOT_DIR/.env" ]; then
|
if [ -f "$ROOT_DIR/.env" ]; then
|
||||||
set -a
|
set -a
|
||||||
source "$ROOT_DIR/.env"
|
source "$ROOT_DIR/.env"
|
||||||
set +a
|
set +a
|
||||||
fi
|
fi
|
||||||
|
UI_PORT="${UI_PORT:-8420}"
|
||||||
|
UI_HOST="${UI_HOST:-0.0.0.0}"
|
||||||
|
S2S_BACKEND_PORT="${S2S_BACKEND_PORT:-8421}"
|
||||||
S2S_PROXY_TARGET="${S2S_PROXY_TARGET:-ws://ai-server:12345/v1/realtime}"
|
S2S_PROXY_TARGET="${S2S_PROXY_TARGET:-ws://ai-server:12345/v1/realtime}"
|
||||||
|
|
||||||
if ! command -v python3 >/dev/null 2>&1; then
|
if (( BASH_VERSINFO[0] < 4 || (BASH_VERSINFO[0] == 4 && BASH_VERSINFO[1] < 4) )); then
|
||||||
echo "Error: python3 was not found."
|
echo "Error: start_ui.sh requires Bash 4.4 or newer."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
for required_command in python3 node npm setsid curl; do
|
||||||
|
if ! command -v "$required_command" >/dev/null 2>&1; then
|
||||||
|
echo "Error: $required_command was not found."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if ! node -e 'const [major, minor] = process.versions.node.split(".").map(Number); process.exit((major === 22 && minor >= 18) || major >= 24 ? 0 : 1)'; then
|
||||||
|
echo "Error: s2s-frontend requires Node.js 22.18+ on the 22.x line, or Node.js 24+."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
for port_name in UI_PORT S2S_BACKEND_PORT; do
|
||||||
|
port_value="${!port_name}"
|
||||||
|
if [[ ! "$port_value" =~ ^[0-9]{1,5}$ ]] || (( 10#$port_value < 1 || 10#$port_value > 65535 )); then
|
||||||
|
echo "Error: $port_name must be a port between 1 and 65535."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
# Normalize leading zeroes before passing ports to Node and Uvicorn.
|
||||||
|
printf -v "$port_name" '%d' "$((10#$port_value))"
|
||||||
|
done
|
||||||
|
if [ "$UI_PORT" = "$S2S_BACKEND_PORT" ]; then
|
||||||
|
echo "Error: UI_PORT and S2S_BACKEND_PORT must be different."
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -29,17 +58,107 @@ if [ ! -f "$REQ_FILE" ]; then
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
if ! python -c "import fastapi, uvicorn, httpx, huggingface_hub, dotenv" >/dev/null 2>&1; then
|
if ! python -c "import fastapi, uvicorn, httpx, dotenv, websockets" >/dev/null 2>&1; then
|
||||||
echo "Installing dependencies from s2s-client/requirements.txt..."
|
echo "Installing dependencies from s2s-client/requirements.txt..."
|
||||||
python -m pip install -r "$REQ_FILE"
|
python -m pip install -r "$REQ_FILE"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# Keep restarts offline when the installed frontend manifests have not changed.
|
||||||
|
if [ ! -f "$FRONTEND_DIR/node_modules/vite/bin/vite.js" ] || \
|
||||||
|
! cmp -s "$FRONTEND_DIR/package-lock.json" "$FRONTEND_DIR/node_modules/.s2s-package-lock.json" || \
|
||||||
|
! cmp -s "$FRONTEND_DIR/package.json" "$FRONTEND_DIR/node_modules/.s2s-package.json"; then
|
||||||
|
echo "Installing frontend dependencies from s2s-frontend/package-lock.json..."
|
||||||
|
npm --prefix "$FRONTEND_DIR" ci --include=dev
|
||||||
|
cp "$FRONTEND_DIR/package-lock.json" "$FRONTEND_DIR/node_modules/.s2s-package-lock.json"
|
||||||
|
cp "$FRONTEND_DIR/package.json" "$FRONTEND_DIR/node_modules/.s2s-package.json"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# The combined launcher always proxies to the backend it starts. An alternate
|
||||||
|
# S2S_BACKEND_URL is only used when running Vite independently.
|
||||||
|
export S2S_BACKEND_URL="http://127.0.0.1:$S2S_BACKEND_PORT"
|
||||||
export S2S_PROXY_TARGET
|
export S2S_PROXY_TARGET
|
||||||
|
|
||||||
echo "💻 Starting the S2S client UI (port $UI_PORT)..."
|
child_pids=()
|
||||||
|
cleanup() {
|
||||||
|
local exit_status=$?
|
||||||
|
trap - EXIT
|
||||||
|
trap '' INT TERM HUP
|
||||||
|
for child_pid in "${child_pids[@]}"; do
|
||||||
|
kill -TERM -- "-$child_pid" 2>/dev/null || true
|
||||||
|
done
|
||||||
|
# Stop whole process groups, including Uvicorn's reload worker. Bound the
|
||||||
|
# graceful shutdown so a stuck connection cannot leave ports occupied.
|
||||||
|
for ((attempt = 0; attempt < 20; attempt++)); do
|
||||||
|
local alive=false
|
||||||
|
for child_pid in "${child_pids[@]}"; do
|
||||||
|
if kill -0 -- "-$child_pid" 2>/dev/null; then
|
||||||
|
alive=true
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ "$alive" = false ]; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 0.05
|
||||||
|
done
|
||||||
|
for child_pid in "${child_pids[@]}"; do
|
||||||
|
kill -KILL -- "-$child_pid" 2>/dev/null || true
|
||||||
|
wait "$child_pid" 2>/dev/null || true
|
||||||
|
done
|
||||||
|
exit "$exit_status"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
trap 'exit 130' INT
|
||||||
|
trap 'exit 143' TERM
|
||||||
|
trap 'exit 129' HUP
|
||||||
|
|
||||||
|
echo "💻 Starting the React UI at http://$UI_HOST:$UI_PORT (Vite hot reload)..."
|
||||||
|
echo "🔧 FastAPI backend: $S2S_BACKEND_URL (Python reload enabled)"
|
||||||
echo "🔌 Realtime proxy target: $S2S_PROXY_TARGET"
|
echo "🔌 Realtime proxy target: $S2S_PROXY_TARGET"
|
||||||
echo " Press Ctrl+C in this terminal to stop it."
|
echo " Press Ctrl+C in this terminal to stop both servers."
|
||||||
echo "----------------------------------------------------"
|
echo "----------------------------------------------------"
|
||||||
|
|
||||||
cd "$ROOT_DIR"
|
cd "$ROOT_DIR"
|
||||||
PYTHONPATH=s2s-client python -m uvicorn server:app --host 0.0.0.0 --port "$UI_PORT" --reload
|
setsid "$VENV_DIR/bin/python" -m uvicorn server:app \
|
||||||
|
--app-dir "$ROOT_DIR/s2s-client" \
|
||||||
|
--host 127.0.0.1 --port "$S2S_BACKEND_PORT" \
|
||||||
|
--reload --reload-dir "$ROOT_DIR/s2s-client" &
|
||||||
|
backend_pid=$!
|
||||||
|
child_pids+=("$backend_pid")
|
||||||
|
|
||||||
|
backend_ready=false
|
||||||
|
startup_deadline=$((SECONDS + 30))
|
||||||
|
while (( SECONDS < startup_deadline )); do
|
||||||
|
if ! kill -0 "$backend_pid" 2>/dev/null; then
|
||||||
|
echo "Error: FastAPI exited before it was ready."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if curl --noproxy '*' --silent --fail --max-time 1 \
|
||||||
|
--output /dev/null "$S2S_BACKEND_URL/api/config"; then
|
||||||
|
backend_ready=true
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 0.2
|
||||||
|
done
|
||||||
|
if [ "$backend_ready" = false ]; then
|
||||||
|
echo "Error: FastAPI did not become ready within 30 seconds."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
(
|
||||||
|
cd "$FRONTEND_DIR"
|
||||||
|
# Keep React Fast Refresh enabled even if PM2 exports NODE_ENV=production.
|
||||||
|
export NODE_ENV=development
|
||||||
|
exec setsid node node_modules/vite/bin/vite.js --host "$UI_HOST" --port "$UI_PORT" --strictPort
|
||||||
|
) &
|
||||||
|
frontend_pid=$!
|
||||||
|
child_pids+=("$frontend_pid")
|
||||||
|
|
||||||
|
# PM2 supervises this script. If either server exits, stop its sibling and fail
|
||||||
|
# the launcher so PM2 can restart the complete pair instead of a partial UI.
|
||||||
|
exit_status=0
|
||||||
|
wait -n "$backend_pid" "$frontend_pid" || exit_status=$?
|
||||||
|
echo "Error: a UI server exited; stopping both servers."
|
||||||
|
if [ "$exit_status" -eq 0 ]; then
|
||||||
|
exit_status=1
|
||||||
|
fi
|
||||||
|
exit "$exit_status"
|
||||||
|
|||||||
Reference in New Issue
Block a user