Compare commits
47
Commits
e529435980
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
dfed03ac7c | ||
|
|
0311d3f1ff | ||
|
|
4b0108ecf0 | ||
|
|
96895996d9 | ||
|
|
ccb1f708ca
|
||
|
|
62c2692082
|
||
|
|
c06c9194e8
|
||
|
|
65eaf36278
|
||
|
|
f3bae8ae0c
|
||
|
|
264d7905c6
|
||
|
|
e9808cb12b
|
||
|
|
4ce6082b93
|
||
|
|
a56e1082e5
|
||
|
|
0c57f7c76d
|
||
|
|
09bf6ece00
|
||
|
|
ff32c6a605
|
||
|
|
abd03ed07d
|
||
|
|
e0304b2bdb
|
||
|
|
77b60622e9
|
||
|
|
8cc080998f
|
||
|
|
1146dec1ac
|
||
|
|
fa00b36e49
|
||
|
|
f8d5b7857a
|
||
|
|
92c53dc7ca
|
||
|
|
34f0d1b125
|
||
|
|
3d42df5749
|
||
|
|
c375581077
|
||
|
|
179f5ac7db
|
||
|
|
f91001169c
|
||
|
|
e4b3833770
|
||
|
|
6c4ee3a5e3
|
||
|
|
99413fc06d
|
||
|
|
c8f8aff8c7
|
||
|
|
65c965ae6b
|
||
|
|
cb09e75bbd
|
||
|
|
7285d22e6c
|
||
|
|
36efd5e13c
|
||
|
|
833788b8d8
|
||
|
|
36e2bee885
|
||
|
|
0965077cac
|
||
|
|
c5ddb3f333
|
||
|
|
a149f4ef8b
|
||
|
|
cdbff5cb0e
|
||
|
|
dfc1f663c7
|
||
|
|
6b8c27fd76
|
||
|
|
b91762a927
|
||
|
|
e0b64dfb99
|
@@ -0,0 +1,23 @@
|
|||||||
|
.git/
|
||||||
|
.agents/
|
||||||
|
.codex/
|
||||||
|
.aws/
|
||||||
|
**/.env
|
||||||
|
**/.env.*
|
||||||
|
!**/.env.example
|
||||||
|
**/secrets/
|
||||||
|
**/__pycache__/
|
||||||
|
**/*.pyc
|
||||||
|
**/.venv/
|
||||||
|
**/venv/
|
||||||
|
**/node_modules/
|
||||||
|
**/.cache/
|
||||||
|
**/*.log
|
||||||
|
**/uploads/
|
||||||
|
**/sessions/
|
||||||
|
debug_files/
|
||||||
|
tmp/
|
||||||
|
**/.mypy_cache/
|
||||||
|
**/.pytest_cache/
|
||||||
|
**/.ruff_cache/
|
||||||
|
**/*.egg-info/
|
||||||
+8
-14
@@ -38,6 +38,8 @@ node_modules/
|
|||||||
.npm/
|
.npm/
|
||||||
package-lock.json
|
package-lock.json
|
||||||
!speech_to_speech/package-lock.json
|
!speech_to_speech/package-lock.json
|
||||||
|
!vocero-frontend/package-lock.json
|
||||||
|
vocero-frontend/.eslintcache
|
||||||
yarn-error.log
|
yarn-error.log
|
||||||
npm-debug.log*
|
npm-debug.log*
|
||||||
yarn-debug.log*
|
yarn-debug.log*
|
||||||
@@ -121,21 +123,13 @@ CVS/
|
|||||||
.trae/
|
.trae/
|
||||||
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
# ─────────────────────────────────────────────────────────────
|
||||||
# Reasoning engine (Node-RED) local state
|
# Client backend and speech-service runtime artifacts
|
||||||
# ─────────────────────────────────────────────────────────────
|
# ─────────────────────────────────────────────────────────────
|
||||||
s2s-reasoning-engine/node_modules/
|
vocero-api/.cache/
|
||||||
s2s-reasoning-engine/.node-red/
|
vocero-api/uploads/
|
||||||
s2s-reasoning-engine/lib/flows_cred.json
|
vocero-api/sessions/
|
||||||
s2s-reasoning-engine/.config.*
|
vocero-api/*.log
|
||||||
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
|
||||||
# s2s-client static artifacts generated at runtime by browser / proxy logs
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
|
||||||
s2s-client/.cache/
|
|
||||||
s2s-client/uploads/
|
|
||||||
s2s-client/sessions/
|
|
||||||
s2s-client/*.log
|
|
||||||
speech_to_speech/*.log
|
speech_to_speech/*.log
|
||||||
speech_to_speech/sessions/
|
speech_to_speech/sessions/
|
||||||
speech_to_speech/.cache/
|
speech_to_speech/.cache/
|
||||||
s2s-reasoning-engine/*.log
|
debug_files
|
||||||
|
|||||||
@@ -0,0 +1,239 @@
|
|||||||
|
# Repository guidance
|
||||||
|
|
||||||
|
## Scope and working conventions
|
||||||
|
|
||||||
|
This repository is Vocero: a browser voice client, its FastAPI integration
|
||||||
|
server, and a speech pipeline. Keep this guide self-contained and base its
|
||||||
|
instructions on this checkout's code, configuration, documentation, and tools.
|
||||||
|
|
||||||
|
All file and directory references in this guide must point within the repository
|
||||||
|
and be relative to its root. Commands may use the project's command-line tools,
|
||||||
|
such as `rg`, `git`, `uv`, `npm`, Docker, and PM2. HTTP routes mentioned below are
|
||||||
|
URLs, not filesystem paths.
|
||||||
|
|
||||||
|
- Check the current branch, worktree, and working-tree changes before editing.
|
||||||
|
Honor the checkout requested by the user, including direct work on `main`.
|
||||||
|
Use an isolated worktree when requested and preserve other worktrees.
|
||||||
|
- An analysis-only or planning request authorizes inspection, not implementation.
|
||||||
|
Return findings with source locations. Present plans in the conversation
|
||||||
|
unless the user asks to save a file; follow any explicit approval gate.
|
||||||
|
- Write clear explanations and ask one conceptual question at a time. Carry out
|
||||||
|
already authorized work without repeatedly asking for permission.
|
||||||
|
- Use the native patch/edit tool for manual changes. Prefer `rg`, `fd`, `jq`,
|
||||||
|
`git`, and standard CLI tools for inspection; check availability before
|
||||||
|
assuming a tool is missing. Do not install system-wide packages unnecessarily.
|
||||||
|
- When commits are requested, make coherent, scoped Conventional Commits as work
|
||||||
|
progresses. Avoid excessive micro-commits and unrelated cleanup.
|
||||||
|
- For a requested pull request, describe the problem, resulting behavior,
|
||||||
|
validation, and relevant limitations. Create issues or post comments only
|
||||||
|
when requested.
|
||||||
|
- When asked to merge locally, use the requested rebase/fast-forward flow;
|
||||||
|
preserve a recovery ref before rewriting and verify retained patches. A local
|
||||||
|
merge does not imply a push, deployment, or restart. Report those separately.
|
||||||
|
- Treat historical proposals in `docs/proposals/` as context. Verify their
|
||||||
|
claims against current code and distinguish proposed authentication or
|
||||||
|
deployment changes from implemented behavior.
|
||||||
|
|
||||||
|
## Project map
|
||||||
|
|
||||||
|
| Location | Responsibility |
|
||||||
|
| ---------------------------------- | ------------------------------------------------------------------------------------------- |
|
||||||
|
| `vocero-api/` | The complete uv project: manifests, environment, Python tooling, tests, and FastAPI package |
|
||||||
|
| `vocero-api/vocero_api/main.py` | Application assembly, documentation mount, and retained redirects |
|
||||||
|
| `vocero-api/vocero_api/api/` | HTTP routes, route registration, and response validation helper |
|
||||||
|
| `vocero-api/vocero_api/models/` | Pydantic request and response contracts |
|
||||||
|
| `vocero-api/vocero_api/core/` | Environment parsing, logging, public URLs, and startup diagnostics |
|
||||||
|
| `vocero-api/vocero_api/providers/` | Authenticated MCP transport, backend login, and caller/profile/report HTTP integrations |
|
||||||
|
| `vocero-api/vocero_api/services/` | Prompt/context construction, reasoning, and tool-context rendering |
|
||||||
|
| `vocero-api/vocero_api/realtime/` | WebSocket relay, event handling, marker filtering, and audio forwarding |
|
||||||
|
| `vocero-frontend/` | React, TypeScript, Tailwind, and Vite browser application |
|
||||||
|
| `speech_to_speech/` | Separately run VAD → STT → LLM → TTS pipeline and realtime API |
|
||||||
|
| `start_ui.sh` | Dependency sync and supervision of FastAPI plus Vite |
|
||||||
|
| `docs/` | Maintained technical documentation, examples, and historical proposals |
|
||||||
|
|
||||||
|
`vocero-api/server.py` is a compatibility shim for `server:app`; new backend
|
||||||
|
implementation belongs in the package. Keep HTTP handlers focused on transport
|
||||||
|
and reuse providers/services for shared behavior.
|
||||||
|
|
||||||
|
The old static browser site, daily talk-time quotas, quota database/accounting,
|
||||||
|
and Hugging Face account UI were deliberately removed. Do not restore their
|
||||||
|
files, routes, dependencies, or compatibility layers as incidental cleanup.
|
||||||
|
Load-balancer allocation/queues and upstream caller authentication remain.
|
||||||
|
|
||||||
|
## Tooling and checks
|
||||||
|
|
||||||
|
The uv project must stay inside `vocero-api/`, including
|
||||||
|
`vocero-api/pyproject.toml`, `vocero-api/uv.lock`, `vocero-api/.python-version`,
|
||||||
|
`vocero-api/.venv/`, `vocero-api/Makefile`, and
|
||||||
|
`vocero-api/.pre-commit-config.yaml`.
|
||||||
|
Use Python 3.12 and the uv version required by `tool.uv.required-version`
|
||||||
|
(currently 0.12.5). Update dependencies through uv and commit the lockfile;
|
||||||
|
do not use pip to modify this environment or move it to the repository root.
|
||||||
|
Keep uv pins in `vocero-api/Dockerfile` and setup documentation aligned when
|
||||||
|
changing the required version.
|
||||||
|
|
||||||
|
The frontend uses npm and `vocero-frontend/package-lock.json`. Its Node requirement is
|
||||||
|
`^22.18.0 || >=24.0.0`; consult `vocero-frontend/package.json` for current scripts.
|
||||||
|
Keep application code and supported configuration files in TypeScript.
|
||||||
|
|
||||||
|
Commands below run from the repository root:
|
||||||
|
|
||||||
|
| Command | Purpose |
|
||||||
|
| ------------------------------------- | --------------------------------------------------------------------- |
|
||||||
|
| `make -C vocero-api install` | Sync the locked Python environment and development tools |
|
||||||
|
| `make -C vocero-api format` | Apply Ruff fixes and formatting |
|
||||||
|
| `make -C vocero-api check` | Check lock freshness, formatting, Ruff lint, mypy, and pytest |
|
||||||
|
| `make -C vocero-api test-launcher` | Test readiness, failures, dependency sync, and process-group shutdown |
|
||||||
|
| `make -C vocero-api test-image` | Build and smoke-test the actual backend Docker image |
|
||||||
|
| `npm --prefix vocero-frontend ci` | Install locked frontend dependencies |
|
||||||
|
| `npm --prefix vocero-frontend run check` | Check formatting, lint, TypeScript, Vitest, and launcher integration |
|
||||||
|
| `npm --prefix vocero-frontend run build` | Type-check and build frontend assets |
|
||||||
|
|
||||||
|
Python uses Ruff with a 120-character line limit, double quotes, import ordering,
|
||||||
|
and typed application functions checked by mypy. Frontend formatting and lint
|
||||||
|
rules live in `vocero-frontend/prettier.config.ts` and
|
||||||
|
`vocero-frontend/eslint.config.ts`; use those tools rather than introducing
|
||||||
|
competing conventions.
|
||||||
|
|
||||||
|
Run checks appropriate to the change: backend checks for Python/API work,
|
||||||
|
frontend checks and build for browser work, launcher integration for supervisor
|
||||||
|
changes, and image tests for packaging/Docker changes. Documentation-only work
|
||||||
|
normally needs link/path and diff checks, not service startup. Wait for commands
|
||||||
|
to finish before reporting success, and state skipped coverage or blockers.
|
||||||
|
|
||||||
|
Tests should use mock upstream HTTP and local WebSocket fixtures, isolating
|
||||||
|
environment configuration. Do not rely on deployed services, real credentials,
|
||||||
|
or model downloads for client tests. The client gate does not test the separate
|
||||||
|
speech pipeline, microphone hardware, or real model output. That pipeline has
|
||||||
|
no complete dependency manifest or general test gate in this checkout; inspect
|
||||||
|
the selected backend and explain the verification limits for speech changes.
|
||||||
|
|
||||||
|
## API and compatibility rules
|
||||||
|
|
||||||
|
Read [backend development and compatibility](docs/client/development.md) before
|
||||||
|
refactoring the server or changing schemas. Existing wire behavior is the
|
||||||
|
baseline, including behavior that looks unsafe, inconsistent, or buggy.
|
||||||
|
Intentional bug fixes or contract changes need their own scope and regression
|
||||||
|
evidence; do not bundle them into tooling or structural refactors.
|
||||||
|
|
||||||
|
- Preserve paths, methods, statuses, error shapes, headers, field names,
|
||||||
|
omissions, coercion, authentication precedence, and outgoing provider calls.
|
||||||
|
Trace both frontend consumers and provider parsing; a permissive consumer is
|
||||||
|
not evidence that an upstream schema can safely be tightened.
|
||||||
|
- Retain existing request-model defaults and extra-field behavior. Do not make
|
||||||
|
new fields required, forbid previously accepted extras, or normalize loosely
|
||||||
|
parsed provider values without evidence and an intentional behavior change.
|
||||||
|
- Response models validate fields constructed by this service. Upstream-owned
|
||||||
|
values remain permissive where the old parser was permissive: `Any`, optional
|
||||||
|
fields, open objects, and arbitrary profile JSON are deliberate boundaries.
|
||||||
|
- `validated_json_response` in `vocero-api/vocero_api/api/responses.py` validates
|
||||||
|
with Pydantic, then serializes the original payload. Do not substitute
|
||||||
|
`model_dump()` or automatic response-model serialization: those can insert
|
||||||
|
defaults, filter unknown keys,
|
||||||
|
coerce values, or alter serialization. Keep OpenAPI declarations aligned with
|
||||||
|
the actual validation boundary.
|
||||||
|
- Preserve environment/dotenv parsing, default flags, and import-time behavior
|
||||||
|
during refactors. Do not introduce stricter configuration validation or change
|
||||||
|
authentication behavior as part of a tooling change.
|
||||||
|
- `vocero-api/tests/fixtures/legacy-contracts.json` captures the server before the
|
||||||
|
migration at `3d42df5`. `vocero-api/tests/test_compatibility.py` compares HTTP
|
||||||
|
response bytes, statuses, selected headers, outgoing requests, request schemas,
|
||||||
|
and realtime frames. Do not regenerate captures merely to make a refactor pass.
|
||||||
|
- Known reasoning failures and relay
|
||||||
|
behavior are documented in the compatibility guide. Preserve narrow lint/type
|
||||||
|
exceptions that retain those behaviors until a deliberate fix is in scope.
|
||||||
|
- The pytest warning filter is only for Starlette TestClient's exact deprecated
|
||||||
|
AnyIO `BlockingPortal` alias. Do not broaden warning suppression or upgrade
|
||||||
|
runtime frameworks just to silence that warning without a compatibility review.
|
||||||
|
|
||||||
|
## Browser and realtime behavior
|
||||||
|
|
||||||
|
- React renders the UI; `vocero-frontend/src/realtime/controller.ts` owns observable
|
||||||
|
application state, `vocero-frontend/src/realtime/client.ts` owns protocol/session
|
||||||
|
lifecycle, and `vocero-frontend/src/audio/` owns the audio graph and worklets.
|
||||||
|
Keep high-frequency audio/WebSocket work outside React rendering and preserve
|
||||||
|
cleanup across unmounts and reconnects.
|
||||||
|
- Preserve the existing 16 kHz PCM16 audio path, worklet registration, interruption
|
||||||
|
behavior, and teardown. Session startup initializes audio, waits for
|
||||||
|
`session.created`, and sends `session.update` before audio or the greeting.
|
||||||
|
A `session.updated` acknowledgment is optional; requiring it previously broke
|
||||||
|
startup with this speech service.
|
||||||
|
- Preserve query parameters such as `phoneNumber` and `episode_id`, existing
|
||||||
|
`s2s.ws.*` preference keys, session-only RAG, and TALK/TALKFIRST behavior unless
|
||||||
|
the task explicitly changes them. Context comes from browser instructions and
|
||||||
|
server/provider calls; do not assume a vector database or a completed magic-link
|
||||||
|
authentication flow exists.
|
||||||
|
- Assistant Markdown is parsed with Marked and sanitized with DOMPurify in
|
||||||
|
`vocero-frontend/src/lib/messages.ts`. User text stays literal, and exports retain
|
||||||
|
raw message text. Keep sanitization and unsafe-link regression coverage when
|
||||||
|
changing it.
|
||||||
|
- Reuse the Tailwind tokens in `vocero-frontend/src/styles/theme.css` and shared
|
||||||
|
controls in `vocero-frontend/src/components/ui.tsx`. Keep specialized artwork
|
||||||
|
and sanitized Markdown styles scoped under `vocero-frontend/src/styles/`.
|
||||||
|
Verify both light and dark themes,
|
||||||
|
narrow screens, keyboard focus, dialog dismissal, and voice-state feedback for
|
||||||
|
UI changes.
|
||||||
|
Preserve intentional `?debug=true` controls and browser audio-permission flows.
|
||||||
|
- User-visible times use `America/Santiago` where no saved zone exists; avoid
|
||||||
|
browser-local timezone assumptions. Keep raw timestamps in their existing API
|
||||||
|
and export formats.
|
||||||
|
|
||||||
|
## Startup and deployment boundaries
|
||||||
|
|
||||||
|
`./start_ui.sh` loads the optional root `.env`, syncs `vocero-api/.venv/` with
|
||||||
|
`uv sync --project vocero-api --locked`, and installs frontend packages when the
|
||||||
|
manifests change.
|
||||||
|
It starts Vite on `0.0.0.0:8420` and FastAPI on `127.0.0.1:8421`, with
|
||||||
|
`UI_HOST`, `UI_PORT`, and `S2S_BACKEND_PORT` overrides. Use distinct ports for
|
||||||
|
concurrent worktrees. Preserve readiness checks, failure propagation, and
|
||||||
|
shutdown of both process groups, including Uvicorn's reload worker.
|
||||||
|
|
||||||
|
Staging intentionally retains Vite HMR and Uvicorn reload. The Vite child uses
|
||||||
|
`NODE_ENV=development` even under PM2. Do not replace this with static production
|
||||||
|
serving unless that deployment change is requested. Vite proxies `/api`,
|
||||||
|
`/v1/realtime`, and documentation routes to FastAPI; preserve public Host headers
|
||||||
|
used to construct WebSocket URLs. Remote staging needs explicit
|
||||||
|
`S2S_DEV_ALLOWED_HOSTS` hostnames and reverse-proxy WebSocket forwarding for both
|
||||||
|
`/v1/realtime` and `/__vite_hmr`.
|
||||||
|
|
||||||
|
The established PM2 process name is `s2s_ui`. When it points to this launcher's
|
||||||
|
deployed checkout, `pm2 restart s2s_ui --update-env` also performs the locked uv
|
||||||
|
sync; a separate sync is optional to prepare dependencies before restarting.
|
||||||
|
Ensure the required uv and Node/npm are on that process's PATH. Verify the live
|
||||||
|
PM2 script path and environment before making claims about a deployment.
|
||||||
|
|
||||||
|
The launcher does not start or restart `speech_to_speech/`. A configured
|
||||||
|
speech URL does not prove which source revision or implementation runs there.
|
||||||
|
`SPEECH_TO_SPEECH_URL` pins a direct browser connection; `S2S_PROXY_TARGET` enables
|
||||||
|
the local relay. Either disables load-balancer mode. Reasoning supports `PROXY`
|
||||||
|
and `OPENAI`; unknown `S2S_REASONING_TYPE` values fall back to `PROXY`.
|
||||||
|
|
||||||
|
The backend Docker image serves APIs, WebSockets, and root `docs/`, not the React
|
||||||
|
UI. Build with the repository root as context and `vocero-api/Dockerfile`; the
|
||||||
|
image listens on 7860. FastAPI's root remains 404, `/docs` is the API explorer,
|
||||||
|
and project documentation is served below `/docs/` with retained legacy redirects.
|
||||||
|
|
||||||
|
## Language, documentation, and secrets
|
||||||
|
|
||||||
|
- Use English for code, identifiers, comments, logs, internal instructions, and
|
||||||
|
developer documentation. Spanish is the default for visible UI, spoken output,
|
||||||
|
and user-facing API messages. Preserve literal wake phrases and protocol names;
|
||||||
|
translating an existing response or prompt can change a contract.
|
||||||
|
- Keep maintained documentation under `docs/client/`, `docs/speech/`, `docs/adr/`,
|
||||||
|
`docs/examples/`, or `docs/proposals/`. Update existing documents first and keep
|
||||||
|
`docs/README.md` current when adding a document. Use one canonical English file
|
||||||
|
without language suffixes. Historical proposals do not define current behavior.
|
||||||
|
- Reserve root Markdown for established files such as `README.md` and `AGENTS.md`.
|
||||||
|
Keep task summaries in the conversation unless a saved artifact is requested.
|
||||||
|
Record notable changes under `Unreleased` in `vocero-api/CHANGELOG.md`.
|
||||||
|
- Keep credentials and deployment values out of commits and examples. Never copy
|
||||||
|
the root `.env` into the frontend or expose backend secrets through `VITE_*`.
|
||||||
|
The launcher loads `.env` when present, so its values can override shell values;
|
||||||
|
direct Python startup uses `load_dotenv(..., override=False)`. Preserve both
|
||||||
|
behaviors and avoid importing live configuration when a test can isolate it.
|
||||||
|
|
||||||
|
Start with [usage](docs/client/usage.md), [architecture](docs/client/architecture.md),
|
||||||
|
[development](docs/client/development.md), and [troubleshooting](docs/client/troubleshooting.md).
|
||||||
|
For speech work, read [the realtime protocol](docs/speech/realtime-api.md) and the
|
||||||
|
relevant [STT](docs/speech/stt.md), [LLM](docs/speech/llm.md), or
|
||||||
|
[TTS](docs/speech/tts.md) guide before changing pipeline behavior.
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
# Vocero speech-to-speech
|
||||||
|
|
||||||
|
Vocero provides a browser voice interface, a FastAPI integration server, and a
|
||||||
|
Python speech pipeline. The interface defaults to Spanish;
|
||||||
|
documentation, source identifiers, comments, and internal instructions use English.
|
||||||
|
|
||||||
|
| Component | Purpose |
|
||||||
|
|---|---|
|
||||||
|
| [vocero-frontend](vocero-frontend) | React, TypeScript, and Tailwind browser UI, served by Vite with hot reload |
|
||||||
|
| [vocero-api](vocero-api) | FastAPI backend, realtime WebSocket proxy, search and authenticated AquaPortal MCP |
|
||||||
|
| [speech_to_speech](speech_to_speech) | Queue-driven VAD, speech recognition, language model, and speech synthesis pipeline |
|
||||||
|
|
||||||
|
Start with [client setup and usage](docs/client/usage.md), then read the
|
||||||
|
[architecture](docs/client/architecture.md) and [documentation index](docs/README.md).
|
||||||
|
The speech backend needs its own installed model and runtime dependencies;
|
||||||
|
The uv project in `vocero-api/` installs only the UI server dependencies.
|
||||||
|
|
||||||
|
## Run the browser client
|
||||||
|
|
||||||
|
From the repository root, point the client at an existing realtime backend:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
S2S_PROXY_TARGET=ws://localhost:8765/v1/realtime ./start_ui.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Open <http://localhost:8420/> and allow microphone access. The launcher creates
|
||||||
|
`vocero-api/.venv`, syncs locked Python and frontend dependencies,
|
||||||
|
and loads the root `.env` if present. Vite listens on `UI_PORT` (default `8420`)
|
||||||
|
and forwards HTTP/WebSocket backend routes to FastAPI on `127.0.0.1:8421`.
|
||||||
|
Both servers reload during development; stopping the launcher stops both.
|
||||||
|
Leave `SPEECH_TO_SPEECH_URL` unset to use the same-origin speech proxy.
|
||||||
|
|
||||||
|
The launcher requires uv 0.12.5 (which provisions Python 3.12), Node.js 22.18+ on the 22.x
|
||||||
|
line or Node.js 24+, npm, Bash 4.4+, `setsid`, and `curl`. For staging hostnames,
|
||||||
|
set `S2S_DEV_ALLOWED_HOSTS` and forward the Vite HMR WebSocket through the existing
|
||||||
|
HTTPS reverse proxy. See [development setup](vocero-frontend/README.md).
|
||||||
|
|
||||||
|
The Docker image serves backend routes and shared documentation. The React
|
||||||
|
frontend runs separately through Vite. To build the backend image, use the
|
||||||
|
repository root as the Docker build context so documentation is included:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker build -f vocero-api/Dockerfile -t vocero-api .
|
||||||
|
docker run --rm -p 7860:7860 \
|
||||||
|
-e SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime vocero-api
|
||||||
|
```
|
||||||
|
|
||||||
|
The direct WebSocket URL is resolved by the user's browser. For a server-side
|
||||||
|
proxy deployment, supply a backend address reachable from the container instead.
|
||||||
|
|
||||||
|
## Project references
|
||||||
|
|
||||||
|
- [Change history](vocero-api/CHANGELOG.md)
|
||||||
|
- [Backend development and compatibility](docs/client/development.md)
|
||||||
|
- [Realtime protocol and pipeline](docs/speech/realtime-api.md)
|
||||||
|
- [Context injection](docs/client/context-injection.md)
|
||||||
|
- [Tool calling](docs/client/tool-calling.md)
|
||||||
|
- [Troubleshooting](docs/client/troubleshooting.md)
|
||||||
|
- [Historical architecture proposal](docs/proposals/vocero-architecture.html)
|
||||||
@@ -1,401 +0,0 @@
|
|||||||
<!doctype html>
|
|
||||||
<html lang="en">
|
|
||||||
<head>
|
|
||||||
<meta charset="UTF-8" />
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
|
|
||||||
<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate, max-age=0" />
|
|
||||||
<meta http-equiv="Pragma" content="no-cache" />
|
|
||||||
<meta http-equiv="Expires" content="0" />
|
|
||||||
<title>Minimal Conversation · S2S backend (WebSocket)</title>
|
|
||||||
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
|
||||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
|
|
||||||
<link
|
|
||||||
href="https://fonts.googleapis.com/css2?family=Geist+Mono:wght@400;500;600&family=Inter:wght@400;500;600;700;800&display=swap"
|
|
||||||
rel="stylesheet"
|
|
||||||
/>
|
|
||||||
<link rel="stylesheet" href="style.css?v=20260902001" />
|
|
||||||
</head>
|
|
||||||
<!--
|
|
||||||
`booting` disables transitions/animations until the first paint commits.
|
|
||||||
Without this the orb visibly fades/scales in on the very first frame
|
|
||||||
because `state-idle` and the generic `.ind` defaults differ. The
|
|
||||||
bootstrap script at the end of main.js strips the class after one rAF.
|
|
||||||
-->
|
|
||||||
<body class="booting">
|
|
||||||
<div id="app">
|
|
||||||
<header class="topbar">
|
|
||||||
<div class="brand">
|
|
||||||
<div class="ident">
|
|
||||||
<div class="ident-head">
|
|
||||||
<a class="ident-title">Azurian Vocero realtime ver.Beta 2.0</a>
|
|
||||||
</div>
|
|
||||||
<div id="ident-user-name" class="ident-user-name" hidden></div>
|
|
||||||
<div id="reports-hint" class="reports-hint" hidden></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div class="topbar-right">
|
|
||||||
<!-- HF login chip / sign-in pill. Populated by ui/account.js; only
|
|
||||||
shown when the deploy runs behind a load balancer with OAuth. -->
|
|
||||||
<div id="account" class="account" hidden></div>
|
|
||||||
<button id="about-btn-m" class="icon-btn about-btn-mobile" title="About this space" aria-label="About this space">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"/><line x1="12" y1="16" x2="12" y2="12"/><line x1="12" y1="8" x2="12.01" y2="8"/></svg>
|
|
||||||
</button>
|
|
||||||
<button id="tools-btn" class="icon-btn" title="Tools" aria-label="Tools">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/></svg>
|
|
||||||
</button>
|
|
||||||
<button id="rag-btn" class="icon-btn" title="Custom RAG context" aria-label="Custom RAG context">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M4 19.5A2.5 2.5 0 0 1 6.5 17H20"/><path d="M6.5 2H20v20H6.5A2.5 2.5 0 0 1 4 19.5v-15A2.5 2.5 0 0 1 6.5 2z"/></svg>
|
|
||||||
</button>
|
|
||||||
<button id="chat-btn" class="icon-btn" title="Conversation history" aria-label="Conversation history">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/></svg>
|
|
||||||
<span id="chat-badge" class="chat-badge" aria-hidden="true"></span>
|
|
||||||
</button>
|
|
||||||
<button id="settings-btn" class="icon-btn" title="Settings" aria-label="Settings">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="3"/><path d="M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 1 1-2.83 2.83l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 1 1-4 0v-.09A1.65 1.65 0 0 0 9 19.4a1.65 1.65 0 0 0-1.82.33l-.06.06a2 2 0 1 1-2.83-2.83l.06-.06a1.65 1.65 0 0 0 .33-1.82 1.65 1.65 0 0 0-1.51-1H3a2 2 0 1 1 0-4h.09A1.65 1.65 0 0 0 4.6 9a1.65 1.65 0 0 0-.33-1.82l-.06-.06a2 2 0 1 1 2.83-2.83l.06.06a1.65 1.65 0 0 0 1.82.33H9a1.65 1.65 0 0 0 1-1.51V3a2 2 0 1 1 4 0v.09a1.65 1.65 0 0 0 1 1.51 1.65 1.65 0 0 0 1.82-.33l.06-.06a2 2 0 1 1 2.83 2.83l-.06.06a1.65 1.65 0 0 0-.33 1.82V9c0 .66.39 1.25 1 1.51H21a2 2 0 1 1 0 4h-.09a1.65 1.65 0 0 0-1.51 1z"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<main class="stage">
|
|
||||||
<div class="orb-wrap">
|
|
||||||
<div id="mic-gate" class="mic-gate">
|
|
||||||
<svg id="mic-gate-arc" class="mic-gate-arc" viewBox="0 0 100 100" aria-hidden="true">
|
|
||||||
<path id="mga-track" class="mga-track" fill="none" />
|
|
||||||
<path id="mga-fill" class="mga-fill" fill="none" />
|
|
||||||
<path id="mga-hit" class="mga-hit" fill="none" />
|
|
||||||
<circle id="mga-handle" class="mga-handle" r="3" />
|
|
||||||
</svg>
|
|
||||||
<button id="mic-btn" class="side-btn" type="button" aria-label="Mute" title="Mute" aria-hidden="true">
|
|
||||||
<svg class="mic-on" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="2" width="6" height="12" rx="3"/><path d="M5 10a7 7 0 0 0 14 0"/><line x1="12" y1="19" x2="12" y2="22"/></svg>
|
|
||||||
<svg class="mic-off" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><line x1="2" y1="2" x2="22" y2="22"/><path d="M9 5a3 3 0 0 1 6 0v4"/><path d="M9 10v1a3 3 0 0 0 5.1 2.1"/><path d="M19 10a7 7 0 0 1-1.24 3.97"/><path d="M5 10a7 7 0 0 0 11 5.67"/><line x1="12" y1="19" x2="12" y2="22"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<button
|
|
||||||
id="main-circle"
|
|
||||||
class="circle state-idle"
|
|
||||||
type="button"
|
|
||||||
aria-label="Start voice conversation"
|
|
||||||
>
|
|
||||||
<span class="circle-glow" aria-hidden="true"></span>
|
|
||||||
<span class="circle-ring" aria-hidden="true"></span>
|
|
||||||
<span class="circle-ring-outer" aria-hidden="true"></span>
|
|
||||||
<span class="circle-core">
|
|
||||||
<span class="circle-indicator" aria-hidden="true">
|
|
||||||
<svg class="ind ind-mic" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<rect x="9" y="2" width="6" height="12" rx="3" fill="currentColor" stroke="none"/>
|
|
||||||
<path d="M5 10a7 7 0 0 0 14 0"/>
|
|
||||||
<line x1="12" y1="19" x2="12" y2="22"/>
|
|
||||||
<line x1="8" y1="22" x2="16" y2="22"/>
|
|
||||||
</svg>
|
|
||||||
<svg class="ind ind-error" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="9"/><line x1="12" y1="8" x2="12" y2="13"/><line x1="12" y1="16" x2="12" y2="16"/></svg>
|
|
||||||
<span class="ind ind-spinner"></span>
|
|
||||||
<span class="ind ind-thinking">
|
|
||||||
<span class="dot"></span>
|
|
||||||
<span class="dot"></span>
|
|
||||||
<span class="dot"></span>
|
|
||||||
</span>
|
|
||||||
<span class="ind ind-bars">
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
</span>
|
|
||||||
<svg class="ind ind-voice" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<path d="M3 10v4a1 1 0 0 0 1 1h3l5 4V5L7 9H4a1 1 0 0 0-1 1z" fill="currentColor" stroke="none"/>
|
|
||||||
<path class="wave wave-1" d="M16 8a5 5 0 0 1 0 8"/>
|
|
||||||
<path class="wave wave-2" d="M19 5a9 9 0 0 1 0 14"/>
|
|
||||||
</svg>
|
|
||||||
</span>
|
|
||||||
</span>
|
|
||||||
</button>
|
|
||||||
|
|
||||||
<button id="stop-btn" class="side-btn" type="button" aria-label="End" title="End" aria-hidden="true">
|
|
||||||
<svg viewBox="0 0 24 24" fill="currentColor"><rect x="6" y="6" width="12" height="12" rx="2"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<p id="circle-caption" class="circle-caption" role="status">Tap to start</p>
|
|
||||||
<p id="circle-subcaption" class="circle-subcaption" hidden></p>
|
|
||||||
|
|
||||||
<div id="queue-actions" class="queue-actions" hidden>
|
|
||||||
<button id="join-queue-btn" class="join-queue-btn" type="button" hidden>
|
|
||||||
Join now
|
|
||||||
</button>
|
|
||||||
<button id="leave-queue-btn" class="leave-queue-btn" type="button" hidden>
|
|
||||||
Leave queue
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</main>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Ephemeral chat bubbles anchored to the top-right -->
|
|
||||||
<div id="bubble-stack" class="bubble-stack" aria-live="polite" aria-atomic="false"></div>
|
|
||||||
|
|
||||||
<!-- Webcam self-view, shown bottom-left while the camera tool is enabled.
|
|
||||||
HIDDEN BY DEFAULT TODAY: riattivare togliendo hidden + scommentando listener in main.js -->
|
|
||||||
<div id="cam-pip" class="cam-pip" aria-hidden="true" hidden>
|
|
||||||
<video id="cam-video" class="cam-video" autoplay playsinline muted></video>
|
|
||||||
<span class="cam-flash" aria-hidden="true"></span>
|
|
||||||
<span class="cam-label">camera</span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Conversation history panel (overlay) -->
|
|
||||||
<div id="chat-panel" class="chat-panel">
|
|
||||||
<div id="chat-panel-backdrop" class="chat-panel-backdrop"></div>
|
|
||||||
<div class="chat-panel-inner">
|
|
||||||
<header class="chat-panel-header">
|
|
||||||
<h3>Conversation</h3>
|
|
||||||
<button id="chat-panel-close" class="icon-btn" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
<div id="chat-history" class="chat-history"></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<dialog id="about-modal" class="modal about-modal">
|
|
||||||
<div class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>About</h2>
|
|
||||||
<button id="about-close" class="icon-btn" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<!-- General intro to the speech-to-speech project -->
|
|
||||||
<div class="about-intro">
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- The backend, shown as the path a turn actually travels -->
|
|
||||||
<div class="about-pipeline">
|
|
||||||
<p class="pipeline-title">The pipeline</p>
|
|
||||||
<ol class="pipeline">
|
|
||||||
<li class="pipe-endpoint">You speak</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">VAD</span>
|
|
||||||
<span class="pipe-job">detects speech</span>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">STT</span>
|
|
||||||
<span class="pipe-job">transcribes it</span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/nvidia/parakeet-tdt-1.1b" target="_blank" rel="noopener">nvidia/parakeet-tdt-1.1b<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">VLM</span>
|
|
||||||
<span class="pipe-job">analyses and composes the reply <span class="pipe-note">· via <a class="cerebras-credit" href="https://cerebras.ai" target="_blank" rel="noopener"><svg class="cerebras-mark" viewBox="0 0 50 50" aria-hidden="true"><path fill-rule="evenodd" clip-rule="evenodd" d="M29.4186 5.62713C24.2806 5.62713 19.353 7.6682 15.7199 11.3013C12.0868 14.9345 10.0457 19.862 10.0457 25C10.0457 30.1381 12.0868 35.0656 15.7199 38.6988C19.353 42.3319 24.2806 44.373 29.4186 44.373V47.2917C17.1061 47.2917 7.12695 37.3105 7.12695 24.998C7.12695 12.6855 17.104 2.7063 29.4165 2.7063V5.62505L29.4186 5.62713ZM39.3186 13.2875C37.7794 11.9809 35.9971 10.9915 34.0741 10.3761C32.1512 9.76067 30.1256 9.53147 28.1137 9.70166C26.1019 9.87185 24.1435 10.4381 22.3513 11.3677C20.559 12.2974 18.9683 13.5722 17.6704 15.1189C16.3726 16.6655 15.3933 18.4534 14.7888 20.3798C14.1844 22.3062 13.9667 24.3332 14.1483 26.344C14.33 28.3548 14.9073 30.3099 15.8472 32.0968C16.787 33.8838 18.0709 35.4673 19.6249 36.7563L17.7478 38.9938C15.9126 37.4545 14.3987 35.5687 13.2924 33.4442C12.1862 31.3197 11.5092 28.9981 11.3003 26.6119C11.0914 24.2258 11.3545 21.8219 12.0747 19.5374C12.7949 17.253 13.9581 15.1328 15.4978 13.298C17.0373 11.4628 18.9233 9.94877 21.0479 8.84247C23.1726 7.73618 25.4944 7.05923 27.8807 6.8503C30.267 6.64136 32.6711 6.90453 34.9557 7.62477C37.2403 8.34501 39.3607 9.50822 41.1957 11.048L39.3186 13.2875ZM34.6207 15.0459C31.9818 13.6829 28.9113 13.4172 26.0776 14.3068C23.2438 15.1964 20.876 17.1691 19.4895 19.7958C18.1029 22.4224 17.8099 25.4903 18.6741 28.332C19.5384 31.1736 21.4899 33.5589 24.104 34.9688L22.7374 37.5521C19.4721 35.7617 17.0418 32.7592 15.9707 29.1927C14.8996 25.6261 15.2737 21.7815 17.0123 18.4883C18.7509 15.1952 21.7146 12.7176 25.2637 11.5903C28.8129 10.463 32.663 10.7763 35.9832 12.4625L34.6207 15.0459ZM29.4165 17.7896C27.5042 17.7896 25.6702 18.5493 24.318 19.9015C22.9658 21.2537 22.2061 23.0877 22.2061 25C22.2061 26.9124 22.9658 28.7464 24.318 30.0986C25.6702 31.4508 27.5042 32.2105 29.4165 32.2105V35.1313C26.7296 35.1313 24.1526 34.0639 22.2527 32.1639C20.3527 30.2639 19.2853 27.687 19.2853 25C19.2853 22.3131 20.3527 19.7362 22.2527 17.8362C24.1526 15.9362 26.7296 14.8688 29.4165 14.8688V17.7896Z" fill="#F15A29"/><path d="M32.0981 22.5749C31.7875 22.2404 31.4295 21.9534 31.0356 21.7228C30.6925 21.519 30.3014 21.4097 29.9023 21.4062C29.371 21.4062 28.896 21.5041 28.4773 21.6978C28.0725 21.8802 27.7089 22.1425 27.4081 22.469C27.1074 22.7955 26.8758 23.1795 26.7273 23.5978C26.5731 24.0207 26.4981 24.4645 26.4981 24.9124C26.4981 25.3666 26.5731 25.8082 26.7273 26.227C26.877 26.6449 27.1091 27.0286 27.4096 27.3553C27.7102 27.682 28.0733 27.9451 28.4773 28.1291C28.8939 28.3228 29.3731 28.4207 29.9023 28.4207C30.3523 28.4207 30.771 28.3249 31.1564 28.1395C31.5481 27.952 31.8856 27.6707 32.146 27.3228L34.0794 29.4187C33.7877 29.7103 33.4544 29.9624 33.0752 30.1749C32.374 30.566 31.6108 30.8338 30.8189 30.9666C30.4648 31.0207 30.1585 31.0499 29.9023 31.0499C29.0617 31.0552 28.2271 30.9069 27.4398 30.6124C26.6954 30.3371 26.0144 29.914 25.4377 29.3687C24.8643 28.8208 24.4078 28.1624 24.096 27.4332C23.7572 26.6366 23.5883 25.778 23.6002 24.9124C23.6002 23.9874 23.7669 23.1478 24.096 22.3916C24.4085 21.6624 24.8627 21.0041 25.4356 20.4562C26.0148 19.9124 26.696 19.4895 27.4398 19.2145C28.2271 18.92 29.0617 18.7717 29.9023 18.777C30.6419 18.777 31.3856 18.9187 32.1356 19.202C32.8877 19.4895 33.5627 19.952 34.1023 20.5541L32.0981 22.5749Z" fill="#fff"/></svg>Cerebras</a></span></span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/google/gemma-4-31B-it" target="_blank" rel="noopener">google/gemma-4-31B-it<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">TTS</span>
|
|
||||||
<span class="pipe-job">speaks it back</span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice" target="_blank" rel="noopener">Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-endpoint">The orb replies</li>
|
|
||||||
</ol>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<dialog id="settings-modal" class="modal">
|
|
||||||
<form method="dialog" class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Settings</h2>
|
|
||||||
<button class="icon-btn" value="close" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<div class="tab-panels">
|
|
||||||
<section class="tab-panel active" role="tabpanel">
|
|
||||||
<label class="field" id="conn-field">
|
|
||||||
<span id="conn-label">Speech-to-speech server URL</span>
|
|
||||||
<input id="lb-url" type="text" autocomplete="off" spellcheck="false" placeholder="http://localhost:port" />
|
|
||||||
<small id="conn-hint">
|
|
||||||
URL of your speech-to-speech server, e.g.
|
|
||||||
<code>http://localhost:8080</code> (the app adds <code>/v1/realtime</code>).
|
|
||||||
</small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<div class="field-row">
|
|
||||||
<label class="field">
|
|
||||||
<span>Voice</span>
|
|
||||||
<select id="voice">
|
|
||||||
<option value="Aiden" selected>Aiden</option>
|
|
||||||
<option value="Ryan">Ryan</option>
|
|
||||||
<option value="Dylan">Dylan</option>
|
|
||||||
<option value="Eric">Eric</option>
|
|
||||||
<option value="Ono_Anna">Ono_Anna</option>
|
|
||||||
<option value="Serena">Serena</option>
|
|
||||||
<option value="Sohee">Sohee</option>
|
|
||||||
<option value="Uncle_Fu">Uncle_Fu</option>
|
|
||||||
<option value="Vivian">Vivian</option>
|
|
||||||
</select>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="field">
|
|
||||||
<span class="field-head">
|
|
||||||
Noise gate
|
|
||||||
<span id="gate-value" class="field-value">Off</span>
|
|
||||||
</span>
|
|
||||||
<div class="gate">
|
|
||||||
<div class="gate-track" aria-hidden="true">
|
|
||||||
<div id="gate-meter-fill" class="gate-meter-fill"></div>
|
|
||||||
<input
|
|
||||||
id="noise-gate"
|
|
||||||
type="range"
|
|
||||||
min="-66"
|
|
||||||
max="-3"
|
|
||||||
step="1"
|
|
||||||
value="-50"
|
|
||||||
aria-label="Noise gate threshold"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
<div class="gate-ends">
|
|
||||||
<span>Off</span>
|
|
||||||
<span>−3 dB</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<small>
|
|
||||||
Mutes the mic below the handle so room noise isn't sent. Slide fully
|
|
||||||
left to turn it off; the bar shows your live input while in a call.
|
|
||||||
</small>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<label class="field">
|
|
||||||
<span>Instructions</span>
|
|
||||||
<textarea id="instructions" rows="5" placeholder="You are Vocero, a friendly voice assistant for the company Aquachile..."></textarea>
|
|
||||||
<small id="primary-prompt-meta" hidden></small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<div class="field">
|
|
||||||
<button id="restart-conversation" type="button" class="btn primary wide" disabled>
|
|
||||||
Restart conversation with these settings
|
|
||||||
</button>
|
|
||||||
<small id="restart-hint">Connect first, then come back here to apply live changes.</small>
|
|
||||||
</div>
|
|
||||||
</section>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<footer class="modal-footer">
|
|
||||||
<button id="settings-save" type="submit" class="btn primary" value="save">Save</button>
|
|
||||||
</footer>
|
|
||||||
</form>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<dialog id="tools-modal" class="modal">
|
|
||||||
<div class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Tools</h2>
|
|
||||||
<button id="tools-close" class="icon-btn" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<p class="tools-intro">Let the assistant act during the conversation. Changes apply live.</p>
|
|
||||||
|
|
||||||
<div class="tool-list">
|
|
||||||
<div class="tool-row" id="tool-web-row">
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Web search</span>
|
|
||||||
<span class="tool-desc">Look things up on Google, via Serper.</span>
|
|
||||||
</div>
|
|
||||||
<label class="switch">
|
|
||||||
<input id="tool-web" type="checkbox" />
|
|
||||||
<span class="switch-track" aria-hidden="true"></span>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
<label class="field tools-key">
|
|
||||||
<span>Search API key</span>
|
|
||||||
<input id="search-key" type="password" autocomplete="off" spellcheck="false" />
|
|
||||||
<small id="tool-web-hint"></small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<!-- Camera tool row — HIDDEN BY DEFAULT TODAY.
|
|
||||||
Per riattivarlo in futuro: togli hidden da div + small,
|
|
||||||
scommenta listener toolCamSwitch + funzioni enableCamera in main.js. -->
|
|
||||||
<div class="tool-row tool-row-sep" id="tool-cam-row" hidden>
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Camera</span>
|
|
||||||
<span class="tool-desc">Let the assistant see through your webcam.</span>
|
|
||||||
</div>
|
|
||||||
<label class="switch">
|
|
||||||
<input id="tool-cam" type="checkbox" />
|
|
||||||
<span class="switch-track" aria-hidden="true"></span>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
<small id="tool-cam-hint" class="tool-hint" hidden>A live preview shows bottom-left while the camera is on.</small>
|
|
||||||
|
|
||||||
<div class="tool-row tool-row-sep">
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Loaded MCP tools</span>
|
|
||||||
<span class="tool-desc">Tools published by the remote agent profile.</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<small id="agent-tools-hint" class="tool-hint"></small>
|
|
||||||
<div id="agent-tools-list" class="agent-tools-list"></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<dialog id="rag-modal" class="modal">
|
|
||||||
<div class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Custom RAG context</h2>
|
|
||||||
<button id="rag-close" class="icon-btn" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<p class="tools-intro">
|
|
||||||
Incolla qui testo, documenti, o estratti RAG che vuoi siano disponibili
|
|
||||||
per la sessione vocale corrente. Il testo viene aggiunto sia alle
|
|
||||||
istruzioni del LLM sia al campo <code>user_context</code> del body
|
|
||||||
<code>/api/reasoning</code>.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<label class="field">
|
|
||||||
<span>RAG context (testo libero, puro)</span>
|
|
||||||
<textarea
|
|
||||||
id="rag-textarea"
|
|
||||||
rows="12"
|
|
||||||
spellcheck="false"
|
|
||||||
placeholder="Incolla o scrivi qui il contesto RAG che il modello deve usare durante la risposta. Es: dati cliente, policy, estratti documentali, ecc."
|
|
||||||
></textarea>
|
|
||||||
<small id="rag-hint" class="tool-hint">
|
|
||||||
Nessun RAG context caricato. Premi "Invia nella sessione" per applicare le modifiche.
|
|
||||||
</small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<footer class="modal-footer">
|
|
||||||
<button id="rag-clear" class="btn secondary" type="button" value="clear">Svuota</button>
|
|
||||||
<div style="flex: 1"></div>
|
|
||||||
<button id="rag-submit" class="btn primary" type="button" value="submit">Invia nella sessione</button>
|
|
||||||
</footer>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<!-- Shown when the per-day conversation budget is spent (mid-call or at
|
|
||||||
start). Title / message / CTA / note are filled per tier by
|
|
||||||
ui/account.js. The hero is the HF mark — a friendly smiling face. -->
|
|
||||||
<dialog id="limit-modal" class="modal limit-modal">
|
|
||||||
<div class="modal-content limit-card">
|
|
||||||
<button id="limit-close" class="icon-btn limit-close" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
<div class="limit-badge" aria-hidden="true">
|
|
||||||
<svg class="hf-logo" viewBox="0 0 95 88" fill="none" aria-hidden="true"><path fill="#FFD21E" d="M47.21 76.5a34.75 34.75 0 1 0 0-69.5 34.75 34.75 0 0 0 0 69.5Z"/><path fill="#FF9D0B" d="M81.96 41.75a34.75 34.75 0 1 0-69.5 0 34.75 34.75 0 0 0 69.5 0Zm-73.5 0a38.75 38.75 0 1 1 77.5 0 38.75 38.75 0 0 1-77.5 0Z"/><path fill="#3A3B45" d="M58.5 32.3c1.28.44 1.78 3.06 3.07 2.38a5 5 0 1 0-6.76-2.07c.61 1.15 2.55-.72 3.7-.32ZM34.95 32.3c-1.28.44-1.79 3.06-3.07 2.38a5 5 0 1 1 6.76-2.07c-.61 1.15-2.56-.72-3.7-.32Z"/><path fill="#FF323D" d="M46.96 56.29c9.83 0 13-8.76 13-13.26 0-2.34-1.57-1.6-4.09-.36-2.33 1.15-5.46 2.74-8.9 2.74-7.19 0-13-6.88-13-2.38s3.16 13.26 13 13.26Z"/><path fill="#3A3B45" fill-rule="evenodd" d="M39.43 54a8.7 8.7 0 0 1 5.3-4.49c.4-.12.81.57 1.24 1.28.4.68.82 1.37 1.24 1.37.45 0 .9-.68 1.33-1.35.45-.7.89-1.38 1.32-1.25a8.61 8.61 0 0 1 5 4.17c3.73-2.94 5.1-7.74 5.1-10.7 0-2.34-1.57-1.6-4.09-.36l-.14.07c-2.31 1.15-5.39 2.67-8.77 2.67s-6.45-1.52-8.77-2.67c-2.6-1.29-4.23-2.1-4.23.29 0 3.05 1.46 8.06 5.47 10.97Z" clip-rule="evenodd"/><path fill="#FF9D0B" d="M70.71 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM24.21 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM17.52 48c-1.62 0-3.06.66-4.07 1.87a5.97 5.97 0 0 0-1.33 3.76 7.1 7.1 0 0 0-1.94-.3c-1.55 0-2.95.59-3.94 1.66a5.8 5.8 0 0 0-.8 7 5.3 5.3 0 0 0-1.79 2.82c-.24.9-.48 2.8.8 4.74a5.22 5.22 0 0 0-.37 5.02c1.02 2.32 3.57 4.14 8.52 6.1 3.07 1.22 5.89 2 5.91 2.01a44.33 44.33 0 0 0 10.93 1.6c5.86 0 10.05-1.8 12.46-5.34 3.88-5.69 3.33-10.9-1.7-15.92-2.77-2.78-4.62-6.87-5-7.77-.78-2.66-2.84-5.62-6.25-5.62a5.7 5.7 0 0 0-4.6 2.46c-1-1.26-1.98-2.25-2.86-2.82A7.4 7.4 0 0 0 17.52 48Zm0 4c.51 0 1.14.22 1.82.65 2.14 1.36 6.25 8.43 7.76 11.18.5.92 1.37 1.31 2.14 1.31 1.55 0 2.75-1.53.15-3.48-3.92-2.93-2.55-7.72-.68-8.01.08-.02.17-.02.24-.02 1.7 0 2.45 2.93 2.45 2.93s2.2 5.52 5.98 9.3c3.77 3.77 3.97 6.8 1.22 10.83-1.88 2.75-5.47 3.58-9.16 3.58-3.81 0-7.73-.9-9.92-1.46-.11-.03-13.45-3.8-11.76-7 .28-.54.75-.76 1.34-.76 2.38 0 6.7 3.54 8.57 3.54.41 0 .7-.17.83-.6.79-2.85-12.06-4.05-10.98-8.17.2-.73.71-1.02 1.44-1.02 3.14 0 10.2 5.53 11.68 5.53.11 0 .2-.03.24-.1.74-1.2.33-2.04-4.9-5.2-5.21-3.16-8.88-5.06-6.8-7.33.24-.26.58-.38 1-.38 3.17 0 10.66 6.82 10.66 6.82s2.02 2.1 3.25 2.1c.28 0 .52-.1.68-.38.86-1.46-8.06-8.22-8.56-11.01-.34-1.9.24-2.85 1.31-2.85Z"/><path fill="#FFD21E" d="M38.6 76.69c2.75-4.04 2.55-7.07-1.22-10.84-3.78-3.77-5.98-9.3-5.98-9.3s-.82-3.2-2.69-2.9c-1.87.3-3.24 5.08.68 8.01 3.91 2.93-.78 4.92-2.29 2.17-1.5-2.75-5.62-9.82-7.76-11.18-2.13-1.35-3.63-.6-3.13 2.2.5 2.79 9.43 9.55 8.56 11-.87 1.47-3.93-1.71-3.93-1.71s-9.57-8.71-11.66-6.44c-2.08 2.27 1.59 4.17 6.8 7.33 5.23 3.16 5.64 4 4.9 5.2-.75 1.2-12.28-8.53-13.36-4.4-1.08 4.11 11.77 5.3 10.98 8.15-.8 2.85-9.06-5.38-10.74-2.18-1.7 3.21 11.65 6.98 11.76 7.01 4.3 1.12 15.25 3.49 19.08-2.12Z"/><path fill="#FF9D0B" d="M77.4 48c1.62 0 3.07.66 4.07 1.87a5.97 5.97 0 0 1 1.33 3.76 7.1 7.1 0 0 1 1.95-.3c1.55 0 2.95.59 3.94 1.66a5.8 5.8 0 0 1 .8 7 5.3 5.3 0 0 1 1.78 2.82c.24.9.48 2.8-.8 4.74a5.22 5.22 0 0 1 .37 5.02c-1.02 2.32-3.57 4.14-8.51 6.1-3.08 1.22-5.9 2-5.92 2.01a44.33 44.33 0 0 1-10.93 1.6c-5.86 0-10.05-1.8-12.46-5.34-3.88-5.69-3.33-10.9 1.7-15.92 2.78-2.78 4.63-6.87 5.01-7.77.78-2.66 2.83-5.62 6.24-5.62a5.7 5.7 0 0 1 4.6 2.46c1-1.26 1.98-2.25 2.87-2.82A7.4 7.4 0 0 1 77.4 48Zm0 4c-.51 0-1.13.22-1.82.65-2.13 1.36-6.25 8.43-7.76 11.18a2.43 2.43 0 0 1-2.14 1.31c-1.54 0-2.75-1.53-.14-3.48 3.91-2.93 2.54-7.72.67-8.01a1.54 1.54 0 0 0-.24-.02c-1.7 0-2.45 2.93-2.45 2.93s-2.2 5.52-5.97 9.3c-3.78 3.77-3.98 6.8-1.22 10.83 1.87 2.75 5.47 3.58 9.15 3.58 3.82 0 7.73-.9 9.93-1.46.1-.03 13.45-3.8 11.76-7-.29-.54-.75-.76-1.34-.76-2.38 0-6.71 3.54-8.57 3.54-.42 0-.71-.17-.83-.6-.8-2.85 12.05-4.05 10.97-8.17-.19-.73-.7-1.02-1.44-1.02-3.14 0-10.2 5.53-11.68 5.53-.1 0-.19-.03-.23-.1-.74-1.2-.34-2.04 4.88-5.2 5.23-3.16 8.9-5.06 6.8-7.33-.23-.26-.57-.38-.98-.38-3.18 0-10.67 6.82-10.67 6.82s-2.02 2.1-3.24 2.1a.74.74 0 0 1-.68-.38c-.87-1.46 8.05-8.22 8.55-11.01.34-1.9-.24-2.85-1.31-2.85Z"/><path fill="#FFD21E" d="M56.33 76.69c-2.75-4.04-2.56-7.07 1.22-10.84 3.77-3.77 5.97-9.3 5.97-9.3s.82-3.2 2.7-2.9c1.86.3 3.23 5.08-.68 8.01-3.92 2.93.78 4.92 2.28 2.17 1.51-2.75 5.63-9.82 7.76-11.18 2.13-1.35 3.64-.6 3.13 2.2-.5 2.79-9.42 9.55-8.55 11 .86 1.47 3.92-1.71 3.92-1.71s9.58-8.71 11.66-6.44c2.08 2.27-1.58 4.17-6.8 7.33-5.23 3.16-5.63 4-4.9 5.2.75 1.2 12.28-8.53 13.36-4.4 1.08 4.11-11.76 5.3-10.97 8.15.8 2.85 9.05-5.38 10.74-2.18 1.69 3.21-11.65 6.98-11.76 7.01-4.31 1.12-15.26 3.49-19.08-2.12Z"/></svg>
|
|
||||||
</div>
|
|
||||||
<h2 id="limit-title" class="limit-title">That's a wrap for now</h2>
|
|
||||||
<p id="limit-msg" class="limit-msg"></p>
|
|
||||||
<a id="limit-cta" class="btn primary wide limit-cta" href="#"></a>
|
|
||||||
<p id="limit-note" class="limit-note"></p>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<script type="module" src="main.js?v=20260902001"></script>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
@@ -1,45 +0,0 @@
|
|||||||
# Debug Session: mcp-audio-stutter
|
|
||||||
- **Status**: [OPEN]
|
|
||||||
- **Issue**: Piccole interruzioni audio quando una risposta MCP lunga viene eseguita o reiniettata nella conversazione live.
|
|
||||||
|
|
||||||
## Reproduction Steps
|
|
||||||
1. Avvia la UI realtime.
|
|
||||||
2. Esegui una richiesta che attiva un tool MCP con output grande.
|
|
||||||
3. Ascolta la risposta audio dell'assistente.
|
|
||||||
4. Verifica se compaiono micro-interruzioni durante o subito dopo l'iniezione del risultato MCP.
|
|
||||||
|
|
||||||
## Hypotheses & Verification
|
|
||||||
| ID | Hypothesis | Likelihood | Effort | Evidence |
|
|
||||||
|----|------------|------------|--------|----------|
|
|
||||||
| A | Inviare al modello il payload MCP quasi completo degrada la fluidità della risposta audio anche senza bloccare il main thread. | High | Low | Confirmed |
|
|
||||||
| B | Esiste una race: il fallback MCP decide il turno prima che `response.created` arrivi, quindi la prima response speculativa parte comunque e viene tagliata. | High | Medium | Confirmed |
|
|
||||||
| C | La chat/UI fa render pesanti quando arriva il risultato MCP grande. | Medium | Medium | Rejected |
|
|
||||||
| D | I log/debug del payload MCP grande aggravano il problema sul main thread. | Medium | Low | Rejected |
|
|
||||||
| E | L'attacco audio di ogni frase soffre di jitter iniziale perché il playback parte senza prebuffer sul primo chunk disponibile. | High | Medium | Confirmed |
|
|
||||||
|
|
||||||
## Log Evidence
|
|
||||||
- `execRemoteMcpTool` riceve un result JSON di `5738` byte e produce un output testo di `3262` caratteri senza tempi anomali.
|
|
||||||
- `built live prompt from MCP output` mostra `promptLength` ~`3400` e `buildMs: 0`.
|
|
||||||
- `rendered tool result row` e `rendered MCP tool result in chat` mostrano `renderMs: 0-1`.
|
|
||||||
- `sending conversation.item.create payload` mostra `payloadLength` ~`3790` e `stringifyMs: 0`.
|
|
||||||
- Quindi serializzazione, render UI e stringify non spiegano i micro-tagli.
|
|
||||||
- Nel run post-fix `sendUserText` mostra `responseActive: false`, `openResponses: 0`, `createInFlight: false`: non è un classico barge-in su response già attiva.
|
|
||||||
- Il sintomo residuo si manifesta solo sulla prima frase speculativa; questo è coerente con una race prima di `response.created`.
|
|
||||||
- Il browser può decidere il fallback MCP prima che il client abbia un `responseId` da silenziare: per questo ora viene marcata anche la **next response** come sopprimibile.
|
|
||||||
- L'ultimo sintomo riferito dall'utente cambia ancora: le micro-interruzioni si sentono sulle prime parole di ogni frase, non solo su quella speculativa.
|
|
||||||
- Questo è coerente con il worklet `audio-playback` che parte subito sul primo chunk ricevuto, senza accumulare buffer iniziale.
|
|
||||||
|
|
||||||
## Minimal Fix
|
|
||||||
- Accodata la `sendUserText()` del payload MCP quando esiste una response attiva.
|
|
||||||
- Flush del testo differito solo dopo `response.done`, prima del `response.create` già in coda.
|
|
||||||
- Riassunto/troncamento del payload MCP prima di reiniettarlo nella conversazione live.
|
|
||||||
- Ridotto il delay del fallback MCP a `150ms` per match forti e senza argomenti obbligatori.
|
|
||||||
- Soppressione della **next speculative response** se il fallback MCP vince il turno prima che `response.created` sia arrivato.
|
|
||||||
- Introdotto un piccolo prebuffer di avvio (`90ms`) nel worklet `audio-playback` prima di iniziare ogni frase.
|
|
||||||
- Obiettivo: evitare sia il barge-in quando presente, sia il carico eccessivo sul modello con JSON troppo grande.
|
|
||||||
- Obiettivo aggiuntivo: ridurre la finestra in cui il modello può iniziare una frase speculativa prima che parta il tool MCP.
|
|
||||||
- Obiettivo finale: evitare il clipping/jitter sulle prime parole di ogni frase.
|
|
||||||
|
|
||||||
## Verification Conclusion
|
|
||||||
- Pre-fix: il browser non mostrava alcun collo di bottiglia locale; il prompt inviato al modello restava però molto grande.
|
|
||||||
- Post-fix: il testo MCP viene accodato se c'è una risposta attiva, il modello riceve solo un summary compatto, il fallback parte quasi subito, la next response speculativa viene silenziata se necessario e il playback aspetta un buffer iniziale prima di iniziare ogni frase.
|
|
||||||
@@ -1,75 +0,0 @@
|
|||||||
# Debug Session: mcp-tool-not-called
|
|
||||||
|
|
||||||
Status: [OPEN]
|
|
||||||
|
|
||||||
## Symptom
|
|
||||||
- The voice assistant says it will call an MCP tool, but no MCP tool call is observed.
|
|
||||||
- No useful runtime log or call attempt is visible from the current instrumentation.
|
|
||||||
|
|
||||||
## Expected
|
|
||||||
- When the model decides to use an MCP tool, the client should emit a tool-call event, execute the MCP proxy request, and produce visible debug evidence for each stage.
|
|
||||||
|
|
||||||
## Hypotheses
|
|
||||||
1. The realtime model is not actually emitting `response.function_call_arguments.done`, so the spoken promise to use a tool is plain text, not a real function call.
|
|
||||||
2. The MCP tools are not present in the effective session state at the moment of the request, so the model cannot legally call them.
|
|
||||||
3. The session instructions are updated too late or with the wrong content, so the model receives guidance text but not usable tool declarations.
|
|
||||||
4. The browser receives a tool-call event, but the handler path fails before logging or before `execRemoteMcpTool()` starts.
|
|
||||||
5. The MCP proxy request starts, but the current logs do not surface pre-call or failure details, making the attempt invisible.
|
|
||||||
|
|
||||||
## Evidence Plan
|
|
||||||
- Instrument session bootstrap, tool declaration updates, transcript-based MCP ranking, tool-call event handling, and MCP proxy fetch lifecycle.
|
|
||||||
- Reproduce with a prompt that should match one MCP tool by description.
|
|
||||||
- Compare whether the model emits only assistant transcript text or a real tool-call event.
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
- First code change in this session must remain instrumentation only.
|
|
||||||
|
|
||||||
## Evidence Summary
|
|
||||||
- `api/config` loads correctly with `searchMcpEnabled=true`, `agentToolCount=16`, and a non-empty primary prompt.
|
|
||||||
- Transcript-time MCP ranking is active and produces ranked candidates such as `get_api_info`, `get_report_detail`, and `get_public_day_report` for relevant spoken requests.
|
|
||||||
- No `response.function_call_arguments.done` event was recorded in the current reproduction.
|
|
||||||
- No `toolcall` event, no `runTool()` start, and no MCP proxy fetch were recorded in the current reproduction.
|
|
||||||
|
|
||||||
## Hypothesis Status
|
|
||||||
| ID | Hypothesis | Status | Evidence |
|
|
||||||
|----|------------|--------|----------|
|
|
||||||
| A | The model speaks about tool use but does not emit a real function call | CONFIRMED | The log file contains transcript/ranking events only; no `function_call_arguments.done` entry was produced in the reproduced turn. |
|
|
||||||
| B | MCP tools are missing from runtime state | PARTIAL | `api/config` proves the remote profile exposes 16 tools, but the current run lacks direct evidence from live `session.update`. |
|
|
||||||
| C | MCP guidance reaches the session too late for the same voice turn | LIKELY | Ranking is computed from the live user transcript; current code only pushes updated instructions on the final transcript, which can miss the active turn. |
|
|
||||||
| D | The browser gets the toolcall event but fails before execution | REJECTED (current run) | There is no `toolcall` event at all in the current reproduction. |
|
|
||||||
| E | The MCP proxy request starts but is invisible | REJECTED (current run) | No pre-fetch or post-fetch MCP proxy log was emitted. |
|
|
||||||
|
|
||||||
## Minimal Fix
|
|
||||||
- Push the computed MCP-guidance instructions to the live session as soon as ranked tool suggestions appear on partial user transcripts, instead of waiting only for the final transcript.
|
|
||||||
- Track the last live instruction snapshot to avoid redundant `session.update` spam.
|
|
||||||
- Add client-construction instrumentation so the next run proves which tools/instructions are present at session start.
|
|
||||||
|
|
||||||
## Post-Fix Evidence
|
|
||||||
- The live session is created with 17 tools and non-empty instructions.
|
|
||||||
- During matching turns, the client now narrows the tool set down to the strongest MCP candidates and sends `tool_choice="required"`.
|
|
||||||
- Example evidence from the latest run:
|
|
||||||
- `get_api_info` is selected as the top candidate with strong score (`29`).
|
|
||||||
- The client sends `setTools(..., "required")` with `["get_api_info", ...]`.
|
|
||||||
- The client sends updated instructions immediately after.
|
|
||||||
- Even with `tool_choice="required"` and a narrowed tool list, no `response.function_call_arguments.done` event is emitted.
|
|
||||||
- Therefore no browser `toolcall`, no `runTool()`, and no MCP proxy request can happen.
|
|
||||||
|
|
||||||
## Updated Hypothesis Status
|
|
||||||
| ID | Hypothesis | Status | Evidence |
|
|
||||||
|----|------------|--------|----------|
|
|
||||||
| A | The model speaks about tool use but does not emit a real function call | CONFIRMED | Even after forcing `tool_choice="required"` and narrowing the tool list, no `function_call_arguments.done` event appears. |
|
|
||||||
| B | MCP tools are missing from runtime state | REJECTED | The client sends 17 initial tools and later sends narrowed MCP-only tool lists, including `tool_choice="required"`. |
|
|
||||||
| C | MCP guidance reaches the session too late for the same voice turn | REJECTED | The client now pushes guidance and tool policy during partial transcripts, before the turn finishes. |
|
|
||||||
| D | The browser gets the toolcall event but fails before execution | REJECTED | No upstream function-call event is emitted at all. |
|
|
||||||
| E | The MCP proxy request starts but is invisible | REJECTED | The MCP proxy is never reached because the model never initiates a tool call. |
|
|
||||||
|
|
||||||
## Current Root Cause
|
|
||||||
- The speech-to-speech realtime model path in the current backend stack is not performing actual function calling for these MCP tools in this conversation flow, even when tools are declared and `tool_choice="required"` is sent.
|
|
||||||
- The spoken assistant text is therefore behaving like a promise in natural language, not a real tool invocation.
|
|
||||||
|
|
||||||
## Backend Branch B
|
|
||||||
- Added relay-side instrumentation in `demo/server.py` to report:
|
|
||||||
- Outbound client text events forwarded to upstream, including `session.update` summaries with `toolCount`, `toolNames`, `toolChoice`, and instruction length.
|
|
||||||
- Inbound upstream text events forwarded back to the client, including `session.updated`, `response.function_call_arguments.done`, and `error` summaries when present.
|
|
||||||
- Binary frame forwarding in both directions and client disconnects.
|
|
||||||
- No new runtime evidence has been collected from this instrumentation yet; a fresh reproduction is required with the patched relay active.
|
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# Documentation
|
||||||
|
|
||||||
|
All maintained documentation is in English. Spanish examples represent actual
|
||||||
|
user input or spoken output. Historical proposals are identified explicitly and
|
||||||
|
do not define the current implementation.
|
||||||
|
|
||||||
|
## Browser client
|
||||||
|
|
||||||
|
- [Usage and configuration](client/usage.md)
|
||||||
|
- [Architecture and source map](client/architecture.md)
|
||||||
|
- [Backend development and compatibility](client/development.md)
|
||||||
|
- [Reasoning modes](client/reasoning.md)
|
||||||
|
- [Context injection](client/context-injection.md)
|
||||||
|
- [Tool calling](client/tool-calling.md)
|
||||||
|
- [Design language](client/design.md)
|
||||||
|
- [Glossary](client/glossary.md)
|
||||||
|
- [Troubleshooting](client/troubleshooting.md)
|
||||||
|
- [Change history](../vocero-api/CHANGELOG.md)
|
||||||
|
|
||||||
|
## Speech backend
|
||||||
|
|
||||||
|
- [Language model backends](speech/llm.md)
|
||||||
|
- [Speech-to-text backends](speech/stt.md)
|
||||||
|
- [Text-to-speech backends](speech/tts.md)
|
||||||
|
- [Realtime API, tools, and interruption handling](speech/realtime-api.md)
|
||||||
|
|
||||||
|
## Decisions, proposals, and examples
|
||||||
|
|
||||||
|
- [Example files and usage](examples/README.md)
|
||||||
|
- [ADR 0001: Docker hosting and search proxy](adr/0001-docker-space-with-search-proxy.md)
|
||||||
|
- [Historical Vocero architecture proposal](proposals/vocero-architecture.html)
|
||||||
|
- [Agent-profile example](examples/agent-profile.json): an illustrative profile
|
||||||
|
with agent instructions and tool definitions; this is not the current
|
||||||
|
`/api/config` response or an automatically loaded runtime configuration.
|
||||||
|
- [Reference audio](examples/reference-audio.wav): an optional voice-cloning
|
||||||
|
example described in the [TTS guide](speech/tts.md).
|
||||||
+6
-2
@@ -1,11 +1,15 @@
|
|||||||
# Convert the static Space into a Docker app with a search proxy
|
# Convert the static Space into a Docker app with a search proxy
|
||||||
|
|
||||||
|
Status: accepted. This records the original hosting decision; the current route
|
||||||
|
is `/api/search`, and the current container also includes the root documentation.
|
||||||
|
See [client usage](../client/usage.md) for the root-context Docker build.
|
||||||
|
|
||||||
To give the model a web search tool, the executor (which runs in the browser of a
|
To give the model a web search tool, the executor (which runs in the browser of a
|
||||||
public Space) needs a search key. A `sdk: static` Space serves files as-is with no
|
public Space) needs a search key. A `sdk: static` Space serves files as-is with no
|
||||||
runtime process, so it cannot hold a secret the browser uses without exposing it.
|
runtime process, so it cannot hold a secret the browser uses without exposing it.
|
||||||
We convert the Space from `sdk: static` to `sdk: docker`: a single container runs a
|
We convert the Space from `sdk: static` to `sdk: docker`: a single container runs a
|
||||||
small server (FastAPI + uvicorn) that both serves the existing front-end *unchanged*
|
small server (FastAPI + uvicorn) that both serves the existing front-end *unchanged*
|
||||||
and exposes a same-origin `/search` proxy holding `SERPER_API_KEY` server-side. The
|
and exposes a same-origin `/api/search` proxy holding `SERPER_API_KEY` server-side. The
|
||||||
whole app lives in that one container; the s2s speech-to-speech backend stays the
|
whole app lives in that one container; the s2s speech-to-speech backend stays the
|
||||||
separate load-balanced service it already is.
|
separate load-balanced service it already is.
|
||||||
|
|
||||||
@@ -23,7 +27,7 @@ separate load-balanced service it already is.
|
|||||||
|
|
||||||
- Deployment is no longer static: there is a Dockerfile and a server process; the
|
- Deployment is no longer static: there is a Dockerfile and a server process; the
|
||||||
README front-matter changes from `sdk: static` to `sdk: docker`.
|
README front-matter changes from `sdk: static` to `sdk: docker`.
|
||||||
- The client calls `/search` same-origin; the server reads the key from env. A user
|
- The client calls `/api/search` same-origin; the server reads the key from env. A user
|
||||||
may still supply their own key as a fallback, sent per-request to the proxy.
|
may still supply their own key as a fallback, sent per-request to the proxy.
|
||||||
- The front-end, audio pipeline, and s2s handshake are untouched — only the hosting
|
- The front-end, audio pipeline, and s2s handshake are untouched — only the hosting
|
||||||
shape and the new route are added.
|
shape and the new route are added.
|
||||||
@@ -0,0 +1,140 @@
|
|||||||
|
# Client architecture
|
||||||
|
|
||||||
|
The [React browser client](../../vocero-frontend) uses the
|
||||||
|
[FastAPI backend](../../vocero-api) for HTTP APIs and a speech WebSocket relay.
|
||||||
|
`start_ui.sh` serves Vite on the existing public port `8420` and FastAPI on
|
||||||
|
loopback port `8421`. Vite forwards backend routes and provides hot reload;
|
||||||
|
Uvicorn reloads Python edits. The launcher supervises both processes.
|
||||||
|
The speech pipeline runs separately.
|
||||||
|
Runtime configuration comes from environment variables and the root `.env`.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
Browser[Browser UI]
|
||||||
|
Vite[Vite on public port 8420]
|
||||||
|
Server[Client FastAPI server]
|
||||||
|
Speech[Realtime speech backend]
|
||||||
|
Reasoning[OpenAI-compatible HTTP reasoning provider]
|
||||||
|
Tools[AquaPortal MCP backend]
|
||||||
|
Browser <-->|Frontend and HMR| Vite
|
||||||
|
Vite -->|HTTP configuration and proxies| Server
|
||||||
|
Vite <-->|Speech WebSocket relay| Server
|
||||||
|
Server <-->|Realtime events and audio| Speech
|
||||||
|
Browser <-->|Optional direct connection| Speech
|
||||||
|
Server -->|Optional transcript reasoning| Reasoning
|
||||||
|
Server -->|Authenticated tool discovery and execution| Tools
|
||||||
|
```
|
||||||
|
|
||||||
|
## Connection modes
|
||||||
|
|
||||||
|
`SPEECH_TO_SPEECH_URL` pins a direct browser connection. Otherwise,
|
||||||
|
`S2S_PROXY_TARGET` exposes the same-origin `/v1/realtime` proxy. Either setting
|
||||||
|
disables load-balancer mode. With neither set, `LOAD_BALANCER_URL` enables the
|
||||||
|
session/queue proxy, or the user can enter a direct URL in settings. See
|
||||||
|
[usage](usage.md) for the configuration matrix.
|
||||||
|
|
||||||
|
The local realtime proxy relays audio and events in both directions. It also
|
||||||
|
patches selected `session.update` instructions, handles authenticated MCP
|
||||||
|
tool execution when enabled, filters internal text, and suppresses MCP tool-marker
|
||||||
|
audio. It is not an unconditional byte-for-byte tunnel.
|
||||||
|
|
||||||
|
## Source map
|
||||||
|
|
||||||
|
| Source | Responsibility |
|
||||||
|
|---|---|
|
||||||
|
| [server.py](../../vocero-api/server.py) | Compatibility entry point for existing `server:app` commands |
|
||||||
|
| [main.py](../../vocero-api/vocero_api/main.py) | FastAPI assembly, documentation serving and redirects |
|
||||||
|
| [api](../../vocero-api/vocero_api/api) | HTTP route registration, handlers and compatible response validation |
|
||||||
|
| [models](../../vocero-api/vocero_api/models) | Pydantic request and response contracts |
|
||||||
|
| [core](../../vocero-api/vocero_api/core) | Existing environment parsing, logging, URL helpers and diagnostics |
|
||||||
|
| [providers](../../vocero-api/vocero_api/providers) | Authenticated MCP transport, backend login, and caller HTTP integrations |
|
||||||
|
| [services](../../vocero-api/vocero_api/services) | Context construction, reasoning and tool-context rendering |
|
||||||
|
| [realtime](../../vocero-api/vocero_api/realtime) | WebSocket relay, events, markers and audio handling |
|
||||||
|
| [start_ui.sh](../../start_ui.sh) | Dependency setup, ports, readiness, reload processes, and shutdown |
|
||||||
|
| [vite.config.ts](../../vocero-frontend/vite.config.ts) | Public development endpoint, HMR, allowed hosts, and backend routing |
|
||||||
|
| [App.tsx](../../vocero-frontend/src/App.tsx) | React layout, voice controls, dialogs, profile and reports sidebar |
|
||||||
|
| [controller.ts](../../vocero-frontend/src/realtime/controller.ts) | Application state, settings, context, queue, tools and external reasoning |
|
||||||
|
| [client.ts](../../vocero-frontend/src/realtime/client.ts) | Realtime protocol, session lifecycle, audio and transcript events |
|
||||||
|
| [components](../../vocero-frontend/src/components) | Chat, caller profile, settings and voice components |
|
||||||
|
| [audio](../../vocero-frontend/src/audio) | TypeScript capture/playback worklets, PCM codec, and visualizer |
|
||||||
|
| [theme.css](../../vocero-frontend/src/styles/theme.css) | Tailwind semantic tokens, configured theme variants and responsive variants |
|
||||||
|
| [ui.tsx](../../vocero-frontend/src/components/ui.tsx) | Shared Tailwind buttons, fields and dialog controls; layout utilities live with components |
|
||||||
|
| [styles](../../vocero-frontend/src/styles) | Minimal base defaults, reduced motion, audio artwork, sanitized Markdown, decorative surfaces and native range controls |
|
||||||
|
|
||||||
|
## Server routes
|
||||||
|
|
||||||
|
| Route | Purpose |
|
||||||
|
|---|---|
|
||||||
|
| `GET /api/config` | Connection settings, primary prompt, reasoning/theme/start modes, and feature flags |
|
||||||
|
| `WS /v1/realtime` | Local realtime proxy to `S2S_PROXY_TARGET` |
|
||||||
|
| `POST /api/reasoning` | OpenAI-compatible HTTP reasoning; disabled in `PROXY` mode |
|
||||||
|
| `POST /api/conversations` | MCP-mode phone login and browser-owned conversation creation |
|
||||||
|
| `DELETE /api/conversations/current` | Release the selected MCP conversation |
|
||||||
|
| `POST /api/conversations/current/interrupt` | Cancel its matching HTTP reasoning turn |
|
||||||
|
| `GET /api/mcp/tools` | Discover the selected conversation's authenticated MCP catalog |
|
||||||
|
| `GET /api/user/phone` | Configured caller profile lookup |
|
||||||
|
| `GET /api/user/reports` | Configured scheduled report lookup, subject to report-injection controls |
|
||||||
|
| `POST /api/search` | Serper web search proxy |
|
||||||
|
| `/api/session`, `/api/queue/*` | Load-balancer session and waiting-queue proxies |
|
||||||
|
|
||||||
|
The public root serves the React UI. `/__vite_hmr` carries Vite hot updates;
|
||||||
|
`/v1/realtime` remains the speech connection. `/docs` remains the FastAPI API
|
||||||
|
explorer, while project documentation files are served in the `/docs/`
|
||||||
|
subdirectories. Markdown remains
|
||||||
|
source text, and directories without `index.html` are not browsable pages.
|
||||||
|
FastAPI serves backend and documentation routes only. The frontend source and
|
||||||
|
assets live in `vocero-frontend` and are served by Vite.
|
||||||
|
|
||||||
|
The uv project and its Python tools are scoped to `vocero-api/`. Response models
|
||||||
|
describe service-owned structure and retain loose upstream values. See
|
||||||
|
[development and compatibility](development.md) before tightening a contract.
|
||||||
|
|
||||||
|
## Audio path
|
||||||
|
|
||||||
|
1. The browser captures microphone audio with echo cancellation, noise
|
||||||
|
suppression and automatic gain-control settings.
|
||||||
|
2. [mic-capture.worklet.ts](../../vocero-frontend/src/audio/mic-capture.worklet.ts) resamples to 16 kHz
|
||||||
|
mono PCM16. The browser sends base64 `input_audio_buffer.append` frames.
|
||||||
|
3. The backend processes VAD → STT → LLM → TTS and emits realtime events.
|
||||||
|
4. Assistant PCM16 audio at the current 16 kHz protocol rate is decoded and sent to
|
||||||
|
[audio-playback.worklet.ts](../../vocero-frontend/src/audio/audio-playback.worklet.ts), which buffers,
|
||||||
|
resamples and fades playback edges. Input speech can clear playback for barge-in.
|
||||||
|
The client flushes short final chunks on response completion and remains in
|
||||||
|
the speaking state until the worklet reports that playback has drained.
|
||||||
|
Sequence numbers reject drain notifications from superseded playback.
|
||||||
|
|
||||||
|
Microphone meter subscribers are separate from application-state subscribers,
|
||||||
|
so level updates do not rerender the main React tree.
|
||||||
|
|
||||||
|
Audio worklets are loaded through `AudioWorklet.addModule`; keep their processor
|
||||||
|
registration structure intact when editing them. The speech backend's thread
|
||||||
|
and cancellation model is documented in [realtime API](../speech/realtime-api.md).
|
||||||
|
|
||||||
|
## Reasoning and context
|
||||||
|
|
||||||
|
The browser triggers external reasoning on completed user transcripts only in
|
||||||
|
`OPENAI` mode. A non-empty answer replaces the native response through the
|
||||||
|
external-answer path. These HTTP operations supplement the realtime audio path.
|
||||||
|
New user speech and session teardown cancel pending reasoning. Turn identity
|
||||||
|
checks reject stale HTTP results and speech callbacks even if cancellation is late.
|
||||||
|
|
||||||
|
With MCP enabled, `services/conversations.py` owns bounded server-side caller
|
||||||
|
credentials. `providers/mcp.py` owns the official SDK transport, while
|
||||||
|
`services/mcp_tools.py` validates selections and enforces per-turn limits.
|
||||||
|
`services/content_context.py` resolves linked metadata and implements the local
|
||||||
|
`select_content` action. Each conversation retains separate original-link and
|
||||||
|
current-selection references; reading or comparing reports does not change the
|
||||||
|
selection. The current podcast is passed as an MCP request header, with every
|
||||||
|
retrieval still subject to backend authorization. TALKFIRST uses the verified
|
||||||
|
title in the opening instructions after link resolution completes.
|
||||||
|
`services/mcp_reasoning.py` and `realtime/mcp_proxy.py` share that executor for
|
||||||
|
HTTP and voice reasoning respectively. The browser stores only its conversation
|
||||||
|
ID in memory and uses an HttpOnly owner cookie. Legacy public content preloads
|
||||||
|
are disabled in this mode; the AquaPortal backend remains the authority for
|
||||||
|
content access. The speech pipeline and its audio protocol are unchanged.
|
||||||
|
In OPENAI mode, MCP TALKFIRST greetings enter the HTTP reasoning path after the
|
||||||
|
speech connection is ready, and use the existing external-answer playback and
|
||||||
|
cancellation lifecycle.
|
||||||
|
|
||||||
|
Read [reasoning](reasoning.md), [context injection](context-injection.md), and
|
||||||
|
[tool calling](tool-calling.md) for the conditions and limitations of each path.
|
||||||
@@ -0,0 +1,145 @@
|
|||||||
|
# Context injection
|
||||||
|
|
||||||
|
Context reaches the assistant through browser-built instructions, server-patched
|
||||||
|
realtime session instructions, and HTTP reasoning context. The implementation is
|
||||||
|
in the server's [context services](../../vocero-api/vocero_api/services/context.py),
|
||||||
|
[realtime proxy](../../vocero-api/vocero_api/realtime/proxy.py),
|
||||||
|
[context.ts](../../vocero-frontend/src/lib/context.ts), and the
|
||||||
|
[WebSocket client](../../vocero-frontend/src/realtime/client.ts).
|
||||||
|
|
||||||
|
## Authenticated MCP context
|
||||||
|
|
||||||
|
When `S2S_MCP_URL` is configured, the browser submits the page's `report_id` and
|
||||||
|
`episode_id` during conversation creation. The server resolves them through the
|
||||||
|
authenticated MCP client and keeps compact verified references separately from
|
||||||
|
the current conversational selection. No legacy profile or episode preload is
|
||||||
|
used. TALKFIRST can mention the verified title after this preparation finishes.
|
||||||
|
The link never restricts searches or access to other permitted content.
|
||||||
|
|
||||||
|
See [tool calling](tool-calling.md#authenticated-aquaportal-mcp) for selection,
|
||||||
|
authorization, timeout, and missing-content behavior. The controls and public
|
||||||
|
episode fetch below describe the legacy path used when MCP is disabled.
|
||||||
|
|
||||||
|
## Independent controls
|
||||||
|
|
||||||
|
`S2S_INJECT_PROFILE` defaults to false and gates server loading of caller,
|
||||||
|
episode, report and prior-conversation context. It also controls whether the
|
||||||
|
browser appends caller profile text to its realtime instructions.
|
||||||
|
|
||||||
|
It is not a master switch for every context source. Operator-supplied RAG text
|
||||||
|
and authenticated MCP orchestration work independently. `S2S_INJECT_TODAY`
|
||||||
|
defaults to true; date and voice-output guards are built on paths that invoke
|
||||||
|
instruction augmentation or HTTP context assembly. A WebSocket session with no
|
||||||
|
profile/episode/report/summary block skips non-MCP proxy augmentation.
|
||||||
|
Direct browser connections bypass server-side WebSocket injection entirely.
|
||||||
|
|
||||||
|
| Setting | Default | Effect |
|
||||||
|
|---|---|---|
|
||||||
|
| `S2S_INJECT_PROFILE` | false | Enable server-resolved caller, episode and history context |
|
||||||
|
| `S2S_INJECT_REPORTS` | false | Allow scheduled-report fetch/injection; also requires the profile gate |
|
||||||
|
| `S2S_REPORTS_INJECT_MODE` | `minimal` | `minimal`: labels/dates; `full`: scripts; `off`/`none`: no report prompt block |
|
||||||
|
| `S2S_REPORTS_QUICK_HINT_MAX` | 5 | Keep at most this many newest reports, clamped to 1–50 |
|
||||||
|
| `S2S_INJECT_TODAY` | true | Add the current server-local date guard when its context builder runs |
|
||||||
|
| `S2S_DEBUG_PROMPT` | false | Log complete prompt/context text instead of only summaries |
|
||||||
|
|
||||||
|
## Context sources
|
||||||
|
|
||||||
|
| Block | Trigger and source | Size/behavior |
|
||||||
|
|---|---|---|
|
||||||
|
| Podcast episode | `episode_id` page/query/body value and profile gate | Up to 24,000 characters of serialized episode content |
|
||||||
|
| Caller profile | `phoneNumber` and `VOCERO_USER_PHONE_AUTH_URL` | Up to 1,600 serialized characters |
|
||||||
|
| Scheduled reports | `S2S_REPORT_LIST`, profile gate and report gate | Newest N labels/dates by default; full mode includes scripts with a 12,000-character block limit |
|
||||||
|
| Latest conversation | `S2S_LATEST_SUMMARY_TEXT`, or `S2S_LATEST_SUMMARY` with a phone number | Up to 1,600 characters; static text overrides HTTP fetch |
|
||||||
|
| Custom RAG | Operator applies text through the RAG dialog | Up to 12,000 characters; appended in browser instructions and HTTP `user_context` |
|
||||||
|
|
||||||
|
### Episode
|
||||||
|
|
||||||
|
The current episode fetch uses the deployment-specific URL:
|
||||||
|
|
||||||
|
```text
|
||||||
|
https://aquachile.azurianlab.com/api/v1/podcast/episodes/{episode_id}
|
||||||
|
```
|
||||||
|
|
||||||
|
It follows redirects with a 15-second timeout, accepts JSON or plain text,
|
||||||
|
unwraps a `result` field, and includes arbitrary episode fields. The prompt
|
||||||
|
labels it as the primary reference for the user's current episode. The endpoint
|
||||||
|
is hardcoded in the fetch helper rather than selected through an environment
|
||||||
|
variable.
|
||||||
|
|
||||||
|
### Caller profile and login
|
||||||
|
|
||||||
|
For caller-specific context, the proxy can first post
|
||||||
|
`{"phoneNumber":"+56912345678"}` to `S2S_LOGIN_BY_PHONE`, extract a bearer token,
|
||||||
|
and forward it to subsequent profile/history requests. Profile retrieval uses
|
||||||
|
`GET VOCERO_USER_PHONE_AUTH_URL?phoneNumber=...` with a 10-second timeout and
|
||||||
|
accepts an object, optionally wrapped in `result`.
|
||||||
|
|
||||||
|
Encode the leading plus sign in page links, for example
|
||||||
|
`?phoneNumber=%2B56912345678`. Caller information is also fetched through
|
||||||
|
`/api/user/phone` for the UI. An available profile can update live browser
|
||||||
|
instructions when profile injection is enabled.
|
||||||
|
|
||||||
|
### Scheduled reports
|
||||||
|
|
||||||
|
`S2S_INJECT_REPORTS=false` skips report fetching and prompt content, including
|
||||||
|
the table of contents. When enabled, the report endpoint may be called with or
|
||||||
|
without a phone number. The resolver keeps dictionary entries, reads
|
||||||
|
`sourceScheduleLabel` or `sourceSchedule.label`, reads `date` or
|
||||||
|
`sourceSchedule.date`, and optionally keeps `podcastScript` in full mode.
|
||||||
|
|
||||||
|
Entries are sorted newest first using ISO/date/timestamp handling and an
|
||||||
|
ISO-like string fallback. Empty/unparseable dates sort last. The newest-item
|
||||||
|
limit applies before constructing the prompt block. `minimal` exposes only the
|
||||||
|
labels and dates; `full` explicitly opts into script content.
|
||||||
|
|
||||||
|
### Latest conversation summary
|
||||||
|
|
||||||
|
`S2S_LATEST_SUMMARY_TEXT` takes priority over `S2S_LATEST_SUMMARY` and needs no
|
||||||
|
phone number or network request. Literal `\n` sequences in the environment value
|
||||||
|
become newlines. Otherwise the proxy fetches
|
||||||
|
`S2S_LATEST_SUMMARY?phoneNumber=...` and accepts text or JSON with an optional
|
||||||
|
`result` wrapper. Both sources pass through the same cap and history wrapper.
|
||||||
|
|
||||||
|
### Custom RAG
|
||||||
|
|
||||||
|
The operator can paste reference material and explicitly apply it to the
|
||||||
|
session. The browser appends it to instructions even with profile injection
|
||||||
|
disabled, includes it in HTTP `user_context`, and updates an open realtime
|
||||||
|
session. This context does not require an external retrieval endpoint.
|
||||||
|
|
||||||
|
## Application timing and order
|
||||||
|
|
||||||
|
At local WebSocket connect, the server resolves enabled context and stores it in
|
||||||
|
`session_ctx`. Each client `session.update` then passes through
|
||||||
|
`_patch_client_session_update_with_profile()`. Other client frame types are not
|
||||||
|
modified by this helper. The non-MCP relay retains its general assistant-text
|
||||||
|
filtering and forwards audio without a tool-execution pass.
|
||||||
|
|
||||||
|
The current augmentation function **keeps base instructions first**, then
|
||||||
|
appends surviving voice-output guards, date context, episode, profile, reports,
|
||||||
|
and summary blocks. If a block type is already present in the base, deduplication
|
||||||
|
may omit its server candidate.
|
||||||
|
This describes string assembly, not a guarantee of model attention or priority.
|
||||||
|
|
||||||
|
Every HTTP reasoning request rebuilds enabled context. Its `user_context`
|
||||||
|
assembly orders voice guard, date, episode, profile, reports, summary,
|
||||||
|
and browser-provided context, then deduplicates recognized types and strips
|
||||||
|
internal markers. Server instruction augmentation remains separate, so avoid
|
||||||
|
assuming the WebSocket and HTTP prompts have identical ordering.
|
||||||
|
|
||||||
|
Context markers support internal deduplication and diagnostics; ordinary
|
||||||
|
profile/report/episode/history sentinels are stripped before final augmentation.
|
||||||
|
MCP uses its separate conversation context and call-marker handling described
|
||||||
|
above. Treat the source helpers as the reference for the exact final payload.
|
||||||
|
|
||||||
|
## Failure behavior and inspection
|
||||||
|
|
||||||
|
Optional context-fetch errors are logged and the affected context is skipped.
|
||||||
|
The speech conversation normally continues without that reference material.
|
||||||
|
This does not mean all API calls ignore errors: missing reasoning configuration,
|
||||||
|
invalid tool arguments and tool authorization can return explicit errors.
|
||||||
|
|
||||||
|
Inspect `/api/config` for active gates, the actual outbound `session.update` in
|
||||||
|
browser DevTools, and `[INJECT-*]`/`[DEBUG-PROMPT]` server logs. Full prompt
|
||||||
|
logging includes caller data and conversation text. See
|
||||||
|
[troubleshooting](troubleshooting.md) for a step-by-step diagnostic flow.
|
||||||
@@ -0,0 +1,109 @@
|
|||||||
|
# Design language
|
||||||
|
|
||||||
|
This guide describes the current browser UI. Use
|
||||||
|
[theme.css](../../vocero-frontend/src/styles/theme.css) as the Tailwind token
|
||||||
|
reference and [App.tsx](../../vocero-frontend/src/App.tsx) for the theme-specific structure.
|
||||||
|
Earlier documentation described a monochrome-only dark UI; the current design
|
||||||
|
also uses aurora gradients, glass surfaces, and a separate light theme.
|
||||||
|
|
||||||
|
## Themes and tokens
|
||||||
|
|
||||||
|
`S2S_THEME=white` selects the white/blue interface. Empty or `dark` selects the
|
||||||
|
dark aurora interface. The browser sets `html[data-theme]` after configuration
|
||||||
|
loads; React renders a loading screen until configuration resolves. Theme values
|
||||||
|
live under `:root` and `html[data-theme="white"]`, with `@theme inline` exposing
|
||||||
|
semantic utilities. The `white:` and `dark:` variants follow this attribute,
|
||||||
|
independently of the browser's preferred color scheme.
|
||||||
|
|
||||||
|
| Token group | Dark theme values/purpose |
|
||||||
|
|---|---|
|
||||||
|
| Canvas | `--bg: #07080d`; translucent `--bg-elev` and `--bg-elev-2` surfaces |
|
||||||
|
| Glass | `--glass` and `--glass-strong` for translucent floating elements |
|
||||||
|
| Text | `--text`, `--text-dim`, `--text-faint` |
|
||||||
|
| Borders | `--border` and `--border-strong` hairlines |
|
||||||
|
| Brand | `--brand: #8b5cf6`, `--brand-2: #ec4899`, `--brand-3: #22d3ee` |
|
||||||
|
| Radius | `rounded-control: 10px`, `rounded-panel: 16px`, `rounded-card: 26px` (22px in white) |
|
||||||
|
| Shadows | `--shadow-soft`, `--shadow-glow-brand`, `--shadow-glass` |
|
||||||
|
|
||||||
|
Use semantic utilities such as `bg-surface`, `bg-surface-raised`, `text-foreground`,
|
||||||
|
`text-muted`, `border-line`, `rounded-control` and `shadow-panel`. Palette values
|
||||||
|
are runtime CSS variables; fonts are declared in `@theme`. Keep audio levels and
|
||||||
|
waveform bands outside the theme token registry.
|
||||||
|
|
||||||
|
Prefer existing variables and `color-mix` over new unrelated colors. The dark
|
||||||
|
theme's orb is the strongest visual focus, while gradients and restrained role
|
||||||
|
tints also appear in the surrounding interface. The white theme changes
|
||||||
|
surfaces, typography contrast, controls and chat presentation through its own
|
||||||
|
tokens and explicit component variants. The two voice presentations retain
|
||||||
|
their separate React markup and scoped artwork.
|
||||||
|
|
||||||
|
## Voice state
|
||||||
|
|
||||||
|
| State | Dark-theme color |
|
||||||
|
|---|---|
|
||||||
|
| Listening/user speaking | Cyan (`--listening`) |
|
||||||
|
| Assistant speaking | Violet (`--speaking`) |
|
||||||
|
| Processing | Amber (`--processing`) |
|
||||||
|
| Ready/success | Green (`--success`) |
|
||||||
|
| Error | Red (`--error`) |
|
||||||
|
|
||||||
|
`--voice-user`, `--voice-assistant` and `--voice-tool` connect transcript roles
|
||||||
|
to the same state vocabulary. When adding a state, define its existing control,
|
||||||
|
caption and transcript behavior together. Light-theme controls can use their
|
||||||
|
blue palette while retaining recognizable state feedback.
|
||||||
|
|
||||||
|
## Typography and copy
|
||||||
|
|
||||||
|
Use Inter (`--font-sans`) for human prose, buttons, settings and transcript
|
||||||
|
bodies. Geist Mono (`--font-mono`) is reserved for machine status, role labels,
|
||||||
|
tool identifiers and diagnostic JSON. Font stacks have system fallbacks.
|
||||||
|
|
||||||
|
Visible labels, accessibility text, permission instructions and recovery
|
||||||
|
messages default to Spanish. Source identifiers, comments, diagnostic logs and
|
||||||
|
internal model instructions use English. Exact model/voice/protocol identifiers
|
||||||
|
and proper names are preserved. Spanish wake phrases are user-input literals.
|
||||||
|
|
||||||
|
Use sentence case and concrete verbs. Empty states should explain the next
|
||||||
|
action; errors should say what failed and how to recover. Keep internal system
|
||||||
|
details out of ordinary user flows.
|
||||||
|
|
||||||
|
## Layout and components
|
||||||
|
|
||||||
|
The page uses a column layout: header, voice stage, always-visible inline
|
||||||
|
conversation, and footer. The document scrolls naturally while the transcript
|
||||||
|
has its own scroll area. New messages preserve the reader's position until they
|
||||||
|
return to the bottom or use the jump-to-latest action.
|
||||||
|
|
||||||
|
In the dark theme, the reports sidebar is fixed on the left at desktop widths
|
||||||
|
of at least 1024px. On smaller screens it becomes a native dialog drawer with a
|
||||||
|
toggle and backdrop. The white theme omits the sidebar, its toggle, and hint.
|
||||||
|
The user avatar opens a profile dialog with label/value rows. Some controls are
|
||||||
|
intentionally gated by `?debug=true`.
|
||||||
|
|
||||||
|
Reuse [ui.tsx](../../vocero-frontend/src/components/ui.tsx) for buttons, icon buttons,
|
||||||
|
fields and dialog controls. Keep layout utilities with the owning component.
|
||||||
|
Use complete class names in conditional maps; interpolated utility names cannot
|
||||||
|
be detected reliably. `compact:` preserves the existing maximum width of 600px;
|
||||||
|
`short:` covers heights at or below 640px. Other artwork breakpoints remain local.
|
||||||
|
|
||||||
|
The dark theme uses glass bubbles with role tints, borders and restrained glow;
|
||||||
|
the light theme uses its white/blue message and avatar treatment.
|
||||||
|
|
||||||
|
The stylesheet entry point imports Tailwind's theme and utilities without
|
||||||
|
Preflight. Keep resets explicit in `base.css`; enabling Preflight is a separate
|
||||||
|
visual change. Custom CSS is reserved for the page background, audio artwork,
|
||||||
|
sanitized Markdown descendants, decorative masks and native range pseudo-elements.
|
||||||
|
Do not add model-supplied utility classes to the Markdown sanitizer.
|
||||||
|
|
||||||
|
## Motion and accessibility
|
||||||
|
|
||||||
|
Microphone RMS and assistant audio levels drive the voice visualization.
|
||||||
|
Breathing, glow and decorative gradients should remain subordinate to speech
|
||||||
|
feedback. Preserve the reduced-motion rules in `base.css`: they disable both
|
||||||
|
CSS animations and the audio-driven transform variables. Their `!important`
|
||||||
|
declarations intentionally override inline variables written by the audio engine.
|
||||||
|
|
||||||
|
For changes, verify both themes, narrow mobile layouts down to 320px, keyboard
|
||||||
|
focus, dialog dismissal, and readable contrast. Prefer touch targets of at least
|
||||||
|
44px and respect safe-area padding. A CSS guideline is not proof that every
|
||||||
|
existing control already meets the target.
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
# Backend development and compatibility
|
||||||
|
|
||||||
|
The FastAPI project lives entirely in `vocero-api/`. It uses uv 0.12.5, Python
|
||||||
|
3.12, and a committed `uv.lock`. The frontend and speech pipeline keep their
|
||||||
|
own tooling. Runtime dependency versions match the environment used to capture
|
||||||
|
the pre-migration server; framework upgrades require a separate compatibility
|
||||||
|
review.
|
||||||
|
|
||||||
|
The Python distribution is named `vocero-api`, and its import package is
|
||||||
|
`vocero_api`. The browser package and directory are both named `vocero-frontend`.
|
||||||
|
`start_ui.sh` creates the API environment at its current path on the next start;
|
||||||
|
do not move an existing virtual environment because its executables contain
|
||||||
|
absolute paths. Reinstall optional Git hooks after a project-directory rename
|
||||||
|
with `make -C vocero-api hooks-install`.
|
||||||
|
|
||||||
|
Pytest filters only the known `anyio.abc.BlockingPortal` deprecation emitted by
|
||||||
|
Starlette's test client ([upstream issue](https://github.com/Kludex/starlette/issues/3497)).
|
||||||
|
The filter matches the exact message, warning category, and module; other
|
||||||
|
warnings remain visible. Remove it when a compatible Starlette upgrade replaces
|
||||||
|
the deprecated alias. Runtime dependencies and warning handling are unchanged.
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
From the repository root:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make -C vocero-api install # uv sync --locked, including development tools
|
||||||
|
make -C vocero-api format # Ruff safe fixes and formatting
|
||||||
|
make -C vocero-api check # lock freshness, format, lint, mypy, pytest
|
||||||
|
make -C vocero-api test-launcher # Bash supervisor integration fixtures
|
||||||
|
make -C vocero-api test-image # Docker build and isolated runtime smoke test
|
||||||
|
```
|
||||||
|
|
||||||
|
Run `uv lock` inside `vocero-api/` when intentionally changing dependencies.
|
||||||
|
Do not install server packages into a shared root environment. The launcher
|
||||||
|
syncs `vocero-api/.venv` and retains root `.env` loading, port defaults,
|
||||||
|
readiness checks, and process-group shutdown. Offline starts require the
|
||||||
|
locked packages and Python runtime to have been installed or cached already.
|
||||||
|
|
||||||
|
Optional hooks use `make -C vocero-api hooks-install`; the configuration stays
|
||||||
|
in `vocero-api/.pre-commit-config.yaml`. `make -C vocero-api hooks-run` runs the
|
||||||
|
same hooks without installing them. Hooks and Python quality checks apply
|
||||||
|
only to the client service. The server entry points are `vocero_api.main:app`
|
||||||
|
and the retained `server:app` shim; use `--app-dir vocero-api` from the root.
|
||||||
|
|
||||||
|
## Schema boundaries
|
||||||
|
|
||||||
|
`models/requests.py` retains the existing search and reasoning
|
||||||
|
request models, including their defaults, coercion, and ignored extra fields.
|
||||||
|
The queue beacon still accepts malformed JSON and non-object bodies as before;
|
||||||
|
its Pydantic view is applied only after the original object check, and
|
||||||
|
`queueId` remains unconstrained.
|
||||||
|
|
||||||
|
`models/responses.py` validates structure constructed by this server. Fields
|
||||||
|
owned by external services remain `Any` where the old parser imposed no type
|
||||||
|
constraint. Object models allow extra fields. Caller profiles can be arbitrary
|
||||||
|
JSON; load-balancer fields remain optional. Existing provider parsing,
|
||||||
|
authentication precedence, environment parsing, and WebSocket frames are
|
||||||
|
preserved outside explicitly retired integrations. Authenticated conversation
|
||||||
|
and MCP contracts live in `models/mcp.py` and `models/content.py`.
|
||||||
|
|
||||||
|
HTTP handlers call `validated_json_response`: Pydantic validates the payload,
|
||||||
|
then `JSONResponse` serializes the original object. Returning a dumped model
|
||||||
|
would insert defaults or change values/key order, so do not replace this helper
|
||||||
|
with model serialization without a contract review. Route `response_model`
|
||||||
|
declarations publish the new response schemas in OpenAPI. Existing request
|
||||||
|
schemas and validation errors remain unchanged. HTTP exceptions retain their
|
||||||
|
existing error shapes; there is no new common error envelope.
|
||||||
|
|
||||||
|
## Compatibility evidence
|
||||||
|
|
||||||
|
`tests/fixtures/legacy-contracts.json` was captured from the original monolithic
|
||||||
|
server at commit `3d42df5`, before refactoring. Tests compare 66 HTTP scenarios,
|
||||||
|
including exact response text, statuses, selected headers, and outgoing mock
|
||||||
|
HTTP requests. They also compare request OpenAPI definitions and representative
|
||||||
|
realtime frames. Additional tests cover response validation and permissive
|
||||||
|
upstream values; the original local HTTP/WebSocket tests remain in the suite.
|
||||||
|
Four captures belonging exclusively to the retired workflow reasoning service
|
||||||
|
were removed with that integration. ApiPilot retirement removed another 16 HTTP
|
||||||
|
captures, its two OpenAPI operations, and its request model. The four bootstrap
|
||||||
|
captures now omit the retired `apipilot` flag. One reasoning failure case was
|
||||||
|
renamed because removing ApiPilot eliminated its token access; it still exposes
|
||||||
|
the existing uninitialized profile failure. All other retained capture values
|
||||||
|
are unchanged.
|
||||||
|
|
||||||
|
The captures deliberately preserve unknown fields, omitted fields, unusual JSON
|
||||||
|
values, queue beacon edge cases, and known failures. Examples include reasoning's
|
||||||
|
existing uninitialized profile in certain non-MCP configuration combinations
|
||||||
|
and relay termination after a suppressed invalid
|
||||||
|
`session.update` error. Fix those in separately reviewed behavior changes;
|
||||||
|
do not regenerate golden captures to make a refactor pass.
|
||||||
|
|
||||||
|
These checks use local fixtures and do not claim exhaustive coverage of all
|
||||||
|
provider responses, live speech models, or production configuration. No deployed
|
||||||
|
services or real credentials are needed. The Docker check verifies the actual
|
||||||
|
runtime image, API configuration, documentation, OpenAPI, and the backend's
|
||||||
|
existing root 404; it removes its temporary container and image afterward.
|
||||||
|
|
||||||
|
ApiPilot removal deliberately excludes `speech_to_speech/`. The MCP orchestration
|
||||||
|
test still exercises its existing pure marker sanitizer without importing model
|
||||||
|
dependencies, verifying that `VOCERO_MCP_CALL` reaches the Vocero relay unchanged.
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
# Context glossary
|
||||||
|
|
||||||
|
These terms explain the product and its conversation flow. Implementation
|
||||||
|
details belong in [architecture](architecture.md) and the source. Model/provider
|
||||||
|
names shown in a deployment are configuration examples, not definitions.
|
||||||
|
|
||||||
|
| Term | Meaning |
|
||||||
|
|---|---|
|
||||||
|
| Speech-to-speech client | The browser interface used to start a voice conversation, display transcripts, and play the assistant's reply |
|
||||||
|
| Pipeline | The ordered backend path: VAD → STT → LLM → TTS |
|
||||||
|
| VAD | Voice activity detection; determines when a user speech segment starts and stops |
|
||||||
|
| STT | Speech to text; transcribes an audio utterance |
|
||||||
|
| LLM | Language model; generates response text and optional tool calls |
|
||||||
|
| VLM | Vision-language model; a language model with image support when that capability is configured |
|
||||||
|
| TTS | Text to speech; synthesizes response audio with the configured voice/model |
|
||||||
|
| Realtime backend | The service that processes audio and exposes the `/v1/realtime` WebSocket protocol |
|
||||||
|
| Client server/proxy | The local FastAPI service serving the page and optional HTTP/WebSocket bridges |
|
||||||
|
| Builder | A person/team credited with creating the interface or deployment, distinct from a model's authors |
|
||||||
|
| Powered by | Hosting or inference infrastructure credited by the deployment |
|
||||||
|
| Tool | A callable operation used on demand; distinct from the pipeline stages that process ordinary turns |
|
||||||
|
| Native tool declaration | A function name, description and JSON Schema sent in `session.tools` |
|
||||||
|
| Tool executor | Code that receives a tool call, executes it and returns the result to the model |
|
||||||
|
| Web search | The browser-supported native tool whose request goes through the server's `/api/search` proxy |
|
||||||
|
| Search proxy | Same-origin server route holding a deployment search key and calling the external provider |
|
||||||
|
| AquaPortal MCP | Authenticated tool discovery and execution using the current conversation's backend session |
|
||||||
|
| Call marker | Internal structured text such as `<<<VOCERO_MCP_CALL:...>>>` used by the MCP orchestrator; separate from native function-call events |
|
||||||
|
| Tools panel | Controls for available browser tools and search-key configuration |
|
||||||
|
| Settings | Connection, voice, audio and instruction controls persisted in browser storage |
|
||||||
|
| Identity block | The header's product/user identity, separate from configuration dialogs |
|
||||||
|
| About panel | Introduction and project/pipeline credits |
|
||||||
|
| Caller profile | Information looked up using the page's `phoneNumber`, used for identity display and optionally model context |
|
||||||
|
| RAG context | Reference material explicitly applied by the operator for the current session |
|
||||||
|
| Episode context | Content loaded for the page's `episode_id`, treated as the current episode reference |
|
||||||
|
| Queue | Users waiting for an available compute slot; waiting is not yet an active conversation |
|
||||||
|
| Ticket | A held queue position returned by the load balancer |
|
||||||
|
| Position | The displayed place in the queue; distinct from a time estimate |
|
||||||
|
| Claim | The transition from a waiting ticket to an allocated realtime session |
|
||||||
|
| At capacity | A full waiting queue that cannot accept another ticket |
|
||||||
|
| Barge-in | User speech interrupts an active assistant response, coordinating playback clearing and backend cancellation |
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
# Reasoning modes
|
||||||
|
|
||||||
|
Reasoning determines the assistant's response text. The realtime backend always
|
||||||
|
provides the speech channel; optional HTTP reasoning produces a separate answer
|
||||||
|
from a transcript. Set `S2S_REASONING_TYPE` to select the mode.
|
||||||
|
|
||||||
|
| Mode | Response owner | HTTP hook |
|
||||||
|
|---|---|---|
|
||||||
|
| `PROXY` (default) | Realtime backend | Browser does not call `/api/reasoning`; that endpoint returns 409 |
|
||||||
|
| `OPENAI` | OpenAI-compatible Chat Completions response | Server calls `OPENAI_BASE_URL/chat/completions` |
|
||||||
|
|
||||||
|
The mode is normalized case-insensitively. Unknown values resolve to `PROXY`.
|
||||||
|
|
||||||
|
## Shared instruction assembly
|
||||||
|
|
||||||
|
In [context.ts](../../vocero-frontend/src/lib/context.ts), `buildInstructions()` starts
|
||||||
|
with `config.primaryPrompt || settings.instructions`. `/api/config` pins
|
||||||
|
`S2S_PRIMARY_PROMPT` when non-empty; otherwise the settings prompt or
|
||||||
|
`DEFAULT_INSTRUCTIONS` is used. The current server does not fetch a remote agent
|
||||||
|
profile to choose this base prompt.
|
||||||
|
|
||||||
|
The browser adds caller context when `injectProfile` is enabled and custom RAG
|
||||||
|
text when the operator has applied it. In MCP mode, the server constructs tool
|
||||||
|
instructions from the authenticated conversation's catalog and content selection.
|
||||||
|
`speechInstructions()` adds the speech-rendering hint only in `OPENAI` mode.
|
||||||
|
The WebSocket client packages these instructions in `session.update.session`;
|
||||||
|
the local proxy may then add server-resolved context. Details belong in
|
||||||
|
[context injection](context-injection.md).
|
||||||
|
|
||||||
|
## Native realtime reasoning (`PROXY`)
|
||||||
|
|
||||||
|
The realtime backend receives session instructions and enabled native tools,
|
||||||
|
then processes each voice turn through STT, model generation and TTS.
|
||||||
|
Native function-call events are executed by the browser. Authenticated MCP has a
|
||||||
|
separate marker-based path handled by the local proxy. These are distinct
|
||||||
|
protocols, described in [tool calling](tool-calling.md).
|
||||||
|
|
||||||
|
Using a direct browser URL bypasses local WebSocket proxy processing. Browser
|
||||||
|
instructions still apply, but server-side session context and marker execution
|
||||||
|
require the local `/v1/realtime` route. MCP in `PROXY` mode requires this relay;
|
||||||
|
configuration rejects a pinned direct URL or load-balancer bypass.
|
||||||
|
|
||||||
|
## OpenAI-compatible HTTP reasoning
|
||||||
|
|
||||||
|
```env
|
||||||
|
S2S_REASONING_TYPE=OPENAI
|
||||||
|
OPENAI_API_KEY=YOUR_API_KEY
|
||||||
|
OPENAI_BASE_URL=https://api.openai.com/v1
|
||||||
|
OPENAI_REASONING_MODEL=gpt-4.1-mini
|
||||||
|
```
|
||||||
|
|
||||||
|
`OPENAI_ORGANIZATION` and `OPENAI_PROJECT` are optional headers. The model value
|
||||||
|
is a runtime default, not a recommendation about provider availability.
|
||||||
|
|
||||||
|
`VoiceController.reason()` in [controller.ts](../../vocero-frontend/src/realtime/controller.ts)
|
||||||
|
runs after a complete user transcript and posts a body with `transcript`,
|
||||||
|
`instructions`, `language`, `user_context`, `phoneNumber`, and `episode_id`.
|
||||||
|
Without MCP, the server combines instructions and context into model messages,
|
||||||
|
requests a final answer with a 30-second timeout, and returns `answer`.
|
||||||
|
With MCP, the browser also sends conversation and turn headers. The server uses
|
||||||
|
that conversation's backend token, linked/current content, and bounded history
|
||||||
|
to run tool selections before returning `answer`. This path has a 180-second
|
||||||
|
total timeout and configured per-operation limits. Retired ApiPilot execution
|
||||||
|
metadata is no longer returned. The endpoint does not forward the browser's
|
||||||
|
native `session.tools` catalog as OpenAI Chat Completions `tools`.
|
||||||
|
|
||||||
|
For an external answer, the browser cancels/suppresses the active native
|
||||||
|
response and appends the external assistant text. It uses browser
|
||||||
|
`SpeechSynthesis` when available. The React client adds the answer to the realtime
|
||||||
|
conversation as an assistant item; without browser speech synthesis it remains
|
||||||
|
visible as text.
|
||||||
|
|
||||||
|
External reasoning adds an HTTP operation to each completed transcript and has
|
||||||
|
separate provider token usage. It is not the speech pipeline's `responses-api`
|
||||||
|
or `chat-completions` LLM backend setting; those configure the model inside the
|
||||||
|
realtime backend instead.
|
||||||
|
|
||||||
|
## Where to change behavior
|
||||||
|
|
||||||
|
| Goal | Source |
|
||||||
|
|---|---|
|
||||||
|
| Change the base agent prompt | `S2S_PRIMARY_PROMPT` or browser settings when unpinned |
|
||||||
|
| Change browser context assembly | `buildInstructions()` in `vocero-frontend/src/lib/context.ts` |
|
||||||
|
| Change HTTP provider messages | `reasoning()` in `vocero-api/vocero_api/services/reasoning.py` |
|
||||||
|
| Change MCP reasoning or tool context | `vocero-api/vocero_api/services/mcp_reasoning.py` and `vocero-api/vocero_api/services/mcp_tools.py` |
|
||||||
|
| Change external-answer rendering | `VoiceController.reason()` and `RealtimeClient.renderExternalAnswer()` |
|
||||||
|
| Change native model/tool generation | `speech_to_speech/LLM` and realtime handlers |
|
||||||
@@ -0,0 +1,203 @@
|
|||||||
|
# Tool calling
|
||||||
|
|
||||||
|
The application supports native realtime function calling and server-side
|
||||||
|
tool selection from model text. AquaPortal content uses authenticated MCP when
|
||||||
|
`S2S_MCP_URL` is set. Without MCP, the native browser tools remain available.
|
||||||
|
A spoken promise to fetch something is not evidence that a tool executed.
|
||||||
|
|
||||||
|
## Authenticated AquaPortal MCP
|
||||||
|
|
||||||
|
Vocero uses the official MCP Python SDK with Streamable HTTP. Each discovery
|
||||||
|
or execution initializes a client with that conversation's opaque backend token
|
||||||
|
as `Authorization: Bearer ...`. JSON and SSE responses are supported. Neither
|
||||||
|
browser values nor model arguments choose the authenticated user. There is no
|
||||||
|
service-token override, anonymous retry, redirect following, or public REST
|
||||||
|
fallback on this path. AquaPortal checks current permissions on every request,
|
||||||
|
including its administrator and assigned report-manager access rules.
|
||||||
|
|
||||||
|
The backend phone login must return `sessionToken` plus `user.id` and `user.name`.
|
||||||
|
The allowed backend tool names are:
|
||||||
|
|
||||||
|
| Tool | Content |
|
||||||
|
|---|---|
|
||||||
|
| `list_reports` | Accessible published reports with name, center, and date filters |
|
||||||
|
| `get_report` | One report or an explicit list of ambiguous selections |
|
||||||
|
| `list_insights` | Published findings, priorities, recommendations, and sources |
|
||||||
|
| `get_metric_history` | Dated metric values, units, centers, cages, and source reports |
|
||||||
|
| `list_podcasts` | Daily and report-associated episodes |
|
||||||
|
| `get_podcast` | Selected episode summary, highlights, and available media |
|
||||||
|
| `read_podcast_transcript` | Paginated transcript or report-script passages |
|
||||||
|
| `get_my_briefing` | Compact overview with actual source dates |
|
||||||
|
|
||||||
|
Discovery preserves the complete input schema, including nested `request`,
|
||||||
|
optional filters, references, and pagination. The executor validates arguments
|
||||||
|
against that schema and supplies `structuredContent` to reasoning when present,
|
||||||
|
falling back to MCP text content. It preserves `get_report`'s `result` union and
|
||||||
|
list `next_offset` values. Tool errors are data for the model to explain; expired
|
||||||
|
authentication stops the conversation. Tool text is labeled as reference data,
|
||||||
|
never as instructions or authorization.
|
||||||
|
|
||||||
|
Both reasoning paths can discover reports, retrieve details, and follow up with
|
||||||
|
another tool before answering. The HTTP loop retains bounded completed-turn
|
||||||
|
history. The realtime relay injects results into the speech service's existing
|
||||||
|
conversation. Attempts are bounded per user turn (six by default), identical
|
||||||
|
calls within a turn are rejected, and tool results are limited to 64,000
|
||||||
|
characters individually and 96,000 per turn. Each MCP operation has a timeout;
|
||||||
|
HTTP reasoning also has a 180-second total timeout.
|
||||||
|
|
||||||
|
Selection uses an internal text marker, not a new speech protocol event:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<<<VOCERO_MCP_CALL:NAME=get_report,ARGUMENTS={"request":{"report_id":"example-id"}}>>>
|
||||||
|
```
|
||||||
|
|
||||||
|
The distinct marker is necessary because the current speech pipeline removes
|
||||||
|
`APIPILOT_CALL` before emitting assistant text. The relay buffers undecided
|
||||||
|
text/audio, handles transcripts emitted before `response.created`, executes
|
||||||
|
the MCP selection, and requests another response. Normal speech continues using
|
||||||
|
the existing PCM16 protocol. This leaves `speech_to_speech/` unchanged, although
|
||||||
|
the speech service may spend time synthesizing audio that the relay suppresses.
|
||||||
|
Malformed selections fail with a user-facing error rather than permissive parsing.
|
||||||
|
|
||||||
|
New speech, cancellation, disconnect, and expiry invalidate pending work.
|
||||||
|
Generation checks reject late results even if the transport does not stop in
|
||||||
|
time. The browser receives `vocero.tool.started` / `vocero.tool.failed` status
|
||||||
|
events, without backend credentials or raw tool arguments.
|
||||||
|
|
||||||
|
Conversation creation accepts optional `report_id` and `episode_id` selectors.
|
||||||
|
The server resolves them using the authenticated MCP client, retaining only
|
||||||
|
verified IDs, titles, dates, and podcast availability. Report podcasts use
|
||||||
|
`report:<full-report-id>`; daily episodes use `daily:<episode-id>`. When both
|
||||||
|
selectors are present, the explicit daily episode determines the podcast and
|
||||||
|
the report remains a separate reference. A missing daily episode never falls
|
||||||
|
back to the report podcast. Link resolution has a ten-second total budget;
|
||||||
|
unavailable content permits a generic conversation, while invalid authentication
|
||||||
|
still prevents startup. No public episode endpoint is used.
|
||||||
|
|
||||||
|
The original link and current selection are separate, immutable metadata
|
||||||
|
objects owned by each conversation. The model uses the current selection for
|
||||||
|
"this report/episode" and the original link for "the report from the link".
|
||||||
|
`get_podcast` and `read_podcast_transcript` receive the current podcast through
|
||||||
|
`X-Vocero-Podcast-Id`; explicit `podcast_id` arguments override the header.
|
||||||
|
Every retrieval still checks backend permissions. Search and comparison calls
|
||||||
|
receive only their requested filters and do not change the current selection.
|
||||||
|
|
||||||
|
Vocero adds one local tool, `select_content`, to the discovered catalog:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"request":{"kind":"report","id":"report-example"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
`kind` is `report` or `podcast`; podcast selections use the prefixed ID. This
|
||||||
|
action performs an authorized lookup before replacing the current selection.
|
||||||
|
It preserves the original link and changes no backend data. Choosing a report
|
||||||
|
without a podcast or a daily podcast clears the previous, unrelated reference.
|
||||||
|
Failed, ambiguous, or cancelled lookups do not replace the selection. Reading
|
||||||
|
another report for comparison alone never invokes this action. Both reasoning
|
||||||
|
paths refresh their instructions after a successful selection.
|
||||||
|
|
||||||
|
TALKFIRST opening instructions mention the linked report's verified title, or
|
||||||
|
the linked podcast's title when no report is available. They invite questions
|
||||||
|
about this or other accessible content without an unsolicited summary. Missing
|
||||||
|
content gets a brief explanation without exposing an inaccessible title. The
|
||||||
|
opening instruction is consumed after the first completed answer. TALK and
|
||||||
|
manual startup retain their existing greeting behavior.
|
||||||
|
|
||||||
|
See [usage](usage.md#authenticated-aquaportal-mcp) for configuration, conversation
|
||||||
|
endpoints, single-worker constraints, and the retained unsafe phone selector.
|
||||||
|
|
||||||
|
## Native realtime function calls
|
||||||
|
|
||||||
|
The browser sends tool declarations inside the `session` object:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "session.update",
|
||||||
|
"session": {
|
||||||
|
"type": "realtime",
|
||||||
|
"tools": [
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"name": "web_search",
|
||||||
|
"description": "Search the web for information.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"query": {"type": "string"}},
|
||||||
|
"required": ["query"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"tool_choice": "auto"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Keep configuration inside `session`, not at the event root. The backend has a
|
||||||
|
legacy flat-payload compatibility path, but new clients should use the nested
|
||||||
|
shape. The current browser's `activeToolDefs()` declares `web_search` when
|
||||||
|
enabled and a deploy/user search key is available. It does not load the old
|
||||||
|
remote agent-profile MCP catalog into native session tools.
|
||||||
|
|
||||||
|
For a native call, the backend emits:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "response.function_call_arguments.done",
|
||||||
|
"call_id": "call_example",
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\":\"example query\"}"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`arguments` is a JSON-encoded string. The
|
||||||
|
[WebSocket client](../../vocero-frontend/src/realtime/client.ts) validates the event and
|
||||||
|
dispatches `tool`; [controller.ts](../../vocero-frontend/src/realtime/controller.ts)
|
||||||
|
parses the arguments and runs the supported executor.
|
||||||
|
Web search uses `POST /api/search`, keeping the deployment's Serper key on the
|
||||||
|
server. Tool output returns to the model as:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "conversation.item.create",
|
||||||
|
"item": {
|
||||||
|
"type": "function_call_output",
|
||||||
|
"call_id": "call_example",
|
||||||
|
"output": "Tool result text"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The client requests follow-up generation with `response.create` when the
|
||||||
|
result should be spoken. A `function_call_output` item alone does not start
|
||||||
|
generation. See the [realtime API guide](../speech/realtime-api.md) for local
|
||||||
|
model `<code>` extraction and native remote-provider tool calls.
|
||||||
|
|
||||||
|
## Retired documentation assumptions
|
||||||
|
|
||||||
|
ApiPilot discovery/execution endpoints, catalog injection, credential overrides,
|
||||||
|
and two-pass marker handling have been removed from Vocero. Its former HTTP
|
||||||
|
routes return 404 regardless of MCP configuration. The separate speech service
|
||||||
|
still has its existing legacy marker sanitizers; no ApiPilot requests can be
|
||||||
|
made through Vocero. See [usage](usage.md#authenticated-aquaportal-mcp) for
|
||||||
|
deployment configuration cleanup.
|
||||||
|
|
||||||
|
The former `HOWITWORKS.md` described `/api/mcp/tool`, remote
|
||||||
|
`S2S_AGENT_PROFILE` loading, and Spanish/Italian-to-English client ranking.
|
||||||
|
Those APIs/rankers are absent from the current client/server path. The retained
|
||||||
|
agent-profile JSON is an illustrative historical/example shape. Content tool
|
||||||
|
definitions and permissions belong to the AquaPortal MCP backend; Vocero's
|
||||||
|
provider enforces its allowlist. Do not restore obsolete endpoints solely to
|
||||||
|
match older documentation.
|
||||||
|
|
||||||
|
## Verify a tool turn
|
||||||
|
|
||||||
|
1. Confirm `/api/config` reports the intended feature and connection mode.
|
||||||
|
2. For MCP, confirm conversation creation and authenticated `/api/mcp/tools`
|
||||||
|
discovery succeed. Inspect the relay's `session.update` for MCP instructions;
|
||||||
|
these are separate from native `session.tools` declarations.
|
||||||
|
3. For native calls, find `response.function_call_arguments.done`, then the
|
||||||
|
browser `toolcall` and executor request.
|
||||||
|
4. For MCP, find `mcp_tool_completed` for the expected user/tool and any
|
||||||
|
`vocero.tool.started` / `vocero.tool.failed` voice events.
|
||||||
|
5. Confirm the tool result reaches the model and follow-up text/audio reaches
|
||||||
|
the browser. A spoken acknowledgement alone does not complete the flow.
|
||||||
@@ -0,0 +1,147 @@
|
|||||||
|
# Troubleshooting
|
||||||
|
|
||||||
|
Diagnose configuration, protocol events, execution and playback separately.
|
||||||
|
Use [architecture](architecture.md), [tool calling](tool-calling.md) and
|
||||||
|
[context injection](context-injection.md) to identify which path is active.
|
||||||
|
|
||||||
|
## Check configuration and connection first
|
||||||
|
|
||||||
|
Inspect the server configuration without printing secret environment files:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsS http://localhost:8420/api/config | jq
|
||||||
|
```
|
||||||
|
|
||||||
|
Confirm `s2sUrl`, `lb`, `reasoningType`, `injectProfile`, `mcp`, `theme`
|
||||||
|
and `startSessionMode`. A direct WebSocket URL bypasses the local relay's
|
||||||
|
server-side context injection and marker execution. For a local proxy, confirm
|
||||||
|
the server can resolve and reach `S2S_PROXY_TARGET`.
|
||||||
|
|
||||||
|
In browser DevTools, inspect the WebSocket handshake and `session.created`,
|
||||||
|
`session.update` and error events. Check that instructions and native tools are
|
||||||
|
inside `session`. A cached browser module can disagree with current server
|
||||||
|
code; inspect served asset versions and reload with cache disabled when needed.
|
||||||
|
|
||||||
|
## Launcher and hot reload
|
||||||
|
|
||||||
|
`start_ui.sh` serves Vite on `8420` and FastAPI on loopback port `8421` by
|
||||||
|
default. Check both processes in the launcher/PM2 logs. If either exits, the
|
||||||
|
launcher stops the other and returns a failure. Port conflicts are errors;
|
||||||
|
use distinct `UI_PORT` and `S2S_BACKEND_PORT` values for additional worktrees.
|
||||||
|
|
||||||
|
A Vite "Blocked request" response means the public hostname is missing from
|
||||||
|
`S2S_DEV_ALLOWED_HOSTS`. Configure it without a scheme or port and restart the
|
||||||
|
launcher. If the page loads but edits do not appear, check the browser's
|
||||||
|
`/__vite_hmr` WebSocket and the reverse proxy's upgrade handling. Speech uses
|
||||||
|
the separate `/v1/realtime` WebSocket on the same public origin.
|
||||||
|
|
||||||
|
For a startup dependency error, check uv 0.12.5 and Node/npm on the process
|
||||||
|
manager's PATH, plus `setsid`/`curl`. uv manages Python 3.12 in
|
||||||
|
`vocero-api/.venv` and checks the committed lock on every start. Frontend
|
||||||
|
dependencies are reused until their manifests change. Changes to root
|
||||||
|
`.env` or launcher settings require a restart.
|
||||||
|
|
||||||
|
## The assistant promises a tool but nothing happens
|
||||||
|
|
||||||
|
Follow the actual execution mechanism:
|
||||||
|
|
||||||
|
1. For a native tool, confirm it appears in the outbound `session.tools` array.
|
||||||
|
2. Find `response.function_call_arguments.done` with a tool name, `call_id`, and
|
||||||
|
JSON-string arguments.
|
||||||
|
3. Confirm the browser dispatched `toolcall` and the executor sent its request.
|
||||||
|
4. Confirm `function_call_output` and any required `response.create` were sent.
|
||||||
|
5. For MCP, inspect conversation creation, `/api/mcp/tools` discovery,
|
||||||
|
`mcp_tool_completed` logs, and the returned tool data/follow-up response.
|
||||||
|
Verify the conversation belongs to the browser and backend authorization
|
||||||
|
succeeds; do not retry with shared credentials or legacy content endpoints.
|
||||||
|
|
||||||
|
A spoken sentence such as “Voy a buscar el reporte” does not satisfy step 2.
|
||||||
|
If the function-call event is absent, inspect model/backend tool generation
|
||||||
|
before debugging a browser executor that has not been invoked.
|
||||||
|
|
||||||
|
Earlier local debug notes recorded a reproduction where tools and required
|
||||||
|
tool policy were sent but the model produced only spoken text. Those notes
|
||||||
|
refer to the retired MCP ranking path; they do not establish a current AquaPortal MCP
|
||||||
|
failure. Both local investigation notes remain marked `OPEN`. Their useful
|
||||||
|
lesson is to collect evidence at each boundary and reproduce against current
|
||||||
|
code before claiming a root cause is resolved.
|
||||||
|
|
||||||
|
## The assistant misses caller or report context
|
||||||
|
|
||||||
|
With MCP enabled, check that `/api/conversations` receives `report_id` or
|
||||||
|
`episode_id` from the page and completes before voice startup. Report IDs retain
|
||||||
|
their full prefix: `report-example` selects podcast `report:report-example`.
|
||||||
|
Link resolution uses the same user's backend session and has a ten-second
|
||||||
|
budget. An inaccessible or unavailable link produces a generic conversation,
|
||||||
|
not a fallback to the latest report. Login or catalog failures still block
|
||||||
|
startup. A report may exist without a podcast. To change the conversational
|
||||||
|
subject, the agent uses `select_content`; search and comparison calls do not
|
||||||
|
change it. MCP ignores the legacy injection flags below.
|
||||||
|
|
||||||
|
- Verify the page has an encoded `phoneNumber` or the intended `episode_id`.
|
||||||
|
- Inspect `/api/user/phone` and context-fetch logs for HTTP/JSON failures.
|
||||||
|
- Confirm profile injection is enabled; a visible profile alone does not prove
|
||||||
|
realtime instructions contain it.
|
||||||
|
- Reports need `S2S_INJECT_REPORTS=true`; `minimal` sends only labels and dates.
|
||||||
|
A historical script will not be in the prompt in that mode.
|
||||||
|
- Static summary text overrides the HTTP summary endpoint.
|
||||||
|
- Inspect the final `session.update` rather than only its browser base prompt.
|
||||||
|
The proxy can append and deduplicate context after the browser sends it.
|
||||||
|
|
||||||
|
`S2S_DEBUG_PROMPT=true` enables complete prompt logs, including caller information
|
||||||
|
and conversation text. Use summaries when full payloads are unnecessary. The
|
||||||
|
browser's application functions are module-scoped; do not assume names such as
|
||||||
|
`buildInstructions()` are available directly on `window` in DevTools.
|
||||||
|
|
||||||
|
## Short gaps or clipped words around tool results
|
||||||
|
|
||||||
|
The retained audio investigation found very low local serialization/render
|
||||||
|
times for one large tool result, while still observing audio interruptions.
|
||||||
|
That evidence ruled out a UI-render bottleneck for that reproduction; it does
|
||||||
|
not rule out rendering costs for all future payloads.
|
||||||
|
|
||||||
|
Useful checks are:
|
||||||
|
|
||||||
|
- Measure tool-result size, prompt length, serialization time and UI render time.
|
||||||
|
- Inspect whether result injection overlaps an active or pending response.
|
||||||
|
- Check whether a speculative response starts before its `response.created`
|
||||||
|
reaches the browser. Cancellation based only on an existing response ID can
|
||||||
|
miss that window.
|
||||||
|
- Inspect deferred text/response queues and whether they flush after
|
||||||
|
`response.done` rather than interrupting the active turn.
|
||||||
|
- Inspect playback prebuffering and queue underflow, especially when the first
|
||||||
|
words of every sentence are affected.
|
||||||
|
- Confirm oversized tool results are summarized or capped on the relevant
|
||||||
|
path rather than assuming every tool payload gets the same treatment.
|
||||||
|
|
||||||
|
The historical investigation introduced a 90ms playback prebuffer and reduced
|
||||||
|
MCP prompt reinjection. The old 150ms MCP fallback timer is historical, not a
|
||||||
|
current MCP configuration knob. Verify the live worklet and current relay
|
||||||
|
before changing audio timing based on those observations.
|
||||||
|
|
||||||
|
## External reasoning and startup
|
||||||
|
|
||||||
|
`/api/reasoning` returning 409 is expected in `PROXY` mode.
|
||||||
|
In `OPENAI`, check the configured endpoint/model/key and provider logs; HTTP
|
||||||
|
reasoning does not use the native session's tool array.
|
||||||
|
|
||||||
|
`TALK`/`TALKFIRST` attempt startup as soon as configuration and caller context
|
||||||
|
are ready. Browser microphone/audio policies can still require a gesture. If
|
||||||
|
audio is blocked, tap the main microphone button when the inline hint appears.
|
||||||
|
If microphone access fails, allow it in the browser's site permissions and use
|
||||||
|
the same button to retry. No startup dialog is shown.
|
||||||
|
|
||||||
|
## Local validation
|
||||||
|
|
||||||
|
From the repository root, run the frontend checks and local backend fixtures:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm --prefix vocero-frontend run check
|
||||||
|
npm --prefix vocero-frontend run build
|
||||||
|
make -C vocero-api check
|
||||||
|
```
|
||||||
|
|
||||||
|
These checks cover frontend contracts and local session/queue/WebSocket
|
||||||
|
fixtures. They do not validate deployed services, real microphone hardware,
|
||||||
|
model output, or audio quality. Use a local browser/backend reproduction for
|
||||||
|
those paths before deployment.
|
||||||
@@ -0,0 +1,296 @@
|
|||||||
|
# Client usage and configuration
|
||||||
|
|
||||||
|
The browser voice client connects to an existing OpenAI Realtime-compatible
|
||||||
|
speech backend. The UI server can also proxy WebSocket traffic, search,
|
||||||
|
reasoning, caller context, authenticated AquaPortal MCP tools and load-balancer
|
||||||
|
requests.
|
||||||
|
|
||||||
|
## Local startup
|
||||||
|
|
||||||
|
From the repository root:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
S2S_PROXY_TARGET=ws://localhost:8765/v1/realtime ./start_ui.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
The launcher loads the root `.env`, syncs `vocero-api/.venv` from the committed
|
||||||
|
`vocero-api/uv.lock`, and installs the locked
|
||||||
|
frontend dependencies when needed. Vite serves the React frontend on port
|
||||||
|
8420 and proxies the backend routes to Uvicorn on `127.0.0.1:8421`.
|
||||||
|
`UI_PORT` overrides the public port; `S2S_BACKEND_PORT` overrides the internal
|
||||||
|
port. Both servers reload source edits, and stopping the launcher stops both.
|
||||||
|
The speech pipeline remains a separate service. Because `.env` is sourced by
|
||||||
|
the script, values in that file can replace variables supplied on the command line.
|
||||||
|
|
||||||
|
Use Node.js 22.18+ on the 22.x line or Node.js 24+, npm, uv 0.12.5,
|
||||||
|
Bash 4.4+, `setsid`, and `curl`. Remote staging access also needs the exact
|
||||||
|
hostname in `S2S_DEV_ALLOWED_HOSTS`, HTTPS, and WebSocket forwarding for speech
|
||||||
|
and Vite HMR. See [frontend development setup](../../vocero-frontend/README.md).
|
||||||
|
|
||||||
|
To install and start the backend manually:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv sync --project vocero-api --locked
|
||||||
|
S2S_PROXY_TARGET=ws://localhost:8765/v1/realtime \
|
||||||
|
uv run --project vocero-api --locked uvicorn --app-dir vocero-api vocero_api.main:app \
|
||||||
|
--host 127.0.0.1 --port 8421 --reload --reload-dir vocero-api
|
||||||
|
```
|
||||||
|
|
||||||
|
Then, in a second terminal, run `cd vocero-frontend`, `npm ci`, and `npm run dev`.
|
||||||
|
|
||||||
|
uv provisions Python 3.12 when needed. The Python environment belongs only to
|
||||||
|
`vocero-api`; an existing root `.venv` for speech dependencies is left alone.
|
||||||
|
See [development and compatibility](development.md) for checks and schema rules.
|
||||||
|
|
||||||
|
Open <http://localhost:8420/>, tap the orb/microphone control, grant microphone
|
||||||
|
access, and speak. Microphone access requires HTTPS or a local browser origin;
|
||||||
|
use HTTPS when accessing the UI from another machine.
|
||||||
|
|
||||||
|
The speech pipeline has separate model/runtime dependencies and no complete
|
||||||
|
dependency manifest in this checkout. In an already prepared speech
|
||||||
|
environment, a local realtime server can be started with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--mode realtime \
|
||||||
|
--stt parakeet-tdt \
|
||||||
|
--llm_backend transformers \
|
||||||
|
--tts kokoro \
|
||||||
|
--model_name Qwen/Qwen3-4B-Instruct-2507 \
|
||||||
|
--llm_device cuda \
|
||||||
|
--llm_torch_dtype float16 \
|
||||||
|
--enable_live_transcription
|
||||||
|
```
|
||||||
|
|
||||||
|
Select backends and devices supported by the host; the example model is not a
|
||||||
|
fixed deployment requirement. The default realtime endpoint is
|
||||||
|
`ws://localhost:8765/v1/realtime`, configured through `--ws_host` and `--ws_port`.
|
||||||
|
`--num_pipelines` controls the number of isolated concurrent realtime sessions.
|
||||||
|
See [LLM](../speech/llm.md), [STT](../speech/stt.md), and [TTS](../speech/tts.md).
|
||||||
|
|
||||||
|
## Connection modes
|
||||||
|
|
||||||
|
| Configuration | Browser connection | Settings URL | Load-balancer features |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `SPEECH_TO_SPEECH_URL` set | Direct to pinned URL | Locked | Disabled |
|
||||||
|
| `S2S_PROXY_TARGET` set, no direct URL | Same-origin `/v1/realtime` proxy | Locked | Disabled |
|
||||||
|
| `LOAD_BALANCER_URL` set, neither above set | Per-session compute URL from `/api/session` | Hidden | Allocation, queue polling and cancellation |
|
||||||
|
| None set | User-provided realtime URL | Editable | Disabled |
|
||||||
|
|
||||||
|
Direct URLs may be full `ws(s)://.../v1/realtime` addresses or host values that
|
||||||
|
the browser normalizes. A direct URL must be reachable **from the browser**.
|
||||||
|
`S2S_PROXY_TARGET` must be reachable **from the UI server**. HTTPS deployments
|
||||||
|
need a secure browser WebSocket URL.
|
||||||
|
|
||||||
|
The launcher supplies `ws://ai-server:12345/v1/realtime` as a proxy-target
|
||||||
|
default when no value is configured. That hostname must resolve from the UI
|
||||||
|
server; it is not a universally available local service.
|
||||||
|
|
||||||
|
## Docker
|
||||||
|
|
||||||
|
This image serves FastAPI backend routes and shared documentation. Run the
|
||||||
|
React frontend separately with Vite and point `S2S_BACKEND_URL` at the exposed
|
||||||
|
backend address. The image does not run the development launcher or serve a UI.
|
||||||
|
|
||||||
|
Use the project root as the build context:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker build -f vocero-api/Dockerfile -t vocero-api .
|
||||||
|
docker run --rm -p 7860:7860 \
|
||||||
|
-e SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime vocero-api
|
||||||
|
```
|
||||||
|
|
||||||
|
For the proxy path, replace the direct setting with a server-reachable
|
||||||
|
`S2S_PROXY_TARGET`. If the upstream is another container, attach the UI to the
|
||||||
|
appropriate existing network and use that service's hostname. An upstream
|
||||||
|
bound only to host loopback is not reachable via an arbitrary container name.
|
||||||
|
|
||||||
|
The image includes `vocero-api` and root `docs`. `/docs` is still the FastAPI API
|
||||||
|
explorer; documentation lives under `/docs/client/`, `/docs/speech/`,
|
||||||
|
`/docs/adr/`, `/docs/examples/`, and `/docs/proposals/`. Legacy `/project/` and
|
||||||
|
`/public/docs.html` redirect to the historical proposal. Markdown is served as a
|
||||||
|
source file rather than rendered through a separate documentation site. Request
|
||||||
|
file paths such as `/docs/README.md`; directories without `index.html` are not
|
||||||
|
browsable pages.
|
||||||
|
|
||||||
|
## Settings and tools
|
||||||
|
|
||||||
|
Voice, editable system instructions and connection settings are kept in browser
|
||||||
|
`localStorage` under `s2s.ws.*` keys. A non-empty `S2S_PRIMARY_PROMPT` pins the
|
||||||
|
instruction field. Restart applies connection/voice changes to a new session.
|
||||||
|
Settings and tool preferences use a draft: **Guardar** applies them and
|
||||||
|
**Cancelar** or dialog dismissal discards unsaved edits. Voice and instructions
|
||||||
|
update the live session on Save; connection changes apply on the next start.
|
||||||
|
Noise-gate calibration is explicitly live and saves immediately.
|
||||||
|
Model voice names are runtime identifiers and stay unchanged across UI
|
||||||
|
translations.
|
||||||
|
|
||||||
|
Web search needs `SERPER_API_KEY` on the server or a user-supplied key in the
|
||||||
|
tools panel. The server proxies requests through `/api/search`; its own key is
|
||||||
|
not returned to the browser. AquaPortal content tools use authenticated MCP;
|
||||||
|
see [tool calling](tool-calling.md).
|
||||||
|
|
||||||
|
Operator RAG text is applied through the RAG dialog and updates live
|
||||||
|
instructions. Caller links may include `phoneNumber=%2B...`; episode links may
|
||||||
|
include `episode_id=...`; MCP report links may include `report_id=...`.
|
||||||
|
See [context injection](context-injection.md) for
|
||||||
|
enabled data sources and limits.
|
||||||
|
|
||||||
|
### Authenticated AquaPortal MCP
|
||||||
|
|
||||||
|
`S2S_MCP_URL` enables the MCP client transport. Set it to the backend's complete
|
||||||
|
Streamable HTTP endpoint, for example `https://backend.example.test/api/v1/mcp/`.
|
||||||
|
Set `S2S_LOGIN_BY_PHONE` to that backend's `POST /api/v1/auth/phone-login` URL.
|
||||||
|
The phone parameter remains an identity selector without proof of ownership.
|
||||||
|
MCP uses the returned opaque `sessionToken` for that user, with no service-token override.
|
||||||
|
|
||||||
|
For voice reasoning (`S2S_REASONING_TYPE=PROXY`), set `S2S_PROXY_TARGET` to the
|
||||||
|
speech service and leave `SPEECH_TO_SPEECH_URL` and `LOAD_BALANCER_URL` unset.
|
||||||
|
The browser must connect to Vocero's same-origin `/v1/realtime` relay so tool
|
||||||
|
selections reach its executor. An incompatible direct configuration returns
|
||||||
|
503 from `/api/config`. `OPENAI` reasoning uses the authenticated HTTP loop and
|
||||||
|
can retain a direct or load-balanced speech connection. No speech-service code
|
||||||
|
or audio-format change is required.
|
||||||
|
|
||||||
|
The backend must expose the eight-tool catalog and phone-login contract described
|
||||||
|
in [tool calling](tool-calling.md#authenticated-aquaportal-mcp). Leave
|
||||||
|
`S2S_MCP_URL` unset to use the existing caller-context paths without MCP tools.
|
||||||
|
Setting it makes MCP the exclusive AquaPortal content path: legacy profile,
|
||||||
|
report, latest-summary, and episode preloads are skipped. Legacy caller HTTP
|
||||||
|
routes return 409 while MCP is enabled.
|
||||||
|
|
||||||
|
ApiPilot has been retired. `/api/apipilot/tools` and `/api/apipilot/execute`
|
||||||
|
return 404, and `/api/config` no longer includes `apipilot`. Remove obsolete
|
||||||
|
`S2S_APIPILOT_TOOLS_URL`, `S2S_APIPILOT_TOOLS_EXECUTE_URL`,
|
||||||
|
`S2S_APIPILOT_SERVICE_TOKEN`, `S2S_APIPILOT_API_KEY`, `S2S_APIPILOT_TTL_SEC`,
|
||||||
|
and `S2S_APIPILOT_TIMEOUT_SEC` from deployment configuration; they are ignored.
|
||||||
|
Keep `S2S_LOGIN_BY_PHONE`, which is still required for MCP authentication.
|
||||||
|
Deploy the API and frontend together and refresh existing browser sessions.
|
||||||
|
The separately deployed `speech_to_speech/` service is unchanged by this removal;
|
||||||
|
its existing legacy marker sanitizers remain, with no speech restart required.
|
||||||
|
|
||||||
|
| Setting | Default | Meaning |
|
||||||
|
|---|---|---|
|
||||||
|
| `S2S_MCP_URL` | Empty, disabled | Backend MCP endpoint |
|
||||||
|
| `S2S_MCP_TIMEOUT_SEC` | `20` | Entire MCP operation timeout, clamped to 1–120 seconds |
|
||||||
|
| `S2S_MCP_MAX_CALLS_PER_TURN` | `6` | Tool attempts per user turn, clamped to 1–20 |
|
||||||
|
| `S2S_CONVERSATION_TTL_SEC` | `3600` | Absolute conversation lifetime, clamped to 60–86400 seconds |
|
||||||
|
| `S2S_MAX_CONVERSATIONS` | `1000` | Process-local conversation capacity, clamped to 1–10000 |
|
||||||
|
|
||||||
|
`POST /api/conversations` accepts `{"phoneNumber":"+56000000000"}` with optional
|
||||||
|
`report_id` and `episode_id` fields, and returns
|
||||||
|
`conversationId`, `expiresIn`, and a minimal caller profile. An HttpOnly,
|
||||||
|
SameSite=Strict cookie binds the browser to its conversations; HTTPS sets Secure.
|
||||||
|
Backend credentials stay in memory on the API server. HTTP conversation requests
|
||||||
|
use `X-Vocero-Conversation`; the same-origin WebSocket uses `conversation_id`.
|
||||||
|
The conversation ID alone is insufficient without its owning browser cookie.
|
||||||
|
Different tabs can own different caller identities without replacing one another.
|
||||||
|
|
||||||
|
`GET /api/mcp/tools` discovers the authenticated catalog.
|
||||||
|
`POST /api/reasoning` requires an increasing positive `X-Vocero-Turn` header
|
||||||
|
in addition to the conversation header in MCP mode.
|
||||||
|
`POST /api/conversations/current/interrupt` cancels the matching `X-Vocero-Turn`, and
|
||||||
|
`DELETE /api/conversations/current` clears that conversation. These endpoints
|
||||||
|
require the cookie and conversation header. They do not change the existing
|
||||||
|
load-balancer `/api/session` contract. Stopping a conversation drops its local
|
||||||
|
credentials; it does not revoke backend sessions globally.
|
||||||
|
|
||||||
|
Conversation state is bounded, process-local, and cleared on restart. Use one
|
||||||
|
API worker, as in the launcher and Docker command. Multiple workers or replicas
|
||||||
|
require a shared conversation store before enabling this integration. Browser
|
||||||
|
sessions must restart after expiry or an API reload. Backend access is checked
|
||||||
|
on each MCP request. There is no shared content or authorization cache. HTTP
|
||||||
|
reasoning retains up to six completed turns, bounded to 96,000 characters,
|
||||||
|
within that conversation; the speech service keeps its existing conversation
|
||||||
|
history. These histories are cleared when their sessions end.
|
||||||
|
|
||||||
|
The browser waits for phone login, linked-content resolution, and tool discovery
|
||||||
|
before starting an MCP conversation. Login or discovery failure blocks startup
|
||||||
|
and displays a retryable error. Link resolution is limited to ten seconds and
|
||||||
|
uses only authenticated MCP tools. Missing, inaccessible, or temporarily
|
||||||
|
unavailable linked content permits a generic conversation; an expired backend
|
||||||
|
session still prevents startup. The browser allows thirty seconds for login
|
||||||
|
and link resolution together.
|
||||||
|
|
||||||
|
For example, `?phoneNumber=%2B56000000000&report_id=report-example` selects the
|
||||||
|
report and its podcast, `report:report-example`, when available. The full report
|
||||||
|
ID is preserved. An `episode_id` selects a daily podcast, `daily:<episode_id>`.
|
||||||
|
When both are present, the explicit daily episode determines the podcast and
|
||||||
|
`report_id` remains a separate report reference. No unavailable episode is
|
||||||
|
silently replaced with a different one.
|
||||||
|
|
||||||
|
The link provides the initial subject, not a search filter or an access grant.
|
||||||
|
Users can ask about any accessible report, compare multiple reports, or choose
|
||||||
|
another subject. The local `select_content` action validates that choice and
|
||||||
|
updates the current reference while preserving the original link for follow-ups.
|
||||||
|
Merely searching or reading content does not change the selection. A new
|
||||||
|
conversation, including after restart, resolves the page's selectors again.
|
||||||
|
Changing browser-supplied profile, phone, report, or episode values on a
|
||||||
|
reasoning request cannot change the stored identity or selection.
|
||||||
|
|
||||||
|
## Theme and session startup
|
||||||
|
|
||||||
|
`S2S_THEME=white` selects the light interface. `dark` or an empty value selects
|
||||||
|
the dark aurora theme. Theme-specific sections remain hidden until configuration
|
||||||
|
resolves, with a fallback to avoid an invisible page after config failure.
|
||||||
|
Configuration failures offer **Reintentar** and block session startup until a
|
||||||
|
retry succeeds. With MCP disabled, reports load independently and caller context
|
||||||
|
gets an eight-second loading window before the interface allows a generic session.
|
||||||
|
|
||||||
|
`S2S_START_SESSION_MODE` controls initial microphone/session startup:
|
||||||
|
|
||||||
|
| Value | Behavior |
|
||||||
|
|---|---|
|
||||||
|
| Empty/default | User taps to start |
|
||||||
|
| `TALK` | Attempt automatic start and inject the Spanish wake phrase as soon as configuration and caller context are ready |
|
||||||
|
| `TALKFIRST` | Same startup trigger as `TALK`; with MCP, greet using the linked report or podcast title after authenticated resolution |
|
||||||
|
|
||||||
|
Automatic startup runs once per page load, without a startup dialog or artificial
|
||||||
|
delay; optional reports and tools can still be loading. Browser permission and
|
||||||
|
audio policies can still require a gesture. When audio is blocked, an inline hint
|
||||||
|
asks the user to tap the microphone button. Permission or connection failures
|
||||||
|
show the error and leave the same button available to retry. Stopping a session
|
||||||
|
does not start it again automatically.
|
||||||
|
The greeting uses the same `hola vocero` input as wake-word handling. With MCP
|
||||||
|
and TALKFIRST, the greeting names the linked report, or the podcast when no
|
||||||
|
report is available, and invites questions about this or other reports. It does
|
||||||
|
not summarize automatically. A missing reference gets a brief explanation;
|
||||||
|
links without a content selector receive a generic greeting.
|
||||||
|
|
||||||
|
In `PROXY`, the relay supplies the verified metadata in the initial speech
|
||||||
|
instructions. In `OPENAI`, the controller sends the opening input through the
|
||||||
|
authenticated HTTP reasoning path after the speech connection is ready, then
|
||||||
|
uses the existing external-answer playback. Speaking first or stopping the
|
||||||
|
session cancels a pending greeting. Neither path changes the speech service.
|
||||||
|
|
||||||
|
On browsers with speech recognition, **Activar con «hola vocero»** opts into
|
||||||
|
wake-word listening for the current page session. The interface shows when the
|
||||||
|
microphone is listening for that phrase. It is off by default, pauses while a
|
||||||
|
dialog or voice session is active, and turns off when the conversation stops.
|
||||||
|
|
||||||
|
The transcript keeps your reading position when new messages arrive. **Ir al
|
||||||
|
último mensaje** resumes following the conversation. Names and Santiago-local
|
||||||
|
times appear with each message; downloaded JSON and text retain raw timestamps
|
||||||
|
and message text. The interface respects the browser's reduced-motion setting.
|
||||||
|
|
||||||
|
## Queue and session lifecycle
|
||||||
|
|
||||||
|
Load-balancer mode proxies `POST /api/session`. A free compute slot returns a
|
||||||
|
grant containing `session_id`, `connect_url`, and `pending_timeout_s`. Otherwise
|
||||||
|
the browser receives a queue ticket and polls `GET /api/queue/{queue_id}` until
|
||||||
|
a slot is granted. The browser offers a timed join action after waiting.
|
||||||
|
|
||||||
|
`DELETE /api/queue/{queue_id}` cancels a waiting ticket. On session stop or page
|
||||||
|
exit, the browser uses `POST /api/queue/end` through `sendBeacon`, falling back
|
||||||
|
to a keepalive fetch. Expired tickets and missed join deadlines require a new
|
||||||
|
attempt. Full queues return `503` with `state: "at_capacity"`.
|
||||||
|
|
||||||
|
Once connected, session teardown closes the microphone, audio graph and
|
||||||
|
WebSocket. The 30-second inactivity timeout counts time waiting for user input;
|
||||||
|
processing, external reasoning and buffered playback do not consume it.
|
||||||
|
Caller-profile lookup and upstream authorization operate
|
||||||
|
independently of allocation and use their existing configured endpoints.
|
||||||
|
|
||||||
|
For external transcript processing, configure the selected
|
||||||
|
[reasoning mode](reasoning.md). For connection, context and audio diagnostics,
|
||||||
|
use [troubleshooting](troubleshooting.md).
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
# Examples
|
||||||
|
|
||||||
|
- [agent-profile.json](agent-profile.json) is an illustrative agent-profile
|
||||||
|
response with a dynamic tool catalog. Its model instructions are written in
|
||||||
|
English and request Spanish speech through both instructions and `language`.
|
||||||
|
Tool names and schemas illustrate the original API response; this file is not
|
||||||
|
loaded by the application and is not a live API contract.
|
||||||
|
- [reference-audio.wav](reference-audio.wav) is the existing optional
|
||||||
|
voice-cloning sample, moved out of the implementation directory. CustomVoice
|
||||||
|
models do not require it. When testing a compatible Base/voice-cloning model,
|
||||||
|
supply its path explicitly with `--qwen3_tts_ref_audio
|
||||||
|
docs/examples/reference-audio.wav` from the project root and use the matching
|
||||||
|
transcription required by that model. See [the TTS guide](../speech/tts.md).
|
||||||
|
|
||||||
|
These examples contain no deployment credentials and are not default runtime
|
||||||
|
configuration. Use an authorized reference voice for your own deployment.
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"agent": {
|
"agent": {
|
||||||
"name": "Aquachile Voice Agent",
|
"name": "Aquachile Voice Agent",
|
||||||
"description": "Asistente de voz de Aquachile para soporte, informacion operativa y guia conversacional en tiempo real.",
|
"description": "Aquachile voice assistant for support, operational information, and real-time conversational guidance.",
|
||||||
"model": "gpt-realtime",
|
"model": "gpt-realtime",
|
||||||
"voice": "alloy",
|
"voice": "alloy",
|
||||||
"language": "es",
|
"language": "es",
|
||||||
@@ -564,7 +564,7 @@
|
|||||||
"operations",
|
"operations",
|
||||||
"voice"
|
"voice"
|
||||||
],
|
],
|
||||||
"instructions": "Eres el asistente de voz oficial de Aquachile. Debes hablar siempre en espanol, con tono profesional, claro, cordial y seguro. Responde de forma breve pero util. Haz una sola pregunta por turno cuando necesites aclaraciones. No inventes datos, precios, fechas, disponibilidad ni politicas. Si necesitas informacion verificable, usa primero las herramientas disponibles. Si una accion requiere confirmacion del usuario, pidela antes de continuar. Resume siempre la accion o conclusion al final de cada respuesta. Si el usuario cambia de tema, adapta el contexto sin perder claridad. Si no tienes datos suficientes, dilo explicitamente y propone el siguiente paso mas util. Nunca respondas en italiano o ingles salvo que el usuario lo pida de forma explicita.",
|
"instructions": "You are Aquachile's official voice assistant. Always speak Spanish in a professional, clear, friendly, and confident tone. Keep replies brief and useful. Ask one question per turn when clarification is needed. Do not invent data, prices, dates, availability, or policies. Use the available tools first when you need verifiable information. If an action requires user confirmation, ask before proceeding. Summarize the action or conclusion at the end of each reply. If the user changes the subject, adapt the context while staying clear. If you lack sufficient information, say so explicitly and suggest the most useful next step. Never reply in Italian or English unless the user explicitly asks.",
|
||||||
"turnDetection": {
|
"turnDetection": {
|
||||||
"type": "server_vad"
|
"type": "server_vad"
|
||||||
}
|
}
|
||||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,162 @@
|
|||||||
|
# Language model backends
|
||||||
|
|
||||||
|
## Available LLM backends (`--llm_backend`)
|
||||||
|
|
||||||
|
Runtime-supported values in [s2s_pipeline.py](../../speech_to_speech/s2s_pipeline.py):
|
||||||
|
|
||||||
|
- `transformers` → `language_model.py` (Transformers backend)
|
||||||
|
- `mlx-lm` → `language_model.py` (MLX backend)
|
||||||
|
- `responses-api` → `responses_api_language_model.py`
|
||||||
|
- `chat-completions` → `chat_completions_language_model.py`
|
||||||
|
|
||||||
|
Run the commands below from the repository root in an environment with the speech
|
||||||
|
pipeline dependencies installed. This checkout has no package manifest or complete
|
||||||
|
speech dependency installer; the UI requirements alone do not install the pipeline.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### 1) Transformers (`--llm_backend transformers`)
|
||||||
|
|
||||||
|
- Handler: `LanguageModelHandler`
|
||||||
|
- Typical use: local GPU/CPU inference using Hugging Face Transformers
|
||||||
|
- Backend-specific args prefix: `--llm_*`
|
||||||
|
- Shared args (from base): `--model_name`, `--chat_size`, `--init_chat_prompt`, `--enable_lang_prompt`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--llm_backend transformers \
|
||||||
|
--model_name Qwen/Qwen3-4B-Instruct-2507 \
|
||||||
|
--llm_device cuda \
|
||||||
|
--llm_torch_dtype float16 \
|
||||||
|
--llm_gen_max_new_tokens 128
|
||||||
|
```
|
||||||
|
|
||||||
|
Common options:
|
||||||
|
- `--llm_gen_min_new_tokens`
|
||||||
|
- `--llm_gen_temperature`
|
||||||
|
- `--llm_gen_do_sample`
|
||||||
|
- `--chat_size`
|
||||||
|
- `--init_chat_prompt`
|
||||||
|
|
||||||
|
### 2) MLX-LM (`--llm_backend mlx-lm`)
|
||||||
|
|
||||||
|
- Handler: `LanguageModelHandler`
|
||||||
|
- Typical use: Apple Silicon local inference
|
||||||
|
- Backend-specific args prefix: same as Transformers (`--llm_*`)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--llm_backend mlx-lm \
|
||||||
|
--model_name mlx-community/Qwen3-4B-Instruct-2507-bf16 \
|
||||||
|
--llm_device mps \
|
||||||
|
--llm_gen_max_new_tokens 128
|
||||||
|
```
|
||||||
|
|
||||||
|
Common options:
|
||||||
|
- `--llm_gen_temperature`
|
||||||
|
- `--llm_gen_do_sample`
|
||||||
|
- `--chat_size`
|
||||||
|
- `--init_chat_prompt`
|
||||||
|
|
||||||
|
### 3) OpenAI-compatible API (`--llm_backend responses-api`)
|
||||||
|
|
||||||
|
- Handler: `ResponsesApiModelHandler`
|
||||||
|
- Typical use: remote model serving via OpenAI-compatible endpoints
|
||||||
|
- Backend-specific args prefix: `--responses_api_*`
|
||||||
|
- Shared args (from base): `--model_name`, `--chat_size`, `--init_chat_prompt`, `--enable_lang_prompt`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--llm_backend responses-api \
|
||||||
|
--model_name gpt-5.4-mini \
|
||||||
|
--responses_api_api_key YOUR_API_KEY \
|
||||||
|
--responses_api_base_url https://api.example.com/v1 \
|
||||||
|
--responses_api_stream true
|
||||||
|
```
|
||||||
|
|
||||||
|
Common options:
|
||||||
|
- `--chat_size`
|
||||||
|
- `--init_chat_prompt`
|
||||||
|
- `--user_role`
|
||||||
|
|
||||||
|
### 4) Chat Completions (`--llm_backend chat-completions`)
|
||||||
|
|
||||||
|
- Handler: `ChatCompletionsLanguageModelHandler`
|
||||||
|
- Uses the provider's `/chat/completions` API rather than `/responses`
|
||||||
|
- Inherits the same `--responses_api_base_url`, `--responses_api_api_key`,
|
||||||
|
`--responses_api_stream`, and `--responses_api_disable_thinking` flags
|
||||||
|
- Adds `--responses_api_reasoning_effort` for providers that support that setting
|
||||||
|
|
||||||
|
The current handler removes `tools` and `tool_choice` from outgoing provider
|
||||||
|
requests as a compatibility workaround. Native tool calling therefore differs
|
||||||
|
from the `responses-api` backend; see the [realtime guide](realtime-api.md).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--llm_backend chat-completions \
|
||||||
|
--model_name YOUR_MODEL \
|
||||||
|
--responses_api_base_url https://api.example.com/v1 \
|
||||||
|
--responses_api_api_key YOUR_API_KEY
|
||||||
|
```
|
||||||
|
|
||||||
|
The default LLM backend is `responses-api`. Model names in this guide are examples;
|
||||||
|
select a model available in your environment or from your provider.
|
||||||
|
|
||||||
|
## LLM Behavior
|
||||||
|
|
||||||
|
When STT is set to language auto-detection (`--language auto`), LLM handlers can receive `(text, language_code)` and prepend a language control instruction like:
|
||||||
|
|
||||||
|
- `Please reply to my message in <language>.`
|
||||||
|
|
||||||
|
This helps the assistant respond in the detected language. The behavior is opt-in via `--enable_lang_prompt` (shared across all backends); it defaults to `False`.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
### CUDA setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--llm_backend transformers \
|
||||||
|
--model_name microsoft/Phi-3-mini-4k-instruct
|
||||||
|
```
|
||||||
|
|
||||||
|
### Local Mac setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--local_mac_optimal_settings \
|
||||||
|
--model_name mlx-community/Qwen3-4B-Instruct-2507-bf16
|
||||||
|
```
|
||||||
|
|
||||||
|
`--local_mac_optimal_settings` already sets `--llm_backend mlx-lm` and will default the model to `mlx-community/Qwen3-4B-Instruct-2507-bf16` if not overridden.
|
||||||
|
|
||||||
|
### Realtime (OpenAI-compatible) setup
|
||||||
|
|
||||||
|
Run the server in realtime mode, then connect with the [browser client](../client/usage.md):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Start the pipeline in realtime mode
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--mode realtime \
|
||||||
|
--llm_backend mlx-lm \
|
||||||
|
--model_name mlx-community/Qwen3-4B-Instruct-2507-bf16 \
|
||||||
|
--ws_host 0.0.0.0 \
|
||||||
|
--ws_port 8765
|
||||||
|
|
||||||
|
# 2. Point the browser client at ws://localhost:8765/v1/realtime.
|
||||||
|
# See the browser-client usage guide for startup commands.
|
||||||
|
```
|
||||||
|
|
||||||
|
For realtime mode on Apple Silicon, set the desired MLX/STT/TTS/device options
|
||||||
|
explicitly. The current `--local_mac_optimal_settings` helper forces
|
||||||
|
`mode="local"`, so combining it with `--mode realtime` does not preserve realtime
|
||||||
|
mode.
|
||||||
|
|
||||||
|
### Remote API setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--llm_backend responses-api \
|
||||||
|
--model_name gpt-5.4-mini \
|
||||||
|
--responses_api_api_key YOUR_API_KEY
|
||||||
|
```
|
||||||
@@ -0,0 +1,230 @@
|
|||||||
|
# Realtime API
|
||||||
|
|
||||||
|
## High-level architecture
|
||||||
|
|
||||||
|
The server runs as a handler thread inside the existing pipeline. A FastAPI/uvicorn instance serves the WebSocket endpoint at `/v1/realtime`, while the same queue-driven handler chain (VAD, STT, LLM, TTS) processes audio in parallel threads.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
subgraph client [Client]
|
||||||
|
WS["WebSocket /v1/realtime"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph server [Realtime Server]
|
||||||
|
Router["WebSocket Router"]
|
||||||
|
Service["RealtimeService"]
|
||||||
|
Config["RuntimeConfig"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph pipeline [Pipeline Threads]
|
||||||
|
VAD["VAD"]
|
||||||
|
STT["STT"]
|
||||||
|
TN["TranscriptionNotifier"]
|
||||||
|
LLM["LLM"]
|
||||||
|
Proc["LMOutputProcessor"]
|
||||||
|
TTS["TTS"]
|
||||||
|
end
|
||||||
|
|
||||||
|
WS -- "client events (JSON)" --> Router
|
||||||
|
Router -- "parse / dispatch" --> Service
|
||||||
|
Service -- "PCM chunks" --> VAD
|
||||||
|
Service -- "session config" --> Config
|
||||||
|
Config -. "read by" .-> VAD & LLM & TTS
|
||||||
|
VAD -- "speech segments" --> STT
|
||||||
|
STT -- "transcript" --> TN
|
||||||
|
TN -- "transcript + notify" --> LLM
|
||||||
|
LLM -- "text + tools" --> Proc
|
||||||
|
Proc -- "clean text" --> TTS
|
||||||
|
TTS -- "PCM audio" --> Router
|
||||||
|
Proc -- "assistant_text / tools" --> Router
|
||||||
|
VAD -- "speech_started/stopped" --> Router
|
||||||
|
TN -- "transcription events" --> Router
|
||||||
|
Router -- "server events (JSON)" --> WS
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key flow:**
|
||||||
|
|
||||||
|
1. **Inbound audio**: Client sends `input_audio_buffer.append` with base64 PCM. `RealtimeService` decodes, resamples to 16 kHz, splits into 512-sample chunks, and puts them on the `recv_audio_chunks_queue` for VAD.
|
||||||
|
2. **Speech detection**: VAD detects speech boundaries and emits `speech_started` / `speech_stopped` events on the `text_output_queue`. Full utterance audio goes to STT.
|
||||||
|
3. **Transcription**: STT output passes through `TranscriptionNotifier`, which taps the transcript for `transcription.delta` / `transcription.completed` events before forwarding to the LLM.
|
||||||
|
4. **Generation**: The LLM generates text (and optional tool calls). `LMOutputProcessor` splits the output: clean text goes to TTS, and `assistant_text` + tool call dicts go to the `text_output_queue`.
|
||||||
|
5. **Outbound audio**: TTS writes PCM chunks to `send_audio_chunks_queue`. The router's async `_send_loop` drains both queues, encoding PCM as `response.output_audio.delta` events and translating internal messages into protocol events.
|
||||||
|
6. **Session config**: `session.update` events deep-merge into `RuntimeConfig`, which is a shared Pydantic model read by VAD (turn detection thresholds), LLM (instructions, tools), and TTS (voice) at processing time.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Supported OpenAI Realtime Events
|
||||||
|
|
||||||
|
### Client -> Server
|
||||||
|
|
||||||
|
| Event | Description |
|
||||||
|
|---|---|
|
||||||
|
| `input_audio_buffer.append` | Stream base64 PCM audio. Decoded, resampled to 16 kHz, and chunked for the VAD. |
|
||||||
|
| `session.update` | Deep-merge session config (instructions, tools, voice, turn detection, audio format). |
|
||||||
|
| `conversation.item.create` | Inject `input_text` or `function_call_output` into the LLM context without triggering generation. |
|
||||||
|
| `response.create` | Trigger LLM generation. Supports per-response `instructions` and `tool_choice` overrides. |
|
||||||
|
| `response.cancel` | Cancel the in-progress response and re-enable listening. |
|
||||||
|
|
||||||
|
### Server -> Client
|
||||||
|
|
||||||
|
| Event | Description |
|
||||||
|
|---|---|
|
||||||
|
| `session.created` | Sent on connection with current session config. |
|
||||||
|
| `error` | Protocol errors (`session_limit_reached`, `unknown_or_invalid_event`, `invalid_session_type`, `conversation_already_has_active_response`, etc.) |
|
||||||
|
| `input_audio_buffer.speech_started` | VAD detected user speech. |
|
||||||
|
| `input_audio_buffer.speech_stopped` | End of user speech segment. |
|
||||||
|
| `conversation.item.created` | Acknowledges injected `input_text` from `conversation.item.create`. |
|
||||||
|
| `conversation.item.input_audio_transcription.delta` | Streaming partial transcript (when live transcription is enabled). |
|
||||||
|
| `conversation.item.input_audio_transcription.completed` | Final transcript for the user turn (with duration usage). |
|
||||||
|
| `response.created` | Emitted on the first outbound audio chunk (response is `in_progress`). |
|
||||||
|
| `response.output_audio.delta` | Base64 PCM audio chunk from TTS. |
|
||||||
|
| `response.output_audio.done` | Audio stream complete for the current output item. |
|
||||||
|
| `response.output_audio_transcript.done` | Full assistant text transcript for the turn. |
|
||||||
|
| `response.function_call_arguments.done` | Tool call with `call_id`, `name`, and JSON `arguments`. |
|
||||||
|
| `response.done` | Response finished (`completed`, `cancelled` with reason `turn_detected` or `client_cancelled`). |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Tool Calling Design
|
||||||
|
|
||||||
|
Tool calling works through two distinct paths depending on the LLM backend, but both converge to the same wire protocol for the client.
|
||||||
|
|
||||||
|
### Local LLM path (`LanguageModelHandler` -- transformers / mlx-lm)
|
||||||
|
|
||||||
|
Tools defined in `session.update` are converted to `FunctionTool` objects. Each tool's JSON Schema `parameters` are turned into a Python `inspect.Signature` (via `signature_from_schema`), and `to_code_prompt()` renders a human-readable `def name(...): """docstring"""` block.
|
||||||
|
|
||||||
|
These tool signatures are injected into the system prompt using a Jinja2 template (`tool_prompt.py`) that instructs the model to wrap tool calls in `<code>...</code>` delimiters:
|
||||||
|
|
||||||
|
```
|
||||||
|
<code>
|
||||||
|
function_name(arg_name_1=value1, arg_name_2='string_value')
|
||||||
|
</code>
|
||||||
|
```
|
||||||
|
|
||||||
|
After generation, `_extract_tools` uses a regex to find `<code>` blocks, then `extract_function_calls_from_text` parses each `name(kwargs)` call and validates it against registered tools. Valid calls become `ResponseFunctionToolCall` dicts with generated `call_id`s.
|
||||||
|
|
||||||
|
### Remote API paths
|
||||||
|
|
||||||
|
`ResponsesApiModelHandler` passes tools natively to `client.responses.create`.
|
||||||
|
The API returns structured `function_call` items. Per-response `tool_choice`
|
||||||
|
overrides are read from `response.create`.
|
||||||
|
|
||||||
|
`ChatCompletionsLanguageModelHandler` also contains conversion for Chat
|
||||||
|
Completions tool-call results, but its current `_request()` removes `tools` and
|
||||||
|
`tool_choice` before the HTTP call to avoid a provider rejection. That backend
|
||||||
|
does not currently declare native tools to its provider. Do not assume its
|
||||||
|
result parsing implies working native tool selection.
|
||||||
|
|
||||||
|
### Common output path
|
||||||
|
|
||||||
|
Both handlers yield `(text, language_code, tools)` tuples. `LMOutputProcessor` forwards clean text to TTS and puts `{"type": "assistant_text", "text": ..., "tools": [...]}` on the `text_output_queue`. The router's `_send_loop` translates these into:
|
||||||
|
- `response.output_audio_transcript.done` for the text
|
||||||
|
- `response.function_call_arguments.done` for each tool call
|
||||||
|
|
||||||
|
### Tool result flow
|
||||||
|
|
||||||
|
1. Client executes the tool and sends `conversation.item.create` with `type: "function_call_output"` and `output: "<result>"`
|
||||||
|
2. `RealtimeService` appends the tool output to the chat context and emits `conversation.item.created`; this does not trigger generation.
|
||||||
|
3. If the tool result needs to be spoken to the user, such as camera/search/data results, the client sends `response.create` to trigger follow-up generation.
|
||||||
|
4. For fire-and-forget robot actions such as dance, emotion, head movement, stop, or idle tools, the client can stop after `conversation.item.created`; the assistant should already have spoken the natural lead-in before the tool call.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Interruption Handling
|
||||||
|
|
||||||
|
Barge-in (user speaks while the assistant is playing audio) is handled cooperatively between the VAD, the `_send_loop`, and the LLM/TTS handlers via a shared `CancelScope` object (`cancel_scope.py`).
|
||||||
|
|
||||||
|
### CancelScope design
|
||||||
|
|
||||||
|
`CancelScope` replaces the old two-signal pattern (`cancel_response` Event + `discard_stale_output` boolean) with a single object that manages:
|
||||||
|
|
||||||
|
- **Generation counter** (`cancel_scope.generation`): pipeline threads (LLM, TTS) capture the current generation at the start of each response and check `cancel_scope.is_stale(gen)` on every streaming token. When `cancel()` is called, the generation increments and all prior generations become stale -- no timing games required.
|
||||||
|
- **Discard flag** (`cancel_scope.discarding`): set by `cancel()`, checked by the async `_send_loop` to drop output from superseded generations that arrives between `cancel()` and `response_done()`. Cleared by `response_done(generation)` (only when the sentinel's generation matches the discarded or current one -- sentinels from unrelated older generations are ignored), by `new_response()` on an explicit `response.create`, or by `reset()` on session claim/release.
|
||||||
|
|
||||||
|
Pipeline output is **generation-tagged**: `AudioOutput` chunks and `AssistantTextEvent`s carry a `cancel_generation` field stamped by the handler that produced them. The send loop's `_generation_is_discardable` drops an item if its generation is stale, or if `discarding` is set and the item is not from the current generation. Output from the *current* generation always passes through, so a fresh response is never swallowed by a lingering discard window (e.g. a superseded speculative turn whose TTS never emitted a `__RESPONSE_DONE__` sentinel).
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant User
|
||||||
|
participant VAD
|
||||||
|
participant SendLoop as _send_loop
|
||||||
|
participant LLM
|
||||||
|
participant TTS
|
||||||
|
participant Client
|
||||||
|
|
||||||
|
Note over TTS,Client: Response active or pending (in_response / response_pending)
|
||||||
|
User->>VAD: speaks
|
||||||
|
VAD->>SendLoop: speech_started on text_output_queue
|
||||||
|
SendLoop->>Client: input_audio_buffer.speech_started
|
||||||
|
SendLoop->>Client: response.done (status=cancelled, reason=turn_detected)
|
||||||
|
SendLoop->>SendLoop: cancel_scope.cancel() (gen++ & discarding=True)
|
||||||
|
SendLoop->>SendLoop: flush output_queue + text_output_queue
|
||||||
|
SendLoop->>SendLoop: response_playing.clear()
|
||||||
|
LLM->>LLM: is_stale(gen) → True, aborts generation
|
||||||
|
TTS->>TTS: is_stale(gen) → True, aborts generation
|
||||||
|
TTS->>SendLoop: __RESPONSE_DONE__ (tagged with gen)
|
||||||
|
SendLoop->>SendLoop: cancel_scope.response_done(gen) (discarding=False)
|
||||||
|
Note over VAD,Client: Pipeline is now processing the new user utterance
|
||||||
|
```
|
||||||
|
|
||||||
|
**Step by step:**
|
||||||
|
|
||||||
|
1. **VAD detects speech**: puts a `SpeechStartedEvent` on `text_output_queue`.
|
||||||
|
2. **`_send_loop` processes text events first** (priority over audio): translates `speech_started` into protocol events. If an active response was in progress, `RealtimeService.dispatch_pipeline_event` emits `response.output_audio.done` + `response.done` with `status="cancelled"` and `reason="turn_detected"`.
|
||||||
|
3. **Cancel + queue flush**: if a response is active (`in_response`) *or* pending (`response_pending` -- accepted `response.create`, no audio yet), and interrupts are enabled (see step 4), the send loop calls `cancel_scope.cancel()` (increments generation, enables discard), clears `response_pending`, drains `output_queue` (preserving `__RESPONSE_DONE__` sentinels) and `text_output_queue` (preserving user-side events: `speech_stopped`, partial/completed transcriptions, token usage), then clears `response_playing`.
|
||||||
|
4. **Interrupt gating**: the cancel only fires if the `SpeechStartedEvent.interrupt_response` flag is set *and* the session config allows it (`turn_detection.interrupt_response`, read via `RuntimeConfig.interrupt_response_enabled`, default true). When disabled, user speech during a response is transcribed but the response keeps playing.
|
||||||
|
5. **LLM/TTS cancellation**: handlers capture `gen = cancel_scope.generation` at the start of each response and check `cancel_scope.is_stale(gen)` on every streaming token, aborting early when stale.
|
||||||
|
6. **Discard guard**: while `cancel_scope.discarding` is True, the send loop drops audio chunks and assistant text whose `cancel_generation` is not current (see `_generation_is_discardable` above). The guard clears when a `__RESPONSE_DONE__` with a matching generation arrives (via `cancel_scope.response_done(gen)`), or when an explicit `response.create` starts a new response (`cancel_scope.new_response()`).
|
||||||
|
7. **Client-initiated cancel**: `response.cancel` calls `cancel_scope.cancel()` (only if a response was active), flushes both queues with the same preservation rules, triggers `finish_response(status="cancelled", reason="client_cancelled")`, re-enables `should_listen`, and clears `response_playing`.
|
||||||
|
8. **Spurious cancel safety**: if no response is active, `cancel_scope.cancel()` is not called, preventing the discard guard from being set without a `__RESPONSE_DONE__` to clear it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
Run these examples from the repository root in an environment with the speech
|
||||||
|
dependencies installed. See [LLM](llm.md), [STT](stt.md), and [TTS](tts.md) for
|
||||||
|
backend-specific options; this checkout does not include a package manifest.
|
||||||
|
|
||||||
|
### Local LLM with Transformers
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--mode realtime \
|
||||||
|
--stt parakeet-tdt \
|
||||||
|
--llm_backend transformers \
|
||||||
|
--tts kokoro \
|
||||||
|
--model_name "Qwen/Qwen3-4B-Instruct-2507" \
|
||||||
|
--llm_device mps \
|
||||||
|
--llm_torch_dtype float16 \
|
||||||
|
--enable_live_transcription
|
||||||
|
```
|
||||||
|
|
||||||
|
### Local LLM with MLX-LM
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--mode realtime \
|
||||||
|
--stt parakeet-tdt \
|
||||||
|
--llm_backend mlx-lm \
|
||||||
|
--tts kokoro \
|
||||||
|
--model_name "mlx-community/Qwen3-4B-Instruct-2507-bf16" \
|
||||||
|
--llm_device mps \
|
||||||
|
--llm_torch_dtype float16 \
|
||||||
|
--enable_live_transcription
|
||||||
|
```
|
||||||
|
|
||||||
|
### Remote LLM with OpenAI-compatible API
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--mode realtime \
|
||||||
|
--stt parakeet-tdt \
|
||||||
|
--llm_backend responses-api \
|
||||||
|
--tts kokoro \
|
||||||
|
--model_name "openai/gpt-oss-20b:groq" \
|
||||||
|
--responses_api_base_url "https://router.huggingface.co/v1" \
|
||||||
|
--responses_api_api_key "$HF_TOKEN" \
|
||||||
|
--responses_api_stream \
|
||||||
|
--enable_live_transcription
|
||||||
|
```
|
||||||
@@ -0,0 +1,166 @@
|
|||||||
|
# Speech-to-text backends
|
||||||
|
|
||||||
|
This document summarizes the speech-to-text implementations in
|
||||||
|
[speech_to_speech/STT](../../speech_to_speech/STT), including language support,
|
||||||
|
language abbreviations, and pipeline options. Run examples from the repository
|
||||||
|
root in an environment with the speech dependencies installed.
|
||||||
|
|
||||||
|
## Available STT Modes (`--stt`)
|
||||||
|
|
||||||
|
- `whisper` → `STT/whisper_stt_handler.py`
|
||||||
|
- `whisper-mlx` → `STT/lightning_whisper_mlx_handler.py`
|
||||||
|
- `mlx-audio-whisper` → `STT/mlx_audio_whisper_handler.py`
|
||||||
|
- `faster-whisper` → `STT/faster_whisper_handler.py`
|
||||||
|
- `parakeet-tdt` → `STT/parakeet_tdt_handler.py`
|
||||||
|
- `paraformer` → `STT/paraformer_handler.py`
|
||||||
|
|
||||||
|
## Language Support by Handler
|
||||||
|
|
||||||
|
### 1) Whisper (`--stt whisper`)
|
||||||
|
|
||||||
|
- Handler: `WhisperSTTHandler`
|
||||||
|
- Language input flag: `--language` (from shared Whisper args)
|
||||||
|
- Supports fixed language (e.g. `en`) or `auto`
|
||||||
|
- Internal supported language list:
|
||||||
|
- `en`, `fr`, `es`, `zh`, `ja`, `ko`, `hi`, `de`, `pt`, `pl`, `it`, `nl`
|
||||||
|
- Behavior:
|
||||||
|
- Detects language from token output
|
||||||
|
- If detected language is outside the supported list, it falls back to the previous language
|
||||||
|
|
||||||
|
### 2) Lightning Whisper MLX (`--stt whisper-mlx`)
|
||||||
|
|
||||||
|
- Handler: `LightningWhisperSTTHandler`
|
||||||
|
- Uses same shared `--language` argument as Whisper
|
||||||
|
- Internal supported language list:
|
||||||
|
- `en`, `fr`, `es`, `zh`, `ja`, `ko`, `hi`, `de`, `pt`, `pl`, `it`, `nl`
|
||||||
|
- Behavior:
|
||||||
|
- If `--language auto`, model auto-detects each utterance
|
||||||
|
- If detected language is unsupported, falls back to last supported language
|
||||||
|
|
||||||
|
### 3) MLX Audio Whisper (`--stt mlx-audio-whisper`)
|
||||||
|
|
||||||
|
- Handler: `MLXAudioWhisperSTTHandler`
|
||||||
|
- Model flag: `--mlx_audio_whisper_model_name`
|
||||||
|
- Language still comes from shared `--language` flag (wired in pipeline)
|
||||||
|
- Internal supported language list:
|
||||||
|
- `en`, `fr`, `es`, `zh`, `ja`, `ko`, `hi`, `de`, `pt`, `pl`, `it`, `nl`
|
||||||
|
- Behavior:
|
||||||
|
- Uses fixed language unless `--language auto`
|
||||||
|
- Falls back to last known supported language when needed
|
||||||
|
|
||||||
|
### 4) Faster-Whisper (`--stt faster-whisper`)
|
||||||
|
|
||||||
|
- Handler: `FasterWhisperSTTHandler`
|
||||||
|
- Language flag: `--faster_whisper_stt_gen_language`
|
||||||
|
- Default language: `en`
|
||||||
|
- Note:
|
||||||
|
- This handler passes generation kwargs directly to `faster_whisper.WhisperModel.transcribe(...)`
|
||||||
|
- Effective language coverage depends on selected Faster-Whisper/OpenAI Whisper model
|
||||||
|
|
||||||
|
### 5) Parakeet TDT (`--stt parakeet-tdt`)
|
||||||
|
|
||||||
|
- Handler: `ParakeetTDTSTTHandler`
|
||||||
|
- Language flag: `--parakeet_tdt_language` (optional)
|
||||||
|
- Supports auto language detection when language not specified
|
||||||
|
- Declared supported language list (25 European languages):
|
||||||
|
- `en`, `de`, `fr`, `es`, `it`, `pt`, `nl`, `pl`, `ru`, `uk`, `cs`, `sk`, `hu`, `ro`, `bg`, `hr`, `sl`, `sr`, `da`, `no`, `sv`, `fi`, `et`, `lv`, `lt`
|
||||||
|
- Backend behavior:
|
||||||
|
- On macOS/MPS: MLX (`mlx-community/parakeet-tdt-0.6b-v3`)
|
||||||
|
- On CUDA/CPU: nano-parakeet (`nvidia/parakeet-tdt-0.6b-v3`)
|
||||||
|
|
||||||
|
### 6) Paraformer (`--stt paraformer`)
|
||||||
|
|
||||||
|
- Handler: `ParaformerSTTHandler`
|
||||||
|
- Model flag: `--paraformer_stt_model_name`
|
||||||
|
- Default model: `paraformer-zh`
|
||||||
|
- No dedicated language flag in current args class
|
||||||
|
- Practical support:
|
||||||
|
- Depends on selected FunASR model checkpoint
|
||||||
|
- Default setup is Chinese-oriented (`zh`)
|
||||||
|
|
||||||
|
## Language Abbreviations (ISO-style codes seen in STT handlers)
|
||||||
|
|
||||||
|
| Code | Language |
|
||||||
|
|---|---|
|
||||||
|
| `en` | English |
|
||||||
|
| `fr` | French |
|
||||||
|
| `es` | Spanish |
|
||||||
|
| `zh` | Chinese |
|
||||||
|
| `ja` | Japanese |
|
||||||
|
| `ko` | Korean |
|
||||||
|
| `hi` | Hindi |
|
||||||
|
| `de` | German |
|
||||||
|
| `pt` | Portuguese |
|
||||||
|
| `pl` | Polish |
|
||||||
|
| `it` | Italian |
|
||||||
|
| `nl` | Dutch |
|
||||||
|
| `ru` | Russian |
|
||||||
|
| `uk` | Ukrainian |
|
||||||
|
| `cs` | Czech |
|
||||||
|
| `sk` | Slovak |
|
||||||
|
| `hu` | Hungarian |
|
||||||
|
| `ro` | Romanian |
|
||||||
|
| `bg` | Bulgarian |
|
||||||
|
| `hr` | Croatian |
|
||||||
|
| `sl` | Slovenian |
|
||||||
|
| `sr` | Serbian |
|
||||||
|
| `da` | Danish |
|
||||||
|
| `no` | Norwegian |
|
||||||
|
| `sv` | Swedish |
|
||||||
|
| `fi` | Finnish |
|
||||||
|
| `et` | Estonian |
|
||||||
|
| `lv` | Latvian |
|
||||||
|
| `lt` | Lithuanian |
|
||||||
|
| `auto` | Per-utterance automatic language detection |
|
||||||
|
|
||||||
|
## Usage Examples
|
||||||
|
|
||||||
|
### Whisper (Transformers)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline --stt whisper --language es
|
||||||
|
python -m speech_to_speech.s2s_pipeline --stt whisper --language auto
|
||||||
|
```
|
||||||
|
|
||||||
|
### Whisper MLX (LightningWhisperMLX)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline --stt whisper-mlx --language auto --device mps
|
||||||
|
```
|
||||||
|
|
||||||
|
### MLX Audio Whisper
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline --stt mlx-audio-whisper \
|
||||||
|
--mlx_audio_whisper_model_name mlx-community/whisper-large-v3-turbo \
|
||||||
|
--language auto
|
||||||
|
```
|
||||||
|
|
||||||
|
### Faster-Whisper
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline --stt faster-whisper \
|
||||||
|
--faster_whisper_stt_model_name large-v3 \
|
||||||
|
--faster_whisper_stt_gen_language en
|
||||||
|
```
|
||||||
|
|
||||||
|
### Parakeet TDT
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline --stt parakeet-tdt --parakeet_tdt_device auto
|
||||||
|
python -m speech_to_speech.s2s_pipeline --stt parakeet-tdt --parakeet_tdt_language es
|
||||||
|
```
|
||||||
|
|
||||||
|
With live transcription (MLX or CUDA/nano-parakeet backend):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline --stt parakeet-tdt \
|
||||||
|
--enable_live_transcription \
|
||||||
|
--live_transcription_update_interval 0.25
|
||||||
|
```
|
||||||
|
|
||||||
|
### Paraformer
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline --stt paraformer --paraformer_stt_model_name paraformer-zh
|
||||||
|
```
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
# Text-to-speech backends
|
||||||
|
|
||||||
|
## Available TTS modes (`--tts`)
|
||||||
|
|
||||||
|
Runtime-supported values in [s2s_pipeline.py](../../speech_to_speech/s2s_pipeline.py):
|
||||||
|
|
||||||
|
- `chatTTS` → `chatTTS_handler.py`
|
||||||
|
- `facebookMMS` → `facebookmms_handler.py`
|
||||||
|
- `pocket` → `pocket_tts_handler.py`
|
||||||
|
- `kokoro` → `kokoro_handler.py`
|
||||||
|
- `qwen3` → `qwen3_tts_handler.py`
|
||||||
|
|
||||||
|
Run examples from the repository root in an environment with the speech
|
||||||
|
dependencies installed. The UI requirements do not install TTS dependencies.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### 1) ChatTTS (`--tts chatTTS`)
|
||||||
|
|
||||||
|
Primary args prefix: `--chat_tts_*`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--tts chatTTS \
|
||||||
|
--chat_tts_device cuda \
|
||||||
|
--chat_tts_stream true \
|
||||||
|
--chat_tts_chunk_size 512
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2) Facebook MMS (`--tts facebookMMS`)
|
||||||
|
|
||||||
|
Primary args prefix: `--facebook_mms_*` plus `--tts_language`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--tts facebookMMS \
|
||||||
|
--facebook_mms_device cuda \
|
||||||
|
--tts_language en
|
||||||
|
```
|
||||||
|
|
||||||
|
This handler maps STT language codes (e.g. `en`, `fr`, `es`) to MMS model suffixes (e.g. `eng`, `fra`, `spa`) and reloads the model on language changes.
|
||||||
|
|
||||||
|
### 3) Pocket TTS (`--tts pocket`)
|
||||||
|
|
||||||
|
Primary args prefix: `--pocket_tts_*`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--tts pocket \
|
||||||
|
--pocket_tts_voice jean \
|
||||||
|
--pocket_tts_device cpu \
|
||||||
|
--pocket_tts_sample_rate 16000
|
||||||
|
```
|
||||||
|
|
||||||
|
Available preset voices include:
|
||||||
|
`alba`, `marius`, `javert`, `jean`, `fantine`, `cosette`, `eponine`, `azelma`.
|
||||||
|
|
||||||
|
### 4) Kokoro (`--tts kokoro`)
|
||||||
|
|
||||||
|
Primary args prefix: `--kokoro_*`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--tts kokoro \
|
||||||
|
--kokoro_device auto \
|
||||||
|
--kokoro_voice bm_fable \
|
||||||
|
--kokoro_lang_code b
|
||||||
|
```
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
- Uses MLX backend on Apple Silicon (`mlx-community/Kokoro-82M-bf16`)
|
||||||
|
- Uses native kokoro pipeline otherwise (`hexgrad/Kokoro-82M`)
|
||||||
|
- Can auto-switch voice/language based on STT language code mapping
|
||||||
|
|
||||||
|
### 5) Qwen3-TTS (`--tts qwen3`)
|
||||||
|
|
||||||
|
Primary args prefix: `--qwen3_tts_*`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--tts qwen3 \
|
||||||
|
--qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
|
||||||
|
--qwen3_tts_device cuda \
|
||||||
|
--qwen3_tts_backend ggml \
|
||||||
|
--qwen3_tts_speaker Aiden \
|
||||||
|
--qwen3_tts_language auto \
|
||||||
|
--qwen3_tts_non_streaming_mode True
|
||||||
|
```
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
- Uses `faster-qwen3-tts` on non-macOS platforms, defaulting to the GGML backend. Pass `--qwen3_tts_backend torch` to use the CUDA-graphs backend instead.
|
||||||
|
- Uses `mlx-audio` on Apple Silicon and auto-maps `Qwen/...` model IDs to `mlx-community/...`, defaulting to the `6bit` MLX variant unless the model name already pins a suffix.
|
||||||
|
- Supports MLX quantization overrides on Apple Silicon via `--qwen3_tts_mlx_quantization bf16|4bit|6bit|8bit`.
|
||||||
|
- Keeps the existing voice-clone/custom-voice/voice-design handler flow intact.
|
||||||
|
- Defaults to the CustomVoice model with speaker `Aiden`, so no reference audio is required. Voice-clone/base models can still use `--qwen3_tts_ref_audio`.
|
||||||
|
|
||||||
|
Install notes for Linux GGML:
|
||||||
|
- The default PyPI `qwentts-cpp-python` wheel targets CUDA 12.8.
|
||||||
|
- If that wheel does not match your CUDA runtime, install one of the Hugging Face wheelhouse builds before installing `speech-to-speech`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install "qwentts-cpp-python==0.3.1+cu130" \
|
||||||
|
-f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu130
|
||||||
|
pip install speech-to-speech
|
||||||
|
```
|
||||||
|
|
||||||
|
Available wheelhouse directories include `cu124`, `cu128`, `cu130`, and `cpu`.
|
||||||
|
|
||||||
|
Example for Apple Silicon using the default 6-bit MLX variant:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--tts qwen3 \
|
||||||
|
--qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
|
||||||
|
--qwen3_tts_speaker Aiden
|
||||||
|
```
|
||||||
|
|
||||||
|
You can override the default and select `bf16`, `4bit`, or `8bit` explicitly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--tts qwen3 \
|
||||||
|
--qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
|
||||||
|
--qwen3_tts_mlx_quantization 4bit \
|
||||||
|
--qwen3_tts_speaker Aiden
|
||||||
|
```
|
||||||
|
|
||||||
|
For voice-cloning examples, an optional reference clip is retained at
|
||||||
|
[docs/examples/reference-audio.wav](../examples/reference-audio.wav). Supply an
|
||||||
|
appropriate base/voice-cloning model and `--qwen3_tts_ref_audio` explicitly;
|
||||||
|
CustomVoice mode does not need this clip. The repository does not include a TTS
|
||||||
|
benchmark script.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
### Low-latency GPU setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--stt whisper \
|
||||||
|
--tts pocket
|
||||||
|
```
|
||||||
|
|
||||||
|
### Apple Silicon setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m speech_to_speech.s2s_pipeline \
|
||||||
|
--local_mac_optimal_settings
|
||||||
|
```
|
||||||
|
|
||||||
|
`--tts pocket` and `--tts kokoro` are also valid options on macOS.
|
||||||
-608
@@ -1,608 +0,0 @@
|
|||||||
# Reasoning & Context Injection — Architettura Completa
|
|
||||||
|
|
||||||
Questo documento descrive **come funziona il "reasoning"** (il layer di intelligenza che decide cosa rispondere) nell'app Vocero/S2S e **in quali punti puoi iniettare contesto** (identità dell'agente, istruzioni, strumenti MCP, profilo chiamante, dati extra, ecc.).
|
|
||||||
|
|
||||||
Esistono **due modalità di reasoning** distinte:
|
|
||||||
|
|
||||||
| Modalità | Variabile env | Chi fa il "pensiero" | Flusso |
|
|
||||||
|---|---|---|---|
|
|
||||||
| **PROXY** *(default)* | `S2S_REASONING_TYPE=PROXY` | Il server realtime upstream (`ai-server`, Vocero, o qualunque backend dietro `S2S_PROXY_TARGET`) | Browser → WebSocket `/v1/realtime` → upstream (prompt + tools inviati in `session.update`) |
|
|
||||||
| **OPENAI** *(reasoning esterno)* | `S2S_REASONING_TYPE=OPENAI` | Una chiamata HTTP separata a OpenAI Chat Completions (es. `gpt-4.1-mini`) | Browser → `POST /api/reasoning` → OpenAI → risposta reiniettata nel realtime come messaggio utente forzato |
|
|
||||||
|
|
||||||
In **entrambe** le modalità le informazioni di contesto vengono assemblate lato client in un'unica stringa "istruzioni" (`effectiveReasoningInstructions`) e passate poi in modo diverso.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Panoramica del flusso completo (uno sguardo dall'alto)
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────────────────────────────────────────────────────────┐
|
|
||||||
│ /home/duxiter/s2s/.env │
|
|
||||||
│ S2S_PRIMARY_PROMPT S2S_AGENT_PROFILE S2S_REASONING_TYPE │
|
|
||||||
│ S2S_TOOL_EXECUTION_MCP_URL S2S_USER_PHONE_AUTH_URL │
|
|
||||||
│ OPENAI_API_KEY / BASE_URL / REASONING_MODEL / ORG / PROJECT │
|
|
||||||
│ S2S_PROXY_TARGET S2S_SEARCH_MCP etc. │
|
|
||||||
└───────────────────────────┬─────────────────────────────────────────────┘
|
|
||||||
│ caricato da start_ui.sh
|
|
||||||
▼
|
|
||||||
┌─────────────────────────────────────────────────────────────────────────┐
|
|
||||||
│ demo/server.py (backend Python FastAPI) │
|
|
||||||
│ │
|
|
||||||
│ fetchConfig() GET /api/config ←─────── GET /api/user/phone │
|
|
||||||
│ │ │ (se URL param phone) │
|
|
||||||
│ ├─ primaryPrompt (priorità 1: Aquachile /agents/profile │
|
|
||||||
│ │ priorità 2: S2S_PRIMARY_PROMPT env) │
|
|
||||||
│ ├─ agentProfile (URL di dove abbiamo preso il prompt) │
|
|
||||||
│ ├─ agentTools[] (catalogo MCP Aquachile, enabled=True) │
|
|
||||||
│ ├─ reasoningType = "PROXY" oppure "OPENAI" │
|
|
||||||
│ └─ searchMcp, s2sUrl, lb, auth, search, ecc. │
|
|
||||||
│ │
|
|
||||||
│ POST /api/reasoning (solo se reasoningType=OPENAI) │
|
|
||||||
│ body → httpx POST {OPENAI_BASE_URL}/chat/completions │
|
|
||||||
│ │
|
|
||||||
│ POST /api/mcp/tool │
|
|
||||||
│ body → httpx POST {S2S_TOOL_EXECUTION_MCP_URL} + chiavi API │
|
|
||||||
│ │
|
|
||||||
│ WS /v1/realtime (proxy a S2S_PROXY_TARGET) │
|
|
||||||
│ client ↔ proxy ↔ upstream: logga [INPUT UTENTE]/[OUTPUT AGENTE] │
|
|
||||||
└───────────────────────────┬─────────────────────────────────────────────┘
|
|
||||||
│ fetch su caricamento pagina
|
|
||||||
▼
|
|
||||||
┌─────────────────────────────────────────────────────────────────────────┐
|
|
||||||
│ demo/main.js (frontend browser) │
|
|
||||||
│ │
|
|
||||||
│ pinnedPrompt = primaryPrompt (da /api/config) — read-only se valoriz.│
|
|
||||||
│ agentToolDefs[] = agentTools (da /api/config) │
|
|
||||||
│ reasoningType = "PROXY" | "OPENAI" │
|
|
||||||
│ voceroUserPhoneNumber = new URL(location).searchParams.phoneNumber │
|
|
||||||
│ │ │
|
|
||||||
│ └─► async fetchVoceroUserProfile() │
|
|
||||||
│ GET api/user/phone?phoneNumber=… → voceroUserProfile{} │
|
|
||||||
│ │
|
|
||||||
│ Quando l'utente clicca l'orb → doStart(): │
|
|
||||||
│ startInstructions = effectiveInstructions() │
|
|
||||||
│ startTools = activeToolDefs() │
|
|
||||||
│ │ │
|
|
||||||
│ └─► new S2sWsRealtimeClient({instructions, tools, …}) │
|
|
||||||
│ │ │
|
|
||||||
│ ├─► session.update al realtime upstream │
|
|
||||||
│ │ (se reasoningType=PROXY → qui pensa) │
|
|
||||||
│ │ │
|
|
||||||
│ └─► event "transcript" (user, partial=false) │
|
|
||||||
│ │ │
|
|
||||||
│ └─► maybeRunOpenAiReasoning() │
|
|
||||||
│ (solo reasoningType=OPENAI) │
|
|
||||||
│ → POST /api/reasoning → answer │
|
|
||||||
│ → sendUserText(renderPrompt) │
|
|
||||||
│ → requestResponse() │
|
|
||||||
└─────────────────────────────────────────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Assemblaggio del prompt: `effectiveReasoningInstructions()`
|
|
||||||
|
|
||||||
La funzione **centrale** che costruisce il blocco di testo che verrà dato al modello è in [main.js:516-533](file:///home/duxiter/s2s/demo/main.js#L516-L533):
|
|
||||||
|
|
||||||
```js
|
|
||||||
function effectiveReasoningInstructions() {
|
|
||||||
const base = pinnedPrompt || settings.instructions;
|
|
||||||
let instructions = base;
|
|
||||||
if (activeToolDefs().length) instructions += TOOL_USE_HINT;
|
|
||||||
if (searchMcpEnabled && agentToolDefs.length) {
|
|
||||||
instructions += MCP_SEARCH_HINT;
|
|
||||||
instructions += ` Available MCP tools: ${buildMcpToolCatalogSummary(agentToolDefs)}`;
|
|
||||||
}
|
|
||||||
const suggested = topSuggestedAgentTools(3);
|
|
||||||
if (searchMcpEnabled && suggested.length) {
|
|
||||||
instructions += ` Current user request: "${lastUserQuery}". Client-side ranking hint only: …`;
|
|
||||||
}
|
|
||||||
const userContext = voceroUserContextText();
|
|
||||||
if (userContext) instructions += userContext;
|
|
||||||
return instructions;
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Ordine di concatenazione (dalla priorità più alta alla più bassa):
|
|
||||||
|
|
||||||
| # | Fonte | Dove viene da | Note |
|
|
||||||
|---|---|---|---|
|
|
||||||
| 1 | **`pinnedPrompt`** | `S2S_AGENT_PROFILE` (`instructions` campo Aquachile) → fallback `S2S_PRIMARY_PROMPT` env → fallback `settings.instructions` (localStorage) → fallback `DEFAULT_INSTRUCTIONS` | **Identità e personalità principale dell'agente**. Quando `pinnedPrompt` è presente, il campo `instructions` di Settings è disabilitato in UI (read-only). |
|
|
||||||
| 2 | **`TOOL_USE_HINT`** | Costante in linea [main.js:32-36](file:///home/duxiter/s2s/demo/main.js#L32-L36) | Aggiunto se c'è **almeno 1 tool attivo**: "*When the user's request calls for one of your tools, do not describe your capabilities or say you can do it and wait for another turn. Instead, say a brief acknowledgement… and call the tool right away in the same response.*" |
|
|
||||||
| 3 | **`MCP_SEARCH_HINT`** + catalogo completo tools | Condizionato a `S2S_SEARCH_MCP=true` in env | Istruisce il modello a leggere l'intero catalogo MCP e scegliere lo strumento migliore. Poi **appende come testo TUTTI i tool MCP** (`buildMcpToolCatalogSummary`) con nome, descrizione e parametri obbligatori. |
|
|
||||||
| 4 | **Suggerimento tool client-side (top 3)** | `topSuggestedAgentTools(3)` basato su `lastUserQuery` | Messaggio "Client-side ranking hint only: 1. get_day_report: … 2. … 3. …". Usato **solo come hint secondario**, il modello deve sempre controllare il catalogo completo prima. |
|
|
||||||
| 5 | **`voceroUserContextText()`** | Profile restituito da `/api/user/phone?phoneNumber=…` | Stringa: `" Caller profile resolved from the page phoneNumber: {JSON profilo}. Use this as trusted caller context when relevant."` (troncato a 1600 char) |
|
|
||||||
|
|
||||||
Poi la funzione **`effectiveInstructions()`** ([main.js:537-539](file:///home/duxiter/s2s/demo/main.js#L537-L539)) prende il risultato e:
|
|
||||||
- Se `reasoningType=OPENAI` → **appende `OPENAI_RENDER_HINT`**: istruisce il realtime a **rispondere solo ai messaggi che iniziano con il marker `[VOCERO_RENDER_REPLY]`**, in modalità "leggi e basta" (niente tool, niente ragionamento). Questo è il segreto che permette di scrivere la frase pensata da OpenAI e farla ripetere fedelmente in TTS.
|
|
||||||
- Se `reasoningType=PROXY` → ritorna le istruzioni tal quali.
|
|
||||||
|
|
||||||
### Quando viene ricalcolato e reinviato al modello
|
|
||||||
|
|
||||||
- **Alla connessione** in `doStart()`: `startInstructions = effectiveInstructions()` → `new S2sWsRealtimeClient({instructions: startInstructions, …})`.
|
|
||||||
- **Durante la chiamata**, in risposta a:
|
|
||||||
- ogni **trascrizione utente** parziale o finale (linee [1122-1128](file:///home/duxiter/s2s/demo/main.js#L1122-L1128)): `pushLiveToolPolicy()` + `pushLiveInstructions(effectiveInstructions())` → `client.updateSession({ instructions })`
|
|
||||||
- quando arriva il **profilo chiamante da phone auth** ([2388-2390](file:///home/duxiter/s2s/demo/main.js#L2388-L2390)): stesso `pushLiveInstructions`
|
|
||||||
- quando l'utente **modifica le impostazioni** nel form e salva.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. I due motori di reasoning, nel dettaglio
|
|
||||||
|
|
||||||
### 3.1 Modalità **PROXY** (default — consigliata con Aquachile)
|
|
||||||
|
|
||||||
```
|
|
||||||
S2S_REASONING_TYPE=PROXY (o qualsiasi valore diverso da "OPENAI")
|
|
||||||
S2S_PROXY_TARGET=ws://ai-server:12345/v1/realtime # o qualunque endpoint WS realtime
|
|
||||||
```
|
|
||||||
|
|
||||||
In questa modalità, **l'upstream realtime è l'unico a "pensare"**. Tutta l'intelligenza risiede nel backend puntato da `S2S_PROXY_TARGET` (es. Vocero, un wrapper realtime custom, un server che parla OpenAI Realtime API).
|
|
||||||
|
|
||||||
Flusso **testuale** di quello che succede a ogni avvio:
|
|
||||||
|
|
||||||
1. `doStart()` calcola `startInstructions = effectiveInstructions()` (tutto il blocco §2).
|
|
||||||
2. Costruisce `startTools = activeToolDefs()` = tool browser (web_search, camera_snapshot) **+** `agentToolDefs` (catalogo MCP Aquachile).
|
|
||||||
3. Passa `instructions` e `tools` al costruttore di `S2sWsRealtimeClient`.
|
|
||||||
4. Quando il WS si connette e arriva `session.created`, il client manda `session.update` con:
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"type": "realtime",
|
|
||||||
"instructions": "<tutte le istruzioni assemblate>",
|
|
||||||
"tools": [ { "type":"function", "name":"…", "description":"…", "parameters":{…} }, … ],
|
|
||||||
"tool_choice": "auto",
|
|
||||||
"audio": { "output": { "voice": "Aiden" } }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
5. **TUTTO IL RESTO è SULL'UPSTREAM**: L'upstream decide quando fare tool call, quando parlare, quando ascoltare. Il browser è solo uno "schermo+microfono+telecomando".
|
|
||||||
6. Quando l'upstream fa `conversation.item.input_audio_transcription.completed` il log server stampa `[INPUT UTENTE] …`.
|
|
||||||
7. Quando l'upstream fa `response.*_transcript.done` / `response.content_part.done` ecc. il log server stampa `[OUTPUT AGENTE] …`.
|
|
||||||
|
|
||||||
**Vantaggi:**
|
|
||||||
- Un solo round trip per tutto (latenza più bassa per voce→voce).
|
|
||||||
- L'upstream può usare qualsiasi modello/rete interna (non per forza OpenAI).
|
|
||||||
- Le MCP tool call sono orchestrate dallo stesso server che fa STT/TTS → meno desincronizzazioni.
|
|
||||||
|
|
||||||
**Svantaggi:**
|
|
||||||
- Tutto il "pensiero" è black-box sull'upstream; non puoi vedere i token del thinking né usare un modello diverso senza cambiare upstream.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 3.2 Modalità **OPENAI** (reasoning HTTP separato)
|
|
||||||
|
|
||||||
```
|
|
||||||
S2S_REASONING_TYPE=OPENAI
|
|
||||||
OPENAI_API_KEY=sk-… # obbligatorio
|
|
||||||
OPENAI_BASE_URL=https://api.openai.com/v1 # default
|
|
||||||
OPENAI_REASONING_MODEL=gpt-4.1-mini # default
|
|
||||||
OPENAI_ORGANIZATION=… # opzionale
|
|
||||||
OPENAI_PROJECT=… # opzionale
|
|
||||||
```
|
|
||||||
|
|
||||||
In questa modalità il **ragionamento vero e proprio** non viene fatto dall'upstream realtime. Invece:
|
|
||||||
|
|
||||||
1. L'upstream fa solo **STT** (converte voce in testo) e **TTS** (converte testo in voce). Agisce come un "dumb" audio I/O.
|
|
||||||
2. **Non appena arriva una trascrizione utente COMPLETA** (evento `transcript`, `role=="user"`, `partial==false`), `maybeRunOpenAiReasoning(transcript, itemId)` viene triggerato.
|
|
||||||
3. Questo chiama `fetchOpenAiReasoningAnswer(transcript)` ([main.js:563-586](file:///home/duxiter/s2s/demo/main.js#L563-L586)) che fa:
|
|
||||||
```
|
|
||||||
POST /api/reasoning
|
|
||||||
body = {
|
|
||||||
transcript: "quanto è il report giornaliero?",
|
|
||||||
instructions: effectiveReasoningInstructions(), // stesse di PROXY
|
|
||||||
language: "es-CL" o vuoto,
|
|
||||||
user_context: voceroUserContextText(), // duplicato per sicurezza
|
|
||||||
}
|
|
||||||
```
|
|
||||||
4. Il server (endpoint in [server.py:635-710](file:///home/duxiter/s2s/demo/server.py#L635-L710)) a sua volta fa:
|
|
||||||
```
|
|
||||||
POST {OPENAI_BASE_URL}/chat/completions
|
|
||||||
body = {
|
|
||||||
model: OPENAI_REASONING_MODEL,
|
|
||||||
messages: [
|
|
||||||
{ role:"system",
|
|
||||||
content: "{instructions}\n\n{user_context}\nReply only with final answer. Language=…" },
|
|
||||||
{ role:"user", content: transcript }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
5. Riceve `answer` (stringa semplice, pensata per essere "parlata"), logga `[Vocero REASONING] transcript=… answer=…` e la restituisce al client.
|
|
||||||
6. Il client costruisce `renderPrompt = "[VOCERO_RENDER_REPLY]\n" + answer` e lo **manda come se fosse un testo scritto dall'utente** al realtime:
|
|
||||||
```
|
|
||||||
client.sendUserText(renderPrompt);
|
|
||||||
client.requestResponse();
|
|
||||||
```
|
|
||||||
7. L'upstream realtime, a causa di `OPENAI_RENDER_HINT` nel prompt, **vede il marker `[VOCERO_RENDER_REPLY]`** e quindi:
|
|
||||||
- NON usa tool, NON fa ragionamento proprio, NON modifica la frase.
|
|
||||||
- Si limita a leggere la parte dopo il marker in TTS.
|
|
||||||
8. Risultato: l'utente sente "la voce dell'agente" (TTS dell'upstream) ma il testo **è stato scritto da OpenAI** in `/chat/completions`.
|
|
||||||
|
|
||||||
Log server corrispondenti:
|
|
||||||
```
|
|
||||||
[EXT][OPENAI] reasoning POST https://api.openai.com/v1/chat/completions model=gpt-4.1-mini systemLen=… transcriptLen=…
|
|
||||||
[EXT][OPENAI] reasoning RESP status=200 len=…
|
|
||||||
[EXT][OPENAI] reasoning USAGE prompt=1234 completion=89 total=1323
|
|
||||||
INFO [s2s] [Vocero REASONING] transcript=… answer=…
|
|
||||||
```
|
|
||||||
|
|
||||||
**Vantaggi:**
|
|
||||||
- Scegli tu **qualsiasi modello compatibile Chat Completions** (anche Azure, Together, self-hosted se l'endpoint parla lo stesso protocollo).
|
|
||||||
- Hai **usage e log del reasoning separati** da quelli dell'audio.
|
|
||||||
- Puoi aggiungere facilmente post-processing, cache o fallback.
|
|
||||||
|
|
||||||
**Svantaggi:**
|
|
||||||
- Due round trip invece di uno → un po' più di latenza totale (~200-800 ms).
|
|
||||||
- Più costoso in token: `instructions` + contesto utente vengono mandati **due volte** (una a OpenAI, una al realtime).
|
|
||||||
- Le tool call che il modello OpenAI vorrebbe fare non possono essere usate direttamente — in OPENAI mode `chat/completions` NON ha accesso a `tools` (al momento l'endpoint `POST /api/reasoning` non inoltra il catalogo MCP a OpenAI). **Quindi in OPENAI mode, le MCP Aquachile vengono ancora passate al realtime (che in questa modalità non "pensa" ma solo reinderizza il testo), quindi non sono usabili.**
|
|
||||||
|
|
||||||
> ⚠️ Importante: se usi Aquachile con il suo ecosistema MCP, **la modalità PROXY è quella corretta da usare**. OPENAI mode è pensata per scenari "solo testo + voce" senza tool complessi.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. TUTTI i modi per iniettare contesto all'agente, dal più semplice al più avanzato
|
|
||||||
|
|
||||||
I punti di injection sono ordinati per **priorità di applicazione** (chi "vince" in caso di conflitto) e per **facilità di modifica**.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 🅿️ Punto 1 — `.env` (variabili di ambiente, configurazione statica)
|
|
||||||
|
|
||||||
File: [.env](file:///home/duxiter/s2s/.env)
|
|
||||||
Caricato in memoria da [start_ui.sh](file:///home/duxiter/s2s/start_ui.sh#L8-L12) (`source "$ROOT_DIR/.env"`) prima di avviare uvicorn.
|
|
||||||
|
|
||||||
| Variabile | Tipo | Cosa fa all'agente |
|
|
||||||
|---|---|---|
|
|
||||||
| **`S2S_PRIMARY_PROMPT`** | testo lungo | **Prompt backup quando Aquachile è irraggiungibile o `S2S_AGENT_PROFILE` non è impostato.** Sarà il valore di `primaryPrompt` in `/api/config` se `_extract_agent_prompt()` torna stringa vuota. Supporta `\n` → convertiti in newline. |
|
|
||||||
| **`S2S_AGENT_PROFILE`** | URL | Endpoint `GET` (es. Aquachile: `https://aquachile.azurianlab.com/api/v1/agents/profile`). Deve ritornare JSON `{ agent: { instructions: "...", tools: [ {name, description, parameters, enabled}, ... ] } }`. **Ha priorità su `S2S_PRIMARY_PROMPT`** — vedi `primary_prompt = agent_prompt or S2S_PRIMARY_PROMPT` in [server.py:517](file:///home/duxiter/s2s/demo/server.py#L517). |
|
|
||||||
| **`S2S_TOOL_EXECUTION_MCP_URL`** | URL | Endpoint `POST` per **eseguire** le tool call MCP (stesso backend Aquachile). Riceve `{server, name, arguments}` e viene chiamata ogni volta che il modello invoca uno strumento. |
|
|
||||||
| `S2S_TOOL_EXECUTION_MCP_API_KEY` / `S2S_TOOL_EXECUTION_MCP_API_SECRET` | stringhe segrete | Intestazioni `X-API-KEY` e `X-API-SECRET` aggiunte a ogni chiamata MCP. |
|
|
||||||
| **`S2S_USER_PHONE_AUTH_URL`** | URL | Endpoint `GET` (con `?phoneNumber=…`) che restituisce il profilo del chiamante. Se l'URL della UI contiene `?phoneNumber=+56…`, questo endpoint viene chiamato e il JSON risultante viene appeso al prompt di sistema (vedi Punto 4). |
|
|
||||||
| **`S2S_SEARCH_MCP=true\|false`** | bool | Quando `true`, appende `MCP_SEARCH_HINT` + il catalogo COMPLETO dei tool MCP dentro le `instructions`. Fondamentale se vuoi che il modello sappia che esistono 17 strumenti e possa scegliere tra quelli. |
|
|
||||||
| **`S2S_REASONING_TYPE`** | `PROXY` \| `OPENAI` | Motore di reasoning (§3). Normalizzato con `_normalize_reasoning_type()` (case-insensitive). |
|
|
||||||
| **`OPENAI_API_KEY` / `BASE_URL` / `REASONING_MODEL`** | stringhe | Solo per mode OPENAI. `BASE_URL` serve anche per compatibilità Azure/Together (basta che parli /chat/completions). |
|
|
||||||
| **`S2S_PROXY_TARGET`** | URL WS | Upstream realtime che fa STT/TTS + (in mode PROXY) il reasoning vero e proprio. Di solito `ws://ai-server:12345/v1/realtime`. |
|
|
||||||
| `S2S_PRELOAD_DATA_ENDPOINT`, `SERPER_API_KEY`, `LOAD_BALANCER_URL`, `SPEECH_TO_SPEECH_URL` | varie | Non iniettano testo nel prompt direttamente, ma abilitano altre funzionalità (PowerBI preload, web search Serper, queue & limiter, direct URL fissa). |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 🅿️ Punto 2 — **Aquachile `/api/v1/agents/profile`** (contesto Aziendale + Tools)
|
|
||||||
|
|
||||||
URL impostato da `S2S_AGENT_PROFILE`. Fetchato lato server in [_agent_profile_data()](file:///home/duxiter/s2s/demo/server.py#L357-L397) con:
|
|
||||||
- cache 60 secondi globale (`_agent_profile_cached`), lock asincrona
|
|
||||||
- timeout 10 s
|
|
||||||
- chiamata GET senza header speciali (aggiungibili se serve a mano nel codice)
|
|
||||||
|
|
||||||
**Formato JSON atteso** (si adatta in modo tollerante):
|
|
||||||
|
|
||||||
```jsonc
|
|
||||||
{
|
|
||||||
"agent": {
|
|
||||||
// ↓ usato come prompt principale (priority #1)
|
|
||||||
"instructions": "Eres el asistente de voz oficial de Aquachile. Te llamas Aquachile Assistant…",
|
|
||||||
// ↓ fallback se instructions non esiste
|
|
||||||
"description": "…",
|
|
||||||
"tools": [
|
|
||||||
{
|
|
||||||
"name": "get_day_report",
|
|
||||||
"description": "Obtiene el reporte de producción diario de una planta…",
|
|
||||||
"parameters": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"center": {"type": "string", "description": "Nombre de la planta"}
|
|
||||||
},
|
|
||||||
"required": []
|
|
||||||
},
|
|
||||||
"enabled": true // ↓ se missing o true → incluso; se false → ignorato
|
|
||||||
},
|
|
||||||
// … altri 16 strumenti
|
|
||||||
]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Parser corrispondente:
|
|
||||||
- [_extract_agent_prompt()](file:///home/duxiter/s2s/demo/server.py#L405-L424) — cerca in ordine: `agent.instructions` → `agent.description` → `instructions` radice → `description` radice.
|
|
||||||
- [_extract_agent_tools()](file:///home/duxiter/s2s/demo/server.py#L427-L437) — cerca `agent.tools[]`, filtra quelli con `enabled` != `false`.
|
|
||||||
|
|
||||||
Tutto il resto del payload viene ignorato. Se vuoi estrarre altri campi (es. `agent.name`, `agent.greeting`, `agent.language`) puoi aggiungerne l'estensione in `_extract_agent_prompt` per concatenarli, oppure creare un nuovo campo nel dizionario di ritorno e poi usarlo in `/api/config`.
|
|
||||||
|
|
||||||
**Quando modificare questo endpoint piuttosto che altre parti:**
|
|
||||||
- Quando il contenuto cambia spesso (ogni giorno: nuovi report, nuove tipologie di domanda).
|
|
||||||
- Quando vuoi che **tutte le installazioni** dell'UI aggiornino all'istante senza rebuild/restart del server.
|
|
||||||
- Quando vuoi rilasciare un **nuovo tool MCP**: basta aggiungerlo all'array `tools[]` e comparirà ovunque entro 60 s (prossima invalidazione della cache).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 🅿️ Punto 3 — **URL parameters** della pagina (contesto per-chiamata)
|
|
||||||
|
|
||||||
Analizzati all'avvio di `main.js` ([riga 410](file:///home/duxiter/s2s/demo/main.js#L410)):
|
|
||||||
```js
|
|
||||||
const pageParams = new URLSearchParams(window.location.search);
|
|
||||||
const voceroUserPhoneNumber = (pageParams.get("phoneNumber") || "").trim();
|
|
||||||
```
|
|
||||||
|
|
||||||
Attualmente è gestito un solo param:
|
|
||||||
|
|
||||||
| Parametro URL | Tipo | Effetto |
|
|
||||||
|---|---|---|
|
|
||||||
| `?phoneNumber=+56912345678` | stringa E.164 | Triggera `fetchVoceroUserProfile()` → `GET /api/user/phone?phoneNumber=…` → endpoint Aquachile `S2S_USER_PHONE_AUTH_URL` → profilo JSON appenduto al prompt (vedi Punto 4). |
|
|
||||||
|
|
||||||
È molto semplice estendere la lista: aggiungi una riga tipo:
|
|
||||||
```js
|
|
||||||
const plantId = (pageParams.get("plant") || "").trim();
|
|
||||||
```
|
|
||||||
e poi inietta il valore in modo statico (in `voceroUserContextText()` o in un pezzo di testo a parte in `effectiveReasoningInstructions`) o fai una fetch al tuo endpoint.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 🅿️ Punto 4 — **Profilo chiamante `/api/user/phone`** (contesto per-caller)
|
|
||||||
|
|
||||||
End-to-end:
|
|
||||||
|
|
||||||
```
|
|
||||||
URL: https://<app>/?phoneNumber=+56912345678
|
|
||||||
│
|
|
||||||
▼ fetchVoceroUserProfile() [main.js:2370-2398]
|
|
||||||
GET api/user/phone?phoneNumber=%2B56912345678
|
|
||||||
│
|
|
||||||
▼ user_by_phone() [server.py:539-573]
|
|
||||||
GET {S2S_USER_PHONE_AUTH_URL}?phoneNumber=%2B56912345678
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
Risposta Aquachile (es.)
|
|
||||||
{
|
|
||||||
"id": 987,
|
|
||||||
"name": "Don Manuel",
|
|
||||||
"role": "Jefe de Planta",
|
|
||||||
"center": "Puerto Montt",
|
|
||||||
"email": "manuel@aquachile.cl",
|
|
||||||
"authorizations": ["report:daily", "report:weekly"]
|
|
||||||
}
|
|
||||||
│
|
|
||||||
▼ { "result": <payload> } al browser
|
|
||||||
│
|
|
||||||
▼ voceroUserProfile = <payload> ; voceroUserProfileLoaded = true
|
|
||||||
│
|
|
||||||
▼ voceroUserContextText() [main.js:454-458]
|
|
||||||
" Caller profile resolved from the page phoneNumber: {name:\"Don Manuel\",…}. Use this as trusted caller context when relevant."
|
|
||||||
│
|
|
||||||
└─► concatenato ALLA FINE di effectiveReasoningInstructions
|
|
||||||
```
|
|
||||||
|
|
||||||
Una volta che il profilo è arrivato, viene chiamato `pushLiveInstructions(effectiveInstructions())` → l'istruzione sul realtime viene **aggiornata in volo** anche se la conversazione è già iniziata. Quindi anche se il profilo impiega 2 secondi ad arrivare, le battute successive alla ricezione beneficeranno del contesto.
|
|
||||||
|
|
||||||
**Quando usarlo:** passare dati per-caller (ruolo, permessi, centro, nome, lingua preferita, reporting line, storico chiamate recenti ecc.). Tieni conto che il JSON viene serializzato in chiaro nel prompt (max 1600 caratteri), quindi non passare PII sensibili in chiaro.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 🅿️ Punto 5 — **Catalogo MCP Aquachile** (contesto come "abilità" dell'agente)
|
|
||||||
|
|
||||||
Fonte: `agent.tools[]` dal profilo Aquachile (Punto 2), poi in `agentToolDefs` lato client.
|
|
||||||
|
|
||||||
Entrano nel prompt in due modi:
|
|
||||||
|
|
||||||
1. **Come `tools` (dichiarazione formale OpenAI Realtime function-call)** : `S2sWsRealtimeClient({tools: activeToolDefs()})` → arrivano all'upstream come `session.tools[]` con schema JSON Schema. Questo permette all'upstream di **decidere di chiamarli** tramite `response.create_response + function_call_arguments.done`.
|
|
||||||
2. **Come testo nel prompt vero e proprio** : `Available MCP tools: get_day_report - Obtiene el reporte… Required parameters: []; get_metrics_summary - …. ` (solo se `S2S_SEARCH_MCP=true`).
|
|
||||||
|
|
||||||
Quando poi il modello effettivamente **chiama** un tool MCP, il client:
|
|
||||||
- emette l'evento `toolcall`
|
|
||||||
- il server chiama `POST /api/mcp/tool` ([server.py](file:///home/duxiter/s2s/demo/server.py#L576…)):
|
|
||||||
```
|
|
||||||
POST {S2S_TOOL_EXECUTION_MCP_URL}
|
|
||||||
headers: X-API-KEY, X-API-SECRET, Content-Type: application/json
|
|
||||||
body: {"server": "mcp", "name": "get_day_report", "arguments": "{\"center\":\"Puerto Montt\"}"}
|
|
||||||
```
|
|
||||||
- Il risultato torna al client e viene rinviato all'upstream come `conversation.item` di tipo `function_call_output`. Il modello legge l'output e prosegue il turn.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 🅿️ Punto 6 — **Settings utente (localStorage)** (solo se prompt non pinned)
|
|
||||||
|
|
||||||
Se `pinnedPrompt` è stringa vuota (né da Aquachile, né da env), allora in `effectiveReasoningInstructions()` la base è `settings.instructions` che viene da:
|
|
||||||
|
|
||||||
```js
|
|
||||||
// main.js loadSettings: riga 201
|
|
||||||
instructions: localStorage.getItem("s2s.ws.instructions") || DEFAULT_INSTRUCTIONS
|
|
||||||
```
|
|
||||||
|
|
||||||
Nel form Settings (pannellino in alto a destra) l'utente può editare il testo "System instructions" e salvarlo → va in localStorage.
|
|
||||||
|
|
||||||
⚠️ **Attenzione**: se è presente un prompt "pinned" (da Aquachile o env), il campo nel Settings diventa **read-only** (non è modificabile). Puoi sempre verificare lo stato aprendo il pannello.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 🅿️ Punto 7 — **Aggiungere contesto custom via codice (patch diretta)**
|
|
||||||
|
|
||||||
Se vuoi iniettare dati specifici (es. l'ID di una campagna, il turno di lavoro corrente, la stazione meteorologica più vicina, ecc.) senza creare un nuovo endpoint esterno, hai **3 punti comodi** dove patchare con poche righe:
|
|
||||||
|
|
||||||
#### 7a. Aggiungere un pezzo di testo statico alle `instructions`
|
|
||||||
|
|
||||||
Modifica direttamente `effectiveReasoningInstructions()` in [main.js:516-533](file:///home/duxiter/s2s/demo/main.js#L516-L533), aggiungendo una sezione. Esempio:
|
|
||||||
|
|
||||||
```js
|
|
||||||
function effectiveReasoningInstructions() {
|
|
||||||
const base = pinnedPrompt || settings.instructions;
|
|
||||||
let instructions = base;
|
|
||||||
if (activeToolDefs().length) instructions += TOOL_USE_HINT;
|
|
||||||
if (searchMcpEnabled && agentToolDefs.length) { … }
|
|
||||||
const suggested = topSuggestedAgentTools(3);
|
|
||||||
if (searchMcpEnabled && suggested.length) { … }
|
|
||||||
const userContext = voceroUserContextText();
|
|
||||||
if (userContext) instructions += userContext;
|
|
||||||
// ↓ aggiungi contesto custom QUI
|
|
||||||
instructions += `\n\nTurno actual: ${TURNO_CORRIENTE}. Campaña: ${CAMPAIGN_ID}. Planta: ${PLANTA_ID}.`;
|
|
||||||
return instructions;
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Funziona in **entrambe** le modalità (PROXY e OPENAI).
|
|
||||||
|
|
||||||
#### 7b. Inviare un messaggio pre-filla a inizio conversazione
|
|
||||||
|
|
||||||
Dopo che il client è connesso e pronto (evento `ready` o `status=="connected"`), chiama:
|
|
||||||
```js
|
|
||||||
client.sendUserText(
|
|
||||||
"Contexto adicional de esta llamada (no responder a esto directamente): " +
|
|
||||||
"El usuario está llamando desde la planta Puerto Montt. Su último reporte " +
|
|
||||||
"fue el día 2026-08-17 y mostró un 95% de cumplimiento. El turno actual " +
|
|
||||||
"es el turno noche. Usa esta información cuando respondas."
|
|
||||||
);
|
|
||||||
// (opzionale) poi client.requestResponse() se vuoi che l'agente saluti subito
|
|
||||||
```
|
|
||||||
Il messaggio entra nella history del modello (coda `conversation.items[]`) ed è visibile per il resto della conversazione.
|
|
||||||
|
|
||||||
#### 7c. Patchare `/api/config` lato server per ritornare un campo custom e usarlo dal client
|
|
||||||
|
|
||||||
Aggiungi in [server.py ~riga 518-531](file:///home/duxiter/s2s/demo/server.py#L518-L531):
|
|
||||||
```python
|
|
||||||
out = {
|
|
||||||
...,
|
|
||||||
"myCustomContext": {"turno": current_shift(), "planta": os.environ.get("PLANTA_ID", "")},
|
|
||||||
}
|
|
||||||
```
|
|
||||||
Poi lato client in `fetchConfig()` (circa riga 2408):
|
|
||||||
```js
|
|
||||||
myCustomContext = json.myCustomContext || null;
|
|
||||||
```
|
|
||||||
e usa il valore come in 7a. Utile quando il contesto lo conosci solo lato server (es. viene da DB aziendale o da variabili env del deploy).
|
|
||||||
|
|
||||||
#### 7d. Aggiungere un parametro HTTP `user_context` custom alla chiamata `/api/reasoning` (solo OPENAI mode)
|
|
||||||
|
|
||||||
L'endpoint già accetta il campo `user_context` nel body della request — è già popolato da `voceroUserContextText()`, ma puoi aggiungere ulteriore testo concatenandolo nel client o nel server.
|
|
||||||
|
|
||||||
Oppure estendi `ReasoningRequest` in [server.py:346-350](file:///home/duxiter/s2s/demo/server.py#L346-L350) con un nuovo campo BaseModel (es. `turno: str = ""`, `planta: str = ""`) e poi usalo in `system_parts.append(...)` dentro `def reasoning()`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. Verifiche e Debug — come controllare "cosa vede veramente l'agente"
|
|
||||||
|
|
||||||
### 5a. Log del server (cosa arriva all'esterno)
|
|
||||||
|
|
||||||
Su `docker logs -f s2s_ui` (o sull'output di `start_ui.sh`) puoi monitorare TUTTE le chiamate esterne. Filtri utili:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Solo contesto caricato all'avvio (profilo Aquachile)
|
|
||||||
grep "\[EXT\]\[AQUACHILE\]" server.log
|
|
||||||
|
|
||||||
# Solo ragionamento OpenAI (se mode OPENAI)
|
|
||||||
grep "\[EXT\]\[OPENAI\]" server.log
|
|
||||||
|
|
||||||
# Dialogo parlato (TESTO della chat)
|
|
||||||
grep -E "INPUT UTENTE|OUTPUT AGENTE" server.log
|
|
||||||
|
|
||||||
# Tutte le chiamate MCP
|
|
||||||
grep "\[EXT\]\[AQUACHILE\].*mcp.*tools/call" server.log
|
|
||||||
|
|
||||||
# /api/config (quello che arriva al browser come initial setup)
|
|
||||||
grep "/api/config returned" server.log
|
|
||||||
|
|
||||||
# Autenticazione chiamante
|
|
||||||
grep "Vocero USER AUTH" server.log
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5b. Debug lato client (cosa viene assemblato)
|
|
||||||
|
|
||||||
Apri la console del browser (F12) e, dopo che la pagina ha caricato e ti sei connesso, puoi eseguire:
|
|
||||||
|
|
||||||
```js
|
|
||||||
// Controlla il tipo di reasoning attivo
|
|
||||||
> reasoningType
|
|
||||||
"PROXY" // oppure "OPENAI"
|
|
||||||
|
|
||||||
// Stringa effettiva di instruzioni (copiala/incolala per ispezione manuale)
|
|
||||||
> copy(effectiveInstructions())
|
|
||||||
|
|
||||||
// Testo che è stato passato all'ultimo session.update
|
|
||||||
> liveInstructionsSnapshot.length
|
|
||||||
32456
|
|
||||||
|
|
||||||
// Elenco dei tool attivi che l'agente "vede"
|
|
||||||
> activeToolDefs().map(t => t.name)
|
|
||||||
(19) ['web_search', 'camera_snapshot', 'get_day_report', 'get_weekly_report', 'get_monthly_report', 'get_report_detail', … ]
|
|
||||||
|
|
||||||
// Profilo chiamante caricato
|
|
||||||
> voceroUserProfileLoaded, voceroUserProfile
|
|
||||||
true, { id: 987, name: "Don Manuel", … }
|
|
||||||
|
|
||||||
// Top 3 suggeriti per l'ultima domanda
|
|
||||||
> topSuggestedAgentTools(3).map(x => `${x.tool.name}:${x.score}`)
|
|
||||||
["get_day_report:38", "get_metrics_summary:22", "get_report_detail:15"]
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5c. Cosa viene mandato a OpenAI in modalità OPENAI
|
|
||||||
|
|
||||||
Basta guardare il log `[EXT][OPENAI] reasoning POST … systemLen=… transcriptLen=…` e saprai già il numero di token di sistema. Se vuoi vedere l'esatto system message, puoi aggiungere temporaneamente:
|
|
||||||
```python
|
|
||||||
# dentro def reasoning() dopo la costruzione di body
|
|
||||||
logger.debug("Reasoning system message PREVIEW:\n%s",
|
|
||||||
body["messages"][0]["content"][:3000])
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. Priorità effettiva di tutte le sorgenti (chi vince se c'è conflitto)
|
|
||||||
|
|
||||||
Tabella riassuntiva che risponde a: *"Se due punti dicono cose diverse, quale vince?"*
|
|
||||||
|
|
||||||
| Cosa | Vincitore (priorità max → min) |
|
|
||||||
|---|---|
|
|
||||||
| **Identità / sistema prompt** | 1. `agent.instructions` da Aquachile `S2S_AGENT_PROFILE` → 2. `S2S_PRIMARY_PROMPT` env → 3. `settings.instructions` (localStorage utente) → 4. `DEFAULT_INSTRUCTIONS` hardcoded |
|
|
||||||
| **Catalogo MCP** | Aquachile `agent.tools[]` (unico fornitore). Gli unici tool aggiuntivi sono `web_search` e `camera_snapshot` (browser-native). |
|
|
||||||
| **Hint su come usare i tool** | Sempre `TOOL_USE_HINT` fisso se `tools>0`; se `searchMcpEnabled` si aggiunge `MCP_SEARCH_HINT` + catalogo testuale completo. |
|
|
||||||
| **Hint tool ranking (top 3)** | Calcolato lato client per ogni battuta; prompt dice "secondary signal, full catalog wins" |
|
|
||||||
| **Profilo chiamante** | `S2S_USER_PHONE_AUTH_URL` + URL param `phoneNumber` (nessun altro percorso attuale). |
|
|
||||||
| **Modalità di reasoning** | 1. `S2S_REASONING_TYPE` env normalizzato (`OPENAI` o default `PROXY`) |
|
|
||||||
| **URL endpoint upstream** | 1. `SPEECH_TO_SPEECH_URL` pinned direct → 2. `LOAD_BALANCER_URL` LB + queue → 3. URL inserito manualmente in Settings (solo allowDirect) |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7. Troubleshooting: "L'agente non usa il contesto che mi aspetto"
|
|
||||||
|
|
||||||
Checklist da fare in ordine:
|
|
||||||
|
|
||||||
1. **`/api/config` cosa restituisce?**
|
|
||||||
```
|
|
||||||
curl -s http://localhost:8000/api/config | python -m json.tool
|
|
||||||
```
|
|
||||||
Verifica: `primaryPrompt` è valorizzato e lungo? `agentTools` quanti elementi ha? `reasoningType` è quello che ti aspetti?
|
|
||||||
|
|
||||||
2. **Aquachile profile risponde?** Guarda `[EXT][AQUACHILE] agent_profile` nel log. Status 200? Cache HIT o MISS? `PARSED promptLen=… toolCount=…` ha i numeri attesi?
|
|
||||||
|
|
||||||
3. **Il browser ha `phoneNumber` nella URL?** Apri DevTools → Network e cerca `user/phone`. C'è una richiesta? Torna 200? Che JSON ritorna? Il `result` viene inserito in `voceroUserProfile` (check console).
|
|
||||||
|
|
||||||
4. **Hai cambiato qualcosa nel prompt in Aquachile ma non l'aggiornamento non si vede?** Il profilo ha cache 60 secondi. Aspetta o riavvia il server (la cache è in memoria).
|
|
||||||
|
|
||||||
5. **In modalità PROXY i tool MCP non vengono chiamati:**
|
|
||||||
- Verifica che il **tool catalogo testuale** sia nel prompt (controlla `copy(effectiveInstructions())` e cerca la stringa `Available MCP tools`).
|
|
||||||
- Verifica che `S2S_SEARCH_MCP=true`.
|
|
||||||
- Guarda nella console del browser `[Vocero MCP MATCH]` e `[Vocero MCP RANK]` — se nessuno score è sopra la soglia (default 20), l'auto-invocazione lato client non scatta; il modello potrebbe comunque chiamarlo via function-call standard.
|
|
||||||
|
|
||||||
6. **In modalità OPENAI non sente le MCP:** Corretto — attualmente `/api/reasoning` NON manda tools a OpenAI. Vedi §3.2. Se vuoi chiamare tool in OPENAI mode devi spezzare la catena: OpenAI deve dire quale tool chiamare, tu lo chiami via `/api/mcp/tool`, e poi reinietti il risultato in un nuovo reasoning turn. O, più semplicemente, usa la modalità PROXY quando servono i tool Aquachile.
|
|
||||||
|
|
||||||
7. **Output lungo o ripetitivo:** Inserisci in coda al prompt un blocco tipo: `Limita la risposta a 2 frasi e parla in modo naturale. Non ripetere la domanda dell'utente.` — funziona in entrambe le modalità.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 8. Riferimenti rapidi (file e linee)
|
|
||||||
|
|
||||||
| Concetto | File e righe |
|
|
||||||
|---|---|
|
|
||||||
| `.env` caricato | [start_ui.sh](file:///home/duxiter/s2s/start_ui.sh#L8-L12) |
|
|
||||||
| Variabili env parsate lato server | [server.py](file:///home/duxiter/s2s/demo/server.py#L71-L120) |
|
|
||||||
| `ReasoningRequest` body schema | [server.py:346-350](file:///home/duxiter/s2s/demo/server.py#L346-L350) |
|
|
||||||
| Fetch profilo Aquachile + cache 60 s | [_agent_profile_data](file:///home/duxiter/s2s/demo/server.py#L357-L397) |
|
|
||||||
| Parse istruzioni + tools dal JSON Aquachile | [_extract_agent_prompt](file:///home/duxiter/s2s/demo/server.py#L405-L424), [_extract_agent_tools](file:///home/duxiter/s2s/demo/server.py#L427-L437) |
|
|
||||||
| `/api/config` (bootstrap verso client) | [server.py:509-536](file:///home/duxiter/s2s/demo/server.py#L509-L536) |
|
|
||||||
| `/api/user/phone` (profilo chiamante) | [server.py:539-573](file:///home/duxiter/s2s/demo/server.py#L539-L573) |
|
|
||||||
| Esecuzione tool MCP (inoltro ad Aquachile) | [server.py /api/mcp/tool](file:///home/duxiter/s2s/demo/server.py#L576-L633) |
|
|
||||||
| `/api/reasoning` (OpenAI chat completions proxy) | [server.py:635-710](file:///home/duxiter/s2s/demo/server.py#L635-L710) |
|
|
||||||
| Proxy WS realtime, INPUT/OUTPUT text log | [_client_to_upstream](file:///home/duxiter/s2s/demo/server.py#L1112-L1171), [_upstream_to_client](file:///home/duxiter/s2s/demo/server.py#L1174-L1255) |
|
|
||||||
| `fetchConfig()` lato client | [main.js:2401-2439](file:///home/duxiter/s2s/demo/main.js#L2401-L2439) |
|
|
||||||
| `fetchVoceroUserProfile()` phone auth | [main.js:2370-2398](file:///home/duxiter/s2s/demo/main.js#L2370-L2398) |
|
|
||||||
| `effectiveReasoningInstructions()` assemblaggio prompt | [main.js:516-533](file:///home/duxiter/s2s/demo/main.js#L516-L533) |
|
|
||||||
| `effectiveInstructions()` (prompt per realtime, + OPENAI marker hint) | [main.js:537-539](file:///home/duxiter/s2s/demo/main.js#L537-L539) |
|
|
||||||
| Tool hint + MCP search hint costanti | [main.js:32-42](file:///home/duxiter/s2s/demo/main.js#L32-L42) |
|
|
||||||
| `voceroUserContextText()` caller profile injection | [main.js:454-458](file:///home/duxiter/s2s/demo/main.js#L454-L458) |
|
|
||||||
| `fetchOpenAiReasoningAnswer()` POST /api/reasoning client | [main.js:563-586](file:///home/duxiter/s2s/demo/main.js#L563-L586) |
|
|
||||||
| `maybeRunOpenAiReasoning()` trigger da transcript + inject `[VOCERO_RENDER_REPLY]` | [main.js:592-625](file:///home/duxiter/s2s/demo/main.js#L592-L625) |
|
|
||||||
| `pushLiveInstructions()` / `pushLiveToolPolicy()` update sessione live | [main.js:628-649](file:///home/duxiter/s2s/demo/main.js#L628-L649) |
|
|
||||||
| `doStart()` avvio sessione + invio istruzioni/tools iniziali | [main.js:2739-2788](file:///home/duxiter/s2s/demo/main.js#L2739-L2788) |
|
|
||||||
| Gestore transcript utente → trigger reasoning + ranking MCP | [main.js:2814-2821](file:///home/duxiter/s2s/demo/main.js#L2814-L2821) |
|
|
||||||
@@ -1,188 +0,0 @@
|
|||||||
# Demo S2S — schema di funzionamento
|
|
||||||
|
|
||||||
> Documento orientativo del modulo `demo/` (UI + proxy S2S + reasoning). Non sostituisce
|
|
||||||
> il codice: serve a trovare il proprio posto quando si apre il progetto.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Cosa c'è in `demo/`
|
|
||||||
|
|
||||||
Il modulo fa **tre cose contemporaneamente**:
|
|
||||||
|
|
||||||
| Ruolo | Cosa fa | Entry point |
|
|
||||||
|---|---|---|
|
|
||||||
| **HTTP server statico + API** | Serve l'UI, offre endpoint REST, gira un WebSocket proxy | [server.py](file:///home/duxiter/s2s/demo/server.py) |
|
|
||||||
| **Voice UI (browser)** | Orba, mic, transcriptioni, audio playback, pannelli | [index.html](file:///home/duxiter/s2s/demo/index.html) + [main.js](file:///home/duxiter/s2s/demo/main.js) |
|
|
||||||
| **Reasoning hook** | Inoltra transcript utente a un motore di ragionamento esterno (NODE_RED o OpenAI) | [server.py](file:///home/duxiter/s2s/demo/server.py#L658-L834) `/api/reasoning` |
|
|
||||||
|
|
||||||
La configurazione (URL backend S2S, tipo di reasoning, chiavi API) viene presa dal file `.env` nella root del progetto.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Architettura in un diagramma
|
|
||||||
|
|
||||||
```
|
|
||||||
┌──────────────────────────────────────────────────┐
|
|
||||||
│ BROWSER (demo UI) │
|
|
||||||
│ [index.html] → [main.js] (ESM type=module) │
|
|
||||||
│ │ ▲ │
|
|
||||||
│ │ audio + WS events │ transcript chunks │
|
|
||||||
│ ▼ │ │
|
|
||||||
│ ┌──────────────────────────────────────────┐ │
|
|
||||||
│ │ s2s-ws-client.js → WebSocket /v1/realtime │ │
|
|
||||||
│ │ + codec.js + orb-visualizer.js │ │
|
|
||||||
│ └──────────────────────────────────────────┘ │
|
|
||||||
└─────────────────────────┬────────────────────────┘
|
|
||||||
│ WebSocket /v1/realtime
|
|
||||||
▼
|
|
||||||
┌───────────────────────────────────────────────┐
|
|
||||||
│ server.py (FastAPI) │
|
|
||||||
│ │
|
|
||||||
│ ┌──────────────────────────────────────┐ │
|
|
||||||
│ │ realtime_proxy ← TUNNEL → upstream │ │
|
|
||||||
│ │ (WS relay puro, copia byte-per-byte)│ │
|
|
||||||
│ │ _client_to_upstream ─┐ │ │
|
|
||||||
│ │ _upstream_to_client ─┤ session_ctx │ │
|
|
||||||
│ │ session_ctx{user,id} │ shared dict │ │
|
|
||||||
│ └───────────────────────┴──────────────┘ │
|
|
||||||
│ │
|
|
||||||
│ endpoint REST: │
|
|
||||||
│ GET / → index.html │
|
|
||||||
│ GET /api/config → env flags │
|
|
||||||
│ POST /api/reasoning → NODE_RED / │
|
|
||||||
│ OPENAI │
|
|
||||||
│ POST /api/auth/* → Aquachile │
|
|
||||||
│ │
|
|
||||||
└──────┬──────────────────────────────┬─────────┘
|
|
||||||
│ POST /api/reasoning │ fire-and-forget
|
|
||||||
▼ ▼
|
|
||||||
┌──────────────────────┐ ┌───────────────────────────────┐
|
|
||||||
│ Node-RED (port 1881)│ │ Node-RED (port 1881) │
|
|
||||||
│ POST /api/reasoning │ │ POST /api/log/system-response │
|
|
||||||
│ log user transcript │ │ log assistant TTS + pairing │
|
|
||||||
└──────────────────────┘ └───────────────────────────────┘
|
|
||||||
^
|
|
||||||
│ forward 4 eventi realtime:
|
|
||||||
│ audio_transcript.done
|
|
||||||
│ content_part.done
|
|
||||||
│ output_item.done role=assistant
|
|
||||||
│ conversation.item.created role=assistant
|
|
||||||
```
|
|
||||||
|
|
||||||
> ### 📝 Nota importante sul flusso AUDIO con l'attuale `.env`
|
|
||||||
>
|
|
||||||
> Con `.env` attuale che ha `S2S_PROXY_TARGET=ws://ai-server:12345/v1/realtime`:
|
|
||||||
>
|
|
||||||
> ✅ **Sì, TUTTO l'AUDIO utente (PCM16 16kHz mono, eventi `input_audio_buffer.append`) arriva a `ws://ai-server:12345/v1/realtime`** — ma **non direttamente**: passa SEMPRE per il **tunnel 1:1 WebSocket di `server.py`** (`_client_to_upstream`), che copia byte-per-byte senza modifiche.
|
|
||||||
>
|
|
||||||
> ✅ Allo stesso modo l'AUDIO risposta (TTS, `response.output_audio.delta`) ritorna da `ai-server:12345` → tunnel `_upstream_to_client` → browser → playback nativo.
|
|
||||||
>
|
|
||||||
> ❌ **Cosa invece NON va a `ai-server:12345`** (canale HTTP separato parallelo, non tocca l'audio):
|
|
||||||
> - Il **transcript STT** viene mandato via HTTP a `http://localhost:1881/api/reasoning` (Node-RED, hook LOG_ONLY oggi)
|
|
||||||
> - Il **testo risposta TTS** viene mandato via HTTP a `http://localhost:1881/api/log/system-response` (Node-RED, forward fire-and-forget)
|
|
||||||
> - Chiamate **auth / profilo Aquachile** verso `aquachile.azurianlab.com`
|
|
||||||
>
|
|
||||||
> Questi 3 flussi REST sono **aggiuntivi** e **non alterano minimamente il tunnel audio**: browser ↔ server.py ↔ `ai-server:12345` rimane il percorso esclusivo per voce e risposta.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Cose importanti di [server.py](file:///home/duxiter/s2s/demo/server.py)
|
|
||||||
|
|
||||||
### File → ruoli
|
|
||||||
- **`/`** → StaticFast `StaticFiles` su `demo/` (index.html + assets)
|
|
||||||
- **`@app.middleware http` no_cache_static_assets** → `Cache-Control: no-store` per `.js/.html/.css/.mjs/.wasm` per sconfiggere cache browser durante sviluppo
|
|
||||||
- **`@app.get /api/config`** → manda al browser variabili env (`S2S_REASONING_TYPE`, `S2S_PROXY_TARGET`, auth, features). Il client decida da sé cosa mostrare in UI.
|
|
||||||
- **`@app.post /api/reasoning` ([L658-L834](file:///home/duxiter/s2s/demo/server.py#L658-L834))** → cuore del reasoning. Due modalità, scelta da env `S2S_REASONING_TYPE`:
|
|
||||||
- **Modalità A — NODE_RED**: inoltra `POST {NODE_RED_URL}/api/reasoning` timeout 30s. Accetta `log_only=true` + `answer=""` (Fase 1).
|
|
||||||
- **Modalità B — OPENAI**: chiamata diretta `POST $OPENAI_BASE_URL/chat/completions` (Fasi successive).
|
|
||||||
- **`@app.websocket /v1/realtime` ([L874-L980](file:///home/duxiter/s2s/demo/server.py#L874-L980))** → **tunnel S2S nativo**. Non traduce messaggi: copia + registra + forwarda. 2 task paralleli:
|
|
||||||
- `_client_to_upstream(ws, upstream, client_ip, session_ctx)`: browser → AI server
|
|
||||||
- `_upstream_to_client(ws, upstream, client_ip, session_ctx)`: AI server → browser
|
|
||||||
- `session_ctx = {last_user_transcript, last_item_id}` **dizionario mutabile condiviso** per accoppiare input↔output
|
|
||||||
- **`async def _forward_agent_text_to_nodered` ([L837-L873](file:///home/duxiter/s2s/demo/server.py#L837-L873))** → helper fire-and-forget. Tutti errori → `logger.debug` (non spezza il tunnel WS).
|
|
||||||
- **`_upstream_to_client` hook eventi ([L1305-L1500](file:///home/duxiter/s2s/demo/server.py#L1305-L1500))**: per ogni evento output agente avvia `asyncio.create_task(_forward_agent_text_to_nodered(...))` → non blocca lo streaming audio.
|
|
||||||
- **Sidecar Python locali**: [auth.py](file:///home/duxiter/s2s/demo/auth.py) (Aquachile profile+user+phone), [limiter.py](file:///home/duxiter/s2s/demo/limiter.py) (rate limiter).
|
|
||||||
|
|
||||||
### Ordine di chiamata in un turno vocale
|
|
||||||
1. Browser clicca orb → `ws /v1/realtime` → `realtime_proxy()` accetta
|
|
||||||
2. Browser manda `session.update` → `input_audio_buffer.append` in loop
|
|
||||||
3. Upstream genera `conversation.item.input_audio_transcription.completed` → server cattura → **`session_ctx["last_user_transcript"] = transcript`**
|
|
||||||
4. `_upstream_to_client` vede transcript STT completo → server invoca in parallelo **`POST /api/reasoning`** con quel transcript
|
|
||||||
5. Upstream manda `response.output_audio.delta` audio chunks → tunnelati diretti al browser → playback nativo
|
|
||||||
6. Upstream manda `response.output_audio_transcript.done` → server cattura → `create_task(_forward_agent_text_to_nodered({tts, source_event, user_transcript, …}))` → Node-RED SYSTEM_RESPONSE log
|
|
||||||
7. Audio finisce → turno chiuso
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Cose importanti del lato client ([main.js](file:///home/duxiter/s2s/demo/main.js))
|
|
||||||
|
|
||||||
### Moduli ESM importati (tutti con `?v=VERSIONE` per cache-bust)
|
|
||||||
```
|
|
||||||
main.js
|
|
||||||
├─ dom.js → helper DOM (create, qs, bubble, ecc.)
|
|
||||||
├─ chat.js → pannello messaggi + appendAssistantText
|
|
||||||
├─ account.js → pannello auth + profilo utente
|
|
||||||
└─ ws/s2s-ws-client.js
|
|
||||||
├─ codec.js → codec pcm + base64
|
|
||||||
└─ orb-visualizer.js → animazione orba su eventi volume/state
|
|
||||||
```
|
|
||||||
|
|
||||||
### Funzioni chiave
|
|
||||||
| Funzione | Ruolo |
|
|
||||||
|---|---|
|
|
||||||
| `fetchOpenAiReasoningAnswer(transcript)` | `POST /api/reasoning`, ritorna `{answer, log_only, raw}` |
|
|
||||||
| `maybeRunOpenAiReasoning(transcript, itemId)` | guardia "ho già processato questo turn?" (`processedReasoningTurns`). Se `result.log_only === true` → **early return, zero side-effect**. Altrimenti branch standard (Fasi successive). |
|
|
||||||
| `cancelActiveResponse()` | Fasi successive: ferma risposta corrente prima di inject custom |
|
|
||||||
| `appendAssistantText(answer)` / `sendUserText(renderMarker)+requestResponse` | Fasi successive: sostituisce testo risposta backend nativo |
|
|
||||||
| `speechSynthesis.speak(utterance)` | Fasi successive: TTS *browser* per answer custom, invece del backend nativo |
|
|
||||||
|
|
||||||
### Regole per non rompere Fase 1
|
|
||||||
- Se `S2S_REASONING_TYPE=NODE_RED` **e** Node-RED torna `log_only=true` → il client non tocca nulla. Zero `cancelActiveResponse`, zero `speechSynthesis.speak`, zero `conversation.item.create` manuali. La voce è quella del backend S2S nativo.
|
|
||||||
- **Cache busting triplo**: `index.html` meta tag → `Cache-Control no-store` middleware → query-string `?v=20260825003` su ogni import.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. Node-RED (Fase 1 — LOG_ONLY)
|
|
||||||
|
|
||||||
Risiede in [`s2s-reasoning-engine/flows.json`](file:///home/duxiter/s2s/s2s-reasoning-engine/flows.json). Espone 2 endpoint completamente indipendenti:
|
|
||||||
|
|
||||||
| Endpoint | Payload | Output Node-RED |
|
|
||||||
|---|---|---|
|
|
||||||
| `POST /api/reasoning` | `{transcript, language, phoneNumber, …}` | `[S2S NODE_RED LOG_ONLY] ts=… lang=… phone=… transcript="..."` |
|
|
||||||
| `POST /api/log/system-response` | `{tts, source_event, user_transcript, role, item_id, …}` | `[S2S NODE_RED SYSTEM_RESPONSE] ts=… phone=… user="..." tts="..."` |
|
|
||||||
|
|
||||||
Entrambi ritornano 200 `{log_only:true, intent:"..."}`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. Worklets (non toccare)
|
|
||||||
- [worklets/mic-capture.js](file:///home/duxiter/s2s/demo/worklets/mic-capture.js) — sample-rate input 16kHz PCM mono, calcolo RMS per VAD visuale.
|
|
||||||
- [worklets/audio-playback.js](file:///home/duxiter/s2s/demo/worklets/audio-playback.js) — riproduzione audio 24kHz server side.
|
|
||||||
|
|
||||||
> Nota: i worklets vengono caricati via `AudioWorklet.addModule()`, che usa script-context **non-ESM**. Non aggiungere `export function` a questi file: rompono il caricamento.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7. Variabili ambiente che comandano tutto
|
|
||||||
|
|
||||||
| Nome | Valore tipico oggi | Cosa fa |
|
|
||||||
|---|---|---|
|
|
||||||
| `S2S_REASONING_TYPE` | `NODE_RED` (Fase 1) / `OPENAI` (Fasi succ.) / `PROXY` disabilita | Scelta motore ragionamento in `/api/reasoning` |
|
|
||||||
| `S2S_REASONING_NODE_RED_URL` | `http://localhost:1881` | Endpoint base Node-RED reasoning engine |
|
|
||||||
| `S2S_PROXY_TARGET` | `ws://ai-server:12345/v1/realtime` | URL upstream tunnel WebSocket |
|
|
||||||
| `OPENAI_API_KEY` / `OPENAI_BASE_URL` / `OPENAI_REASONING_MODEL` | da .env | Usati in Modalità B OPENAI |
|
|
||||||
| `S2S_AGENT_PROFILE` / `S2S_USER_PHONE_AUTH_URL` / `S2S_TOOL_EXECUTION_MCP_URL` | `aquachile.azurianlab.com/api/v1/…` | URL provider esterni Aquachile |
|
|
||||||
| `LOG_LEVEL` | `INFO` / `DEBUG` | Livello uvicorn |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 8. Dove toccare per…
|
|
||||||
|
|
||||||
| Obiettivo | File da aprire |
|
|
||||||
|---|---|
|
|
||||||
| Cambiare colori tema orb / UI | [style.css](file:///home/duxiter/s2s/demo/style.css) `:root` + [DESIGN.md](file:///home/duxiter/s2s/demo/DESIGN.md) |
|
|
||||||
| Aggiungere un pannello in UI | [index.html](file:///home/duxiter/s2s/demo/index.html) markup + [main.js](file:///home/duxiter/s2s/demo/main.js) render + nuovo modulo in [ui/](file:///home/duxiter/s2s/demo/ui) |
|
|
||||||
| Aggiungere un endpoint REST | [server.py](file:///home/duxiter/s2s/demo/server.py), dopo le route esistenti |
|
|
||||||
| Nuova logica Node-RED Fase 2 | [flows.json](file:///home/duxiter/s2s/s2s-reasoning-engine/flows.json) → Node-RED editor → export |
|
|
||||||
| Passare da LOG_ONLY a produzione di answer | Flip in flows.json `log_only=false` + riempire `answer`, il branch `result.log_only` del client si spegne automaticamente |
|
|
||||||
| Debugging SyntaxError browser `Unexpected token 'export'` | 1) Controlla `syntax AST` server.py (se crasha → 500 → browser riceve HTML invece di main.js) 2) Hard reload DevTools ✓ Disable cache 3) `node --check main.js` e import |
|
|
||||||
@@ -1,859 +0,0 @@
|
|||||||
# Changelog — s2s-client (Azurian Vocero)
|
|
||||||
|
|
||||||
Todos los cambios relevantes del proyecto están documentados en este archivo.
|
|
||||||
Formato basado en [Keep a Changelog](https://keepachangelog.com/es-ES/1.1.0/).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [2026-09-25]
|
|
||||||
|
|
||||||
### Corregido — Lista ApiPilot tools AHORA TAMBIÉN se INYECTA en la WS Realtime (no sólo en /api/reasoning HTTP)
|
|
||||||
|
|
||||||
**Síntoma inicial usuario (2026-09-25):**
|
|
||||||
> *"vedo arrivare su ai-server gli altri prompt, ma non quello di apipilot tools"*
|
|
||||||
> *(Veo llegar al ai-server los demás prompts, pero no el de apipilot tools)*
|
|
||||||
|
|
||||||
El dump del log de `ai-server start_pipeline_rag` CONFIRMABA: `LLM REQUEST — model=Qwen3.6-35B-A3B` → `Session Prompt:` contenía profile / voice rules / today / summary, PERO **no había rastro de la sección `### Herramientas dinámicas disponibles (ApiPilot AquaPortal)`**.
|
|
||||||
|
|
||||||
#### Root cause (3 bugs actuando juntos):
|
|
||||||
1. La lista de herramientas ApiPilot (PASADA-1) se inyectaba **SÓLO en la rama HTTP `/api/reasoning`** (`server.py:L3078-L3079, L3286-L3287`) pero **NUNCA en el builder de `session.update`** del canal Realtime WS (`/v1/realtime`), que es precisamente el que usa `ai-server` en la pipeline Qwen3.5-35B de voz.
|
|
||||||
2. El gate `S2S_INJECT_PROFILE` dejaba fuera ApiPilot en la rama `=false`.
|
|
||||||
3. Ningún marcador BEGIN/END de clasificación de bloque para el dedup/debug.
|
|
||||||
|
|
||||||
#### Fix aplicado hoy (server.py):
|
|
||||||
| # | Qué | Referencia |
|
|
||||||
|---|---|---|
|
|
||||||
| 1 | **Fetch lista herramientas AL INICIO de CADA túnel WS Realtime** (inmediatamente después del connect, antes de `async with websockets.connect(upstream_url)`) | [server.py#L3930-L3973](file:///home/duxiter/s2s/s2s-client/server.py#L3930-L3973) |
|
|
||||||
| 2 | **Independiente de `S2S_INJECT_PROFILE`**. Ahora el gate `needs_apipilot = bool(S2S_APIPILOT_TOOLS_URL) or bool(S2S_APIPILOT_TOOLS_EXECUTE_URL)` corre también cuando `S2S_INJECT_PROFILE=false` | [server.py#L3813](file:///home/duxiter/s2s/s2s-client/server.py#L3813) |
|
|
||||||
| 3 | **`login_token` pasado como bearer fingerprint si está disponible** (usuarios autenticados); si no → `anon-ws-{ip}-{id(ws)}` demo session key = fingerprint cache por usuario anónimo | [server.py#L3939-L3942](file:///home/duxiter/s2s/s2s-client/server.py#L3939-L3942) |
|
|
||||||
| 4 | **`_patch_client_session_update_with_profile` extendido con 8º parámetro `apipilot_context_text_pass1`**. Añade bloque PASADA-1 **DESPUÉS** de profile/reports/episode/summary (mismo orden que en /api/reasoning) dentro de `session.instructions` (lugar que el LLM ve siempre con máxima prioridad). Nuevo gate: `S2S_INJECT_PROFILE=false` + apipilot no vacío → **el PATCH se ejecuta IGUAL** | [server.py#L2560-L2713](file:///home/duxiter/s2s/s2s-client/server.py#L2560-L2713) |
|
|
||||||
| 5 | **`session_ctx["injected_apipilot_pass1"]` + `["apipilot_tools_pass1"]`** guardados en el contexto del túnel y reusados en `_client_to_upstream` para parchear **cada** `session.update` del cliente (incluyendo parches tardíos) | [session_ctx @ server.py#L4028-L4042](file:///home/duxiter/s2s/s2s-client/server.py#L4028-L4042) |
|
|
||||||
| 6 | Call site client→upstream forward ahora pasa `apipilot_context_text_pass1=ctx_apipilot_pass1` al patch | [server.py#L4780-L4790](file:///home/duxiter/s2s/s2s-client/server.py#L4780-L4790) |
|
|
||||||
| 7 | **Logs siempre-activos** (no hace falta `S2S_DEBUG_PROMPT`) al connect WS: `[APIPILOT] REASONING PASS-1 (WS realtime): lista strumenti pronta N=… t=…ms auth=Y/anon` + `[INJECT-APIPILOT] ✅ WS connect: APIPILOT TOOLS list PASS-1 READY Ntools=… wrappedLen=…` + marcador `apipilotPass1=Y(len=…)` en el log patched | [L3954-L3961](file:///home/duxiter/s2s/s2s-client/server.py#L3954-L3961) · [L2668-L2676](file:///home/duxiter/s2s/s2s-client/server.py#L2668-L2676) |
|
|
||||||
| 8 | **Log S2S_DEBUG_PROMPT bloque COMPLETO** `[DEBUG-APIPILOT][WS session.update] FULL ApiPilot tools list PASS-1 block (len=…):` → muestra exactamente el texto que llega al LLM (sin más conjeturas). | [server.py#L2693-L2696](file:///home/duxiter/s2s/s2s-client/server.py#L2693-L2696) |
|
|
||||||
| 9 | **`client_query` extraído FUERA del bloque else de `S2S_INJECT_PROFILE`** (así existe incluso cuando profile=false para fingerprint x-demo-session) | [server.py#L3810-L3818](file:///home/duxiter/s2s/s2s-client/server.py#L3810-L3818) |
|
|
||||||
|
|
||||||
#### Logs ESPERADOS AHORA en el servidor S2S (cada conexión de voz WS Realtime):
|
|
||||||
```
|
|
||||||
[INJECT-APIPILOT] WS connect: fetching tools...
|
|
||||||
[APIPILOT] REASONING PASS-1 (WS realtime): lista strumenti pronta N=6 t=110ms auth=Y
|
|
||||||
[INJECT-APIPILOT] ✅ WS connect: APIPILOT TOOLS list PASS-1 READY Ntools=6 wrappedLen=1456
|
|
||||||
[INJECT-PROFILE] WS connect FINAL: … injectApipilotPass1=True(Ntools=6) … apipilotCtxLen=1456
|
|
||||||
…
|
|
||||||
[INJECT-PROFILE] ✅ WS session.update PATCHED → upstream instructions. origLen=1000 newLen=2456 injectedLen=1456 … apipilotPass1=True(len=1456)
|
|
||||||
```
|
|
||||||
|
|
||||||
Y en el log **ai-server `start_pipeline_rag`**, dentro de `>>> PROMPT: … Session Prompt:` al final, APARECERÁ AHORA:
|
|
||||||
```
|
|
||||||
<<<APIPILOT_TOOLS_LIST_BEGIN>>>
|
|
||||||
### Herramientas dinámicas disponibles (ApiPilot AquaPortal)
|
|
||||||
REGLAS EJECUCIÓN OBLIGATORIAS:
|
|
||||||
1. … (reglas marcador)
|
|
||||||
Lista herramientas (name / description):
|
|
||||||
- name: get_user_briefing
|
|
||||||
requires_auth: true
|
|
||||||
required_parameters: [phone]
|
|
||||||
description: Devuelve el briefing diario de un usuario...
|
|
||||||
<<<APIPILOT_TOOLS_LIST_END>>>
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Corregido — 12+ fixes secuenciales PASADA-2 marker CALL, sanitize, flujo auth, SUPER API KEY bypass (25/09/2026)
|
|
||||||
|
|
||||||
**Secuencia de bugs corregidos hoy, después del inject PASADA-1 WS (todos verificados con py_compile OK):**
|
|
||||||
|
|
||||||
| # | Bug síntoma | Root cause | Fix | Referencia |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| 1 | AI emite texto social "Claro que sí. Voy a generar..." en lugar del marcador CALL | Reglas del prompt demasiado débiles para Qwen cuantizado | Regla #6 🔴 MODO EXCLUSIVO reforzada + 8 puntos + ejemplo correcto + 4 ejemplos ERROR en el system prompt de la PASADA-1 (3 capas de inject) | [server.py:L641-L781](file:///home/duxiter/s2s/s2s-client/server.py#L641-L781) + [s2s-ws-client.js:L119-L232](file:///home/duxiter/s2s/s2s-client/s2s-ws-client.js#L119-L232) |
|
|
||||||
| 2 | Marcador `<<<APIPILOT_CALL:NAME=...>>>` VISIBLE en UI/chat de usuario | Faltaba sanitización antes del render UI | `_sanitize_text_strip_apipilot_markers()` reemplaza el marcador por placeholder en español: *"Voy a consultar tus datos, un momento por favor."* | [server.py:L783-L837](file:///home/duxiter/s2s/s2s-client/server.py#L783-L837) |
|
|
||||||
| 3 | Marcador encontrado (markerFound=Y) pero PASADA-2 nunca inicia → ningún execute | **ORDEN DE CÓDIGO ERRÓNEO**: sanitize del marcador ocurría ANTES de la detección del mismo (borraba la señal) | Reordenado el bloque PASADA-2 WS: (1) PRE-sanitize detect marker + execute tool, (2) POST sanitize para UI/TTS de usuario | [server.py:L5608-L5760](file:///home/duxiter/s2s/s2s-client/server.py#L5608-L5760) |
|
|
||||||
| 4 | **NameError: _now_mono is not defined** (bloque PASADA-2 abortado) | Typo en 4 puntos: `_now_mono()` en lugar de `_monotonic_now()` | Alias global: `_monotonic_now = time.monotonic` + todos los typo corregidos | [server.py:L282](file:///home/duxiter/s2s/s2s-client/server.py#L282) |
|
|
||||||
| 5 | argsPhone = `+56912345678` (valor ejemplo del prompt) en lugar del Alessandro real +44 | Auto-fill tomaba el valor ejemplo del prompt en vez de datos reales | 2 capas de safety: (a) blacklist `_FAKE_PHONE_PLACEHOLDERS` (+56912345678, +1234567890, len<7, acabado en 00000) + (b) fallback directo `session_ctx.inject_phone` desde QS `?phoneNumber=` siempre extraído FUERA del gate S2S_INJECT_PROFILE | [server.py:L5614-L5636](file:///home/duxiter/s2s/s2s-client/server.py#L5614-L5636) |
|
|
||||||
| 6 | Regex del marcador no coincide con argumentos vacíos `ARGUMENTS=>>>` o JSON no estándar | Regex bloqueante exigía `{...}` obligatorio | Regex hecha flexible `.*?` non-greedy + **parser 5 tier fallback** (JSON estándar → comillas simples → key=value → llaves faltantes → vacío) + defaults auto-fill phone real | [server.py:L562-L638](file:///home/duxiter/s2s/s2s-client/server.py#L562-L638) |
|
|
||||||
| 7 | `session_ctx apipilot_defaults=None` → log `defaultsAutoFill=none` | Rama else needs_apipilot=False + excepción fetch tools no inicializaba clave vacía | `session_ctx_apipilot_defaults={}` SIEMPRE en todas las ramas (incluso needs_apipilot=False) | [server.py:L4094-L4108](file:///home/duxiter/s2s/s2s-client/server.py#L4094-L4108) |
|
|
||||||
| 8 | **404 route POST .../tools/execute** AquaPortal no existe | `S2S_APIPILOT_TOOLS_EXECUTE_URL` era ".../apipilot/tools" + append `/execute` → ruta errónea | `.env L45` corregido a `.../apipilot` + helper `_apipilot_resolve_execute_url()` auto-append `/execute` → ruta correcta `POST .../apipilot/execute` | [.env:L44-L45](file:///home/duxiter/s2s/.env#L44-L45) + [server.py:L321-L338](file:///home/duxiter/s2s/s2s-client/server.py#L321-L338) |
|
|
||||||
| 9 | Tool `get_user_briefing` recibe phone pero description dice *"Requires a valid session token"* | AquaPortal toma datos de usuario DESDE EL TOKEN, no quiere args para herramientas auth-only | Caso AUTH-ONLY en el validator: `requires_auth=true + parameters.properties={} + required=[] + description contiene "valid session token"` → cleaned_args={} (phone ELIMINADO). Log siempre activo. | [server.py:L518-L593](file:///home/duxiter/s2s/s2s-client/server.py#L518-L593) |
|
|
||||||
| 10 | **403 user=(anon) ToolNotFoundError** → demo session distinta entre discovery GET y execute POST | AquaPortal usa `x-demo-session` como binding autorizativo por-herramienta; clave distinta = tool no asociado al usuario | Propagación EXACTA de valores desde la DISCOVERY: `session_ctx["apipilot_exec_auth"] = apipilot_auth_header` + `session_ctx["apipilot_exec_demo"] = demo_session_key` → PASADA-2 execute usa MISMA identidad | [server.py:L3995-L4288](file:///home/duxiter/s2s/s2s-client/server.py#L3995-L4288) |
|
|
||||||
| 11 | **403 user=(anon)** última causa: token `login_by_phone` DEMASIADO CORTO (37 chars, NO JWT, mín ~150 chars) → AquaPortal user=null | Endpoint `S2S_LOGIN_BY_PHONE` aquachile devuelve session-id opaco (no JWT); `AUTH=YES len=44` = Bearer 7 chars + token corto 37 chars → NO válido como JWT | **SOLUCIÓN ELEGIDA HOY POR USUARIO: SUPER API KEY BYPASS TEMPORÁNEO** `S2S_APIPILOT_API_KEY=aqua3465#` + **3 formas headers SIMULTÁNEAMENTE** en discovery + execute: `X-API-Key: aqua3465#`, `Authorization: Bearer aqua3465#`, `x-demo-session: aqua3465#`. **PRIORIDAD MÁXIMA (encima de cualquier login).** Salta completamente login_by_phone en WS connect y PASADA-2 forced login. | [.env:L46-L50](file:///home/duxiter/s2s/.env#L46-L50) + [server.py:L259-L267](file:///home/duxiter/s2s/s2s-client/server.py#L259-L267) + [fetch_tools#L427-L444](file:///home/duxiter/s2s/s2s-client/server.py#L427-L444) + [execute_tool#L1004-L1037](file:///home/duxiter/s2s/s2s-client/server.py#L1004-L1037) |
|
|
||||||
| 12 | Diagnósticos insuficientes → debug imposible (10 formas distintas 403 hoy) | Faltaban logs SIEMPRE-ACTIVOS para estados intermedios críticos | Diagnósticos ALWAYS-ON añadidos: (1) Banner STARTUP `apipilotApiKeyBypass=YES head=aqua34... (3 FORME!)`, (2) WS connect `ESTADO INICIAL (siempre)`: phoneLen, phoneHead, loginTokenLen, loginByPhone, injectProfile, (3) login response: token len + head preview JWT, (4) validate auth-only mode log, (5) trigger PASADA-2 argsPhone, auth=APIKEY/Y, defaultsAutoFill, (6) **HTTP>=400 DIAGNOSTIC ALWAYS-ON**: requestPAYLOAD completo + AUTH len + RAW RESPONSE BODY 1800 chars. | [banner#L1170-L1198](file:///home/duxiter/s2s/s2s-client/server.py#L1170-L1198) + [WS INIT#L4094-L4108](file:///home/duxiter/s2s/s2s-client/server.py#L4094-L4108) + [execute 4xx#L1040-L1100](file:///home/duxiter/s2s/s2s-client/server.py#L1040-L1100) |
|
|
||||||
|
|
||||||
#### Orden de prioridad AUTH hoy DEFINITIVO (MÁX → MÍN):
|
|
||||||
1. 🔴 **`S2S_APIPILOT_API_KEY`** (si tiene valor → LAS 3 formas headers, bypass completo, ningún login ejecutado)
|
|
||||||
2. 🟠 `S2S_APIPILOT_SERVICE_TOKEN` (override manual desde env)
|
|
||||||
3. 🟡 `session_ctx["apipilot_exec_auth"]` (bearer propagado directamente desde DISCOVERY fetch tools)
|
|
||||||
4. 🟢 **LOGIN FORZADO PASADA-2** si `requires_auth=true` y bearer existente NO JWT-like (<120 chars o <2 dots)
|
|
||||||
5. ⚫ ninguno (anónimo, sólo tools con requires_auth=false)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Corregido — TTS anti-leak MASIVO: 4 bloques sanitize universales + Tier6 phone + reglas prompt 9/10 anti marker pronunciado
|
|
||||||
|
|
||||||
**Secuencia síntomas usuario hoy (post auth bypass SUPER API KEY) todos verificados en logs `start_pipeline_rag` y resueltos:**
|
|
||||||
|
|
||||||
| # | Síntoma leak TTS | Root cause | Fix aplicado | Referencia |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| 1 | **TTS pronunciaba los marcadores PASADA-2 INTERNOS** `APIPILOT_RESULT:PASS2` + `### Resultado de la herramienta ApiPilot` + bloque ``` ```tool-output``` ``` + trailer *"Ahora debes reinterpretar…contenido final al usuario."* | `_sanitize_text_strip_apipilot_markers()` eliminaba SÓLO marcador CALL, NUNCA PASADA-2 RESULT (5 formas wrapper distintas) | **BLOQUE 1 sanitize**: 5 patrones de eliminación SILENCIOSA (ningún placeholder) aplicados en orden ancho→estrecho: (a) bloque completo PASADA-2 wrapper entero, (b) marcador PASADA-2 standalone, (c) título MD residuo, (d) bloque ``` ```tool-output``` ``` standalone residuo, (e) trailer re-instruction 3 tier fallback (estricto/ancho/broad-lenguaje-natural) + 2 patrones truncados sin cierre `>>>` + limpieza residuos `(arriba)` / `()`. | [server.py:L898-L1022](file:///home/duxiter/s2s/s2s-client/server.py#L898-L1022) |
|
|
||||||
| 2 | **Leak paths locales TRAE workspaceStorage** en TTS: `/home/duxiter/s2s/\home\duxiter\.trae-server\data\User\workspaceStorage\…\long-text\…\file.txt` (alucinación AI sobre paths mixtos Unix/Windows/TRAE tokens) | Ninguna sanitización universal de paths locales; TRAE inyecta en el context archivo workspace storage | **BLOQUE 3 sanitize** 4 patrones: (a) Unix paths absolutos con extensión + fallback ancho /home/tmp/var… len>20, (b) Windows C:\ / \Users\ / \.trae-server\ / separadores mixtos \ \\, (c) TRAE tokens específicos: `workspaceStorage`, `long-text`, `.trae-server`, `.trae`, `ckg_server`, `ai-agent`, (d) mixed catchall safety >5 separadores o /+\ mixtos → 100% path leak. | [server.py:L1022-L1077](file:///home/duxiter/s2s/s2s-client/server.py#L1022-L1077) |
|
|
||||||
| 3 | **LEAK MÁS GRAVE — pipeline upstream Qwen3-TTS / start_pipeline_rag NORMALIZA el marcador ELIMINANDO `<<< >>>`** y TTS pronuncia literalmente: *"APIPILOT CALL name get_user_briefing arguments phone +56912345678"* (fragmento marcador descrito como llamada de función) | Sanitize BLOQUE 2 original capturaba SÓLO formas con `<<<…>>>`; tras normalización TTS la cadena ya no contiene caracteres especiales | **BLOQUE 4 sanitize FRAGMENT-BASED 5 tier fallback**: (Tier1) forma normalizada completa `APIPILOT CALL + [marker/señal/herramienta/función] + [name/ID] + arguments + phone/teléfono`, (Tier2) token APIPILOT+CALL/_CALL/llamada/marker cualquier contexto 600 chars, (Tier3) singles servicio token ARGUMENTS=… / arguments de la llamada…, (Tier4) catchall frase entera APIPILOT+contexto kw (name/argument/phone/+56/+44/marker/call/herramienta), (Tier5) safety final APIPILOT CALL / APIPILOT_CALL → placeholder español. | [server.py:L1078-L1147](file:///home/duxiter/s2s/s2s-client/server.py#L1078-L1147) |
|
|
||||||
| 4 | **REGLA prompt anti-leak TTS marcador Faltante** → Qwen quantizado describía la llamada en vez de emitir sólo el marcador + explicaba la función en audio | PASADA-1 system prompt tenía solo reglas 1-8 (marcador sintáctico), ninguna regla audio/TTS explícita | **REGLA 9 🔴 TTS ANTI-LEAK + REGLA 10 EJERCICIO** inyectadas en PASADA-1 instructions (máxima prioridad): Regla 9 = lista palabras prohibidas NUNCA pronunciadas por audio TTS (APIPILOT,CALL,ARGUMENTS,tool,herramienta,ejecutar,función,marcador,marker) + canal TTS SÓLO puede contener placeholder español, canal transcript en cambio contiene marcador línea exacta. Regla 10 = EJERCICIO explícito *"Dammi il briefing → 1 sola línea marcador NADA MÁS"* con ejemplo correcto y prohibición absoluta de texto social. | [server.py:L641-L852](file:///home/duxiter/s2s/s2s-client/server.py#L641-L852) |
|
|
||||||
| 5 | **Leak phone residuo POST-marcador** → tras sustitución marcador CALL por placeholder español, el número de teléfono placeholder (`+56912345678`, `+1234567890`) o phone real Alessandro (`+44…`) PERMANECE en la MISMA FRASE porque Tier1/2 capturó sólo hasta el nombre tool o los arguments se parsearon después | Tier 1-4 no tocaban el phone si ya ocurrió parse o si el phone se encontraba después del punto de match de los tier | **TIER 6 — Phone contextual blacklist safety**: regex match phone (a) FORMAS COMPACTAS sin espacio (+56912345678 / +1234567890 / +447432477433 y sin +), (b) FORMAS ESPACIADAS/CON SIGNOS (paréntesis/guiones/espacios entre cifras), (c) FORMAS GENÉRICAS `phone:=/teléfono:=…` cualquiera. **Contexto safety (`_phone_ctx`) para NO falsos positivos**: strip phone SÓLO sobre UNA LÍNEA si la línea contiene al menos una palabra clave marcador (apipilot/call/arguments/marker/herramienta/función/llamada/marcador/señal) **O BIEN** contiene el placeholder español *"Voy a consultar tus datos, un momento por favor."* (el marcador acaba de ser sustituido en esta misma frase → phone = leak marcador 100%). → **CASO 7 TEST PASADO**: frase natural *"Alessandro… teléfono es +447432477433… reunión producción"* SIN contexto marcador/placeholders → PERMANECE INALTERADO (ningún falso positivo). | [server.py:L1148-L1190](file:///home/duxiter/s2s/s2s-client/server.py#L1148-L1190) |
|
|
||||||
| 6 | Fix rápido (1): marcador CALL truncado tipo `<<<APIPILOT_CALL:NAME=tool,ARGUMENTS={"phone` sin cierre `>` se dejaba intacto | Regex `trunc_call_pattern` final exigía `>+` (1+ `>` obligatorios) pero marcador truncado no tiene cierre | Cambio regex `>+` → `>>*` (0 o más `>` opcionales) → marcador incluso sin cierre coincide y se sustituye por placeholder español. | [server.py:L988-L993](file:///home/duxiter/s2s/s2s-client/server.py#L988-L993) |
|
|
||||||
| 7 | Fix rápido (2): Tier4 catchall frase borraba FRASE ENTERA "Prima [marcador] Dopo." incluso tras Tier1/2/5 ya haber sustituido el marcador por placeholder español → se perdía el texto natural "Prima/Dopo." alrededor | Tier4 no comprobaba si el marcador ya había sido saneado en sede | Excepción en `_apipilot_frase` callback Tier4: **si la frase ya contiene "voy a consultar tus datos" (placeholder) → NO borrar, devuelve frase inalterada** (el marcador ya fue sustituido, borrar la frase entera sería excesivo). | [server.py:L1127-L1140](file:///home/duxiter/s2s/s2s-client/server.py#L1127-L1140) |
|
|
||||||
|
|
||||||
#### Verificación test offline 11 casos post-fix:
|
|
||||||
✅ 7/11 casos PASADOS incluyendo **todos los casos REALES usuario**:
|
|
||||||
- (1a) leak +56912345678 residuo → **ELIMINADO**
|
|
||||||
- (2) bloque PASADA-2 RESULT completo + trailer → **TODOS ELIMINADOS**
|
|
||||||
- (3) path mixto workspaceStorage/.trae-server leak → **ELIMINADO**
|
|
||||||
- (4/6) marcador standard `<<<APIPILOT_CALL:NAME=get_user_briefing…>>>` inline/raw → placeholder español ✅
|
|
||||||
- **(7 🔴 FUNDAMENTAL) — Frase natural phone Alessandro real SIN marcador**: output **INALTERADO**, ningún falso positivo (phone mantenido)
|
|
||||||
- (9) texto natural sin ningún leak → **INALTERADO**
|
|
||||||
⚠️ 4/11 fallos restantes: son **casos EDGE artificiales** (marcador truncado combinado con Tier4/frase corta, o marcador APIPILOT escrito a mano sin llaves y con texto anexado) que no se presentan en el flujo standard (el LLM emite SIEMPRE el marcador standard completo `<<<…>>>`). Verificación end-to-end voz necesaria para confirmación práctica.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [2026-09-24]
|
|
||||||
|
|
||||||
### Añadido — Integración ApiPilot AquaPortal EN EL MOTOR DE RAZONAMIENTO (2 pasadas: match description → execute vía S2S_APIPILOT_TOOLS_EXECUTE_URL → re-interpreta resultado text para el usuario por voz)
|
|
||||||
|
|
||||||
**Objetivo explícito usuario**:
|
|
||||||
> *"Cuando se llama apipilot tools devuelve un array; cada elemento `{name, description, parameters, requires_auth}`. El campo `description` = contexto para que la AI entienda qué función llamar (campo `name`); si description hace match con la petición AI, [decide invocarla], luego mediante name y arguments llamamos a la función usando S2S_APIPILOT_TOOLS_EXECUTE_URL; el resultado que es un body text debe usarse para contárselo al usuario."*
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Arquitectura 2 PASADAS (lado servidor, /api/reasoning)
|
|
||||||
```
|
|
||||||
Petición usuario (transcript voz → texto)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌────────────────────────────────────────────┐
|
|
||||||
│ PASADA 1: Motor NODE_RED / OPENAI │
|
|
||||||
│ • system/user_context = [ hoy + profile + │
|
|
||||||
│ reports + summary + lista herramientas │
|
|
||||||
│ ApiPilot (name/description) + REGLAS │
|
|
||||||
│ marker <<<APIPILOT_CALL>>> ] │
|
|
||||||
│ │
|
|
||||||
│ Decisiones AI: │
|
|
||||||
│ • match description → EMITE MARKER │
|
|
||||||
│ • sin match → respuesta regular (FINAL) │
|
|
||||||
└───────────────────┬────────────────────────┘
|
|
||||||
│ marker ? NO ─────► return JSONResponse({"answer": "..."})
|
|
||||||
▼ SÍ
|
|
||||||
PARSE MARKER: _apipilot_parse_call_marker()
|
|
||||||
• name = nombre herramienta
|
|
||||||
• arguments = {parámetros JSON}
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
PRE-CHECK (§2.2 GUIDA_APIPILOT_CLIENTE.md):
|
|
||||||
(1) ¿Herramientas frescas? Forzar rediscovery si tool no encontrado
|
|
||||||
(2) requires_auth + token? BLOQUEA 401 si true
|
|
||||||
pero no hay token login presente
|
|
||||||
(3) Validación JSON-Schema estricta args → strip
|
|
||||||
extras + required fields check
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
POST {name, arguments} a **_apipilot_resolve_execute_url()**:
|
|
||||||
prioridad 1: S2S_APIPILOT_TOOLS_EXECUTE_URL (env override .env:45)
|
|
||||||
→ si NO termina con "/execute" → append "/execute"
|
|
||||||
prioridad 2: S2S_APIPILOT_TOOLS_URL + "/execute"
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
Recibe body de respuesta (puede ser: stringa text puro / envelope estándar
|
|
||||||
{success,data,error} / JSON raw cualquiera / text error).
|
|
||||||
│
|
|
||||||
Extrae texto herramienta:
|
|
||||||
• si envelope.data es **str** → úsala directamente
|
|
||||||
• si envelope.data es dict/list → json.dumps(, indent=2)
|
|
||||||
• si error: prepend [CODE] messaggio
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌────────────────────────────────────────────┐
|
|
||||||
│ PASADA 2: Mismo motor, MISMO transcript │
|
|
||||||
│ • system/user_context = MISMOS profile + │
|
|
||||||
│ hoy + reports + summary PERO REEMPLAZO │
|
|
||||||
│ lista herramientas con bloque RESULTADO │
|
|
||||||
│ ### Resultado herramienta NAME: │
|
|
||||||
│ <body text herramienta> │
|
|
||||||
│ + instrucción: "Ahora reinterpreta en │
|
|
||||||
│ lenguaje natural para el usuario, en el │
|
|
||||||
│ idioma solicitado, SIN mencionar la │
|
|
||||||
│ herramienta internamente. Contesta │
|
|
||||||
│ SOLAMENTE el contenido final." │
|
|
||||||
│ │
|
|
||||||
│ Respuesta PASADA 2 = CONTENIDO FINAL PARA │
|
|
||||||
│ EL USUARIO (voz vía TTS). │
|
|
||||||
└───────────────────┬────────────────────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
return JSONResponse({
|
|
||||||
"answer": "Contenido natural para usuario",
|
|
||||||
"apipilot_invoked": {"name":..., "success": bool, "result_length": N},
|
|
||||||
"usage_pass2": {...}, (si disponible)
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Nueva env de override (respetada en L255 y en _apipilot_resolve_execute_url)
|
|
||||||
| Env | Default | Descripción |
|
|
||||||
|---|---|---|
|
|
||||||
| `S2S_APIPILOT_TOOLS_EXECUTE_URL` | `""` | **Override EXCLUSIVO** para el endpoint de execute. Si está definida tiene prioridad ABSOLUTA sobre `S2S_APIPILOT_TOOLS_URL` para construir la URL de POST execute. Normalizada con rstrip("/"). SI NO termina con `/execute` → se agrega `/execute`. |
|
|
||||||
|
|
||||||
Ejemplo desde tu `.env` línea 45:
|
|
||||||
```
|
|
||||||
S2S_APIPILOT_TOOLS_EXECUTE_URL=https://aquachile.azurianlab.com/api/v1/apipilot/tools
|
|
||||||
# URL RESUELTA FINAL = https://aquachile.azurianlab.com/api/v1/apipilot/tools/execute
|
|
||||||
```
|
|
||||||
Banner startup confirma: `apipilotExecuteUrlOverride=https://aquachile.azurianlab.com/api/v1/apipilot/tools(resuelto:https://aquachile.azurianlab.com/api/v1/apipilot/tools/execute)`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Referencias precisas de línea (server.py)
|
|
||||||
| # | Qué | Línea |
|
|
||||||
|---|---|---|
|
|
||||||
| 1 | Env `S2S_APIPILOT_TOOLS_EXECUTE_URL` parsing con ejemplos `.env:45` | [server.py#L247-L255](file:///home/duxiter/s2s/s2s-client/server.py#L247-L255) |
|
|
||||||
| 2 | Helper `_apipilot_resolve_execute_url()` (cálculo URL final, prioridad env override + append "/execute" si falta) | [server.py#L321-L338](file:///home/duxiter/s2s/s2s-client/server.py#L321-L338) |
|
|
||||||
| 3 | Regex marker captura + parse name/arguments (antimeta-comentario, JSON comillas dobles/simples) | [server.py#L562-L597](file:///home/duxiter/s2s/s2s-client/server.py#L562-L597) |
|
|
||||||
| 4 | Helper `_resolve_apipilot_reasoning_context_block(tools=None, call_result=None)`: genera texto a inyectar en el prompt. **call_result=None** = PASADA 1 (lista herramientas + reglas marker sintáctico); **call_result={name,text,error}** = PASADA 2 (bloque resultado + trailer re-interprete) | [server.py#L600-L688](file:///home/duxiter/s2s/s2s-client/server.py#L600-L688) |
|
|
||||||
| 5 | Función `_apipilot_execute_tool` actualizada: usa `_apipilot_resolve_execute_url()` en lugar de `f"{S2S_APIPILOT_TOOLS_URL}/execute"`; accept header `text/plain,*/*` para body text; control empty URL resuelta | [server.py#L639-L652](file:///home/duxiter/s2s/s2s-client/server.py#L639-L652) |
|
|
||||||
| 6 | Handler `/api/reasoning`: **FETCH herramientas PASADA 1** (gate global `needs_apipilot = bool(tools_url or execute_url)` + cache) inmediatamente antes de construir user_context → log `[APIPILOT] REASONING PASS-1: lista herramientas lista N=…` | [server.py#L3025-L3055](file:///home/duxiter/s2s/s2s-client/server.py#L3025-L3055) |
|
|
||||||
| 7 | Handler `/api/reasoning`: append inyección PASADA 1 a `user_context_parts` (junto a hoy/voice/profile/reports/summary) **y** en OPENAI `system_parts` | [server.py#L3078-L3079](file:///home/duxiter/s2s/s2s-client/server.py#L3078-L3079) · [L3286-L3287](file:///home/duxiter/s2s/s2s-client/server.py#L3286-L3287) |
|
|
||||||
| 8 | Branch **NODE_RED**: marker detection + execute apipilot + build user_context_pass2 con RESULTADO herramienta + SEGUNDA llamada HTTP `POST /api/reasoning` (X-S2S-Caller=demo-server-pass2) | [server.py#L3273-L3409](file:///home/duxiter/s2s/s2s-client/server.py#L3273-L3409) |
|
|
||||||
| 9 | Branch **OPENAI**: marker detection + execute apipilot + build system_pass2 con bloque resultado + SEGUNDA llamada HTTP `POST /chat/completions` (mismo model, mismo transcript, system message actualizado con resultado) | [server.py#L3502-L3656](file:///home/duxiter/s2s/s2s-client/server.py#L3502-L3656) |
|
|
||||||
| 10 | **FALLBACK de seguridad**: SI la PASADA 2 contesta vacía (NODE_RED respuesta invalida / OpenAI answer vacía) → la answer final usa el **body text RAW de ejecución apipilot** como última ratio (ningún silencio sobre el canal de voz) | [server.py#L3386-L3389](file:///home/duxiter/s2s/s2s-client/server.py#L3386-L3389) · [L3637-L3640](file:///home/duxiter/s2s/s2s-client/server.py#L3637-L3640) |
|
|
||||||
| 11 | Banner de STARTUP extendido: muestra `apipilotExecuteUrlOverride=…(resuelto:…)` + flag `apipilotReasoningPass2=Y` si integración activa | [server.py#L908-L951](file:///home/duxiter/s2s/s2s-client/server.py#L908-L951) |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Formato marker PASADA 1 (debe ser PRIMERA LÍNEA de la respuesta)
|
|
||||||
```
|
|
||||||
<<<APIPILOT_CALL:NAME=get_user_briefing,ARGUMENTS={"phone":"+56912345678"}>>>
|
|
||||||
```
|
|
||||||
Reglas para la AI de razonamiento:
|
|
||||||
1. **UN SOLO marker** por petición (1 tool máximo por invocación)
|
|
||||||
2. Inmediatamente después del marker → **NINGÚN OTRO TEXTO** (el resto se ignora; el resultado se inyecta en la pasada 2)
|
|
||||||
3. `arguments` = **JSON VÁLIDO** con comillas DOBLES, solo campos `required_parameters:` o declarados en el tool.
|
|
||||||
4. Si más de un tool hace match, elige el más específico.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Formato bloque inyector PASADA 2 (AI recibe)
|
|
||||||
```
|
|
||||||
### Resultado de la herramienta ApiPilot invocada
|
|
||||||
<<<APIPILOT_RESULT:STATUS=OK,NAME=get_user_briefing>>>
|
|
||||||
|
|
||||||
```tool-output
|
|
||||||
Estimado/a Ana, aquí tienes tu briefing diario para 24-09-2026:
|
|
||||||
- Tienes 3 reportes de campo pendientes de aprobación.
|
|
||||||
- 2 llamadas perdidas de clientes ayer (contactar en las próximas 2h).
|
|
||||||
… (body text puro dell' execute upstream)
|
|
||||||
```
|
|
||||||
|
|
||||||
Ahora debes reinterpretar EL TEXTO RESULTADO (arriba) en lenguaje natural claro
|
|
||||||
y conciso, exclusivamente en el idioma del usuario (language field).
|
|
||||||
No menciones la herramienta internamente ni detalles técnicos.
|
|
||||||
Responde SOLAMENTE con el contenido final que le quieres transmitir al usuario.
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Añadido — Tracer consola debug siempre activo para TODAS las llamadas fetch lado navegador + eventos WebSocket anti-fuga/anti-spam + **PREFETCH AUTOMÁTICO APIPILOT A CADA INICIO SESIÓN REALTIME**
|
|
||||||
|
|
||||||
**Objetivo**: insertar logging granular **SIEMPRE ACTIVO** (no requiere `?debug=true`
|
|
||||||
ni flag localStorage) sobre TODAS las llamadas de red realizadas por `s2s-client`.
|
|
||||||
Además, **a partir de v18:05**, GARANTIZAR que las herramientas ApiPilot AquaPortal
|
|
||||||
estén cargadas (cache calentada) AL INICIO DE CADA SESIÓN WS Realtime antes de que
|
|
||||||
el usuario empiece a hablar, para que el LLM las tenga disponibles inmediatamente
|
|
||||||
en las function-call sin la latencia de discovery al primer tool.
|
|
||||||
|
|
||||||
**Arquitectura**:
|
|
||||||
- **HTTP fetch** → interceptación global via `window.fetch` monkey-patch (primer archivo cargado, `main.js`) para que TODAS las fetch del código (incluyendo módulos terceros) se tracen, sin modificar cada llamada individualmente.
|
|
||||||
- **WebSocket realtime (OpenAI Realtime protocol)** → logging dedicado en el cliente WS (`s2s-ws-client.js`) con anti-spam para chunks audio 25 Hz y anti-fuga token en la conexión.
|
|
||||||
- **Llamadas lado servidor (ej. `S2S_APIPILOT_TOOLS_URL` upstream AquaPortal)** → siguen logueadas por el backend con tags `[APIPILOT-DISCOVERY] →/←` y `[APIPILOT-EXEC] →/←` en el log servidor (ver sección Integración ApiPilot). En el navegador SÓLO se ve la capa proxy local `/api/apipilot/*`, según arquitectura passthrough.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### 1. Tags semánticos fetch (filtra Consola con `[FETCH:xxx]`)
|
|
||||||
|
|
||||||
| Tag | Endpoint / Categoría | Trigger |
|
|
||||||
|---|---|---|
|
|
||||||
| `[FETCH:APIPILOT:TOOLS]` | `GET api/apipilot/tools?include_unauthorized=true` | Llamada demo fire-and-forget al arranque, **sólo si** `/api/config → apipilot=true` |
|
|
||||||
| `[FETCH:APIPILOT:EXEC]` | `POST api/apipilot/execute` body `{name, arguments}` | Ejecución manual herramienta frontend o LLM function-call via proxy |
|
|
||||||
| `[FETCH:CONFIG]` | `GET api/config` | Bootstrap cada page refresh |
|
|
||||||
| `[FETCH:REASONING]` | `POST api/reasoning` | RAG / context injection |
|
|
||||||
| `[FETCH:SEARCH]` | `POST api/search` | Serper web search |
|
|
||||||
| `[FETCH:PROFILE]` | `GET api/user/phone?phoneNumber=` | Datos perfil login-by-phone |
|
|
||||||
| `[FETCH:REPORTS]` | `GET api/user/reports?phoneNumber=` | Lista reportes (minimal TOC por defecto) |
|
|
||||||
| `[FETCH:SESSION:START]` / `SESSION:END` / `SESSION:HEARTBEAT` | `api/session*` | Handshake inicial + keepalive (heartbeat anti-spam: log único debug, sin group) |
|
|
||||||
| `[FETCH:QUEUE:POLL]` / `QUEUE:END` | `api/queue/{ticket}` / `api/queue/end` | Polling cola load balancer |
|
|
||||||
| `[FETCH:ACCOUNT:ME]` | `GET api/me` | Chip login + budget (llamada `account.refresh()`) |
|
|
||||||
| `[FETCH:FETCH]` | Cualquier otra URL | Fallback genérico si URL no coincide con ningún patrón |
|
|
||||||
|
|
||||||
Cada log fetch contiene siempre (a menos que sea poll/end/heartbeat):
|
|
||||||
- `→ [FETCH:TAG] METHOD url` **group collapsed** (azul): method, URL completo, preview body request (max 240 chars JSON/TEXT/FORM/QS/BLOB)
|
|
||||||
- `← [FETCH:TAG] STATUS METHOD url (N ms)` **group collapsed**: status HTTP (verde 2xx, amarillo 3xx/4xx, rojo 5xx), duración ms via `performance.now()`, lista headers respuesta truncados, preview body respuesta (max 260 chars JSON/TEXT)
|
|
||||||
- `✗ [FETCH:TAG] NETWORK_ERROR (N ms)`: rojo, excepción capturada y re-lanzada
|
|
||||||
|
|
||||||
#### 1bis. Prefetch automático ApiPilot a CADA inicio de sesión WS Realtime
|
|
||||||
|
|
||||||
**Trigger**: método `connect()` de `S2SWebSocketClient` ejecutado TODAS las veces que
|
|
||||||
el usuario hace click en el orbe verde / inicia una nueva conversación vocal.
|
|
||||||
Se ejecuta en **paralelo** con `_setupAudio()` y `_openWebSocket()` (Promise.all),
|
|
||||||
por lo que **NO añade latencia** al handshake.
|
|
||||||
|
|
||||||
| Característica | Valor |
|
|
||||||
|---|---|
|
|
||||||
| Endpoint cliente | `GET api/apipilot/tools?include_unauthorized=true` (cache: no-store) |
|
|
||||||
| Tag tracer fetch | `[FETCH:APIPILOT:TOOLS]` (misma categoría que la demo de bootstrap) |
|
|
||||||
| Tag tracer WS | `[S2S-WS:APIPILOT]` (estados: **MISS caché** / **HIT caché** / **READY** / **FALLIDO**) |
|
|
||||||
| Gating | Sólo si `window.__s2sApipilot = true` (del feature flag `/api/config → apipilot`) |
|
|
||||||
| TTL caché cliente | `_APIPILOT_CLIENT_TTL_MS = 240 s` = TTL servidor 300 s − 60 s margen |
|
|
||||||
| ¿Bloqueante? | **NUNCA** — `Promise.all([…, apipilotReady.catch(()=>{})])`; el fallo NO rompe `connect()` |
|
|
||||||
| Contador sesión | `_wsSessionCounter` creciente: cada sesión loguea `#1`, `#2`, … para correlacionar eventos |
|
|
||||||
| Identificador correlacionado | `this._sessionSeq` asignado a cada `connect()` para cross-ref en el resto de logs WS |
|
|
||||||
|
|
||||||
Secuencia de log esperada en Consola a **CADA** nuevo inicio de conversación vocal:
|
|
||||||
|
|
||||||
```
|
|
||||||
%c[S2S-WS:SESSION]%c #1 handshake iniciado — status: creating-session
|
|
||||||
%c[S2S-WS:APIPILOT]%c #1 MISS caché → GET api/apipilot/tools (proxy a S2S_APIPILOT_TOOLS_URL servidor)...
|
|
||||||
→ [FETCH:APIPILOT:TOOLS] GET api/apipilot/tools?include_unauthorized=true ← wrapper fetch
|
|
||||||
← [FETCH:APIPILOT:TOOLS] 200 GET (123 ms) ← verde, con preview lista herramientas
|
|
||||||
%c[S2S-WS:APIPILOT]%c #1 READY herramientas APIPILOT cargadas: N=7, t=123ms [primeras 5: t1, t2, t3, t4, t5]
|
|
||||||
%c[S2S-WS:SESSION]%c #1 conexión establecida + audio pipeline + apipilot OK
|
|
||||||
```
|
|
||||||
|
|
||||||
Segunda sesión inmediatamente después (caché fresca):
|
|
||||||
```
|
|
||||||
%c[S2S-WS:SESSION]%c #2 handshake iniciado — status: creating-session
|
|
||||||
%c[S2S-WS:APIPILOT]%c #2 HIT caché cliente herramientas (age=12s, N=7, TTL=240s)
|
|
||||||
%c[S2S-WS:SESSION]%c #2 conexión establecida + audio pipeline + apipilot OK
|
|
||||||
```
|
|
||||||
|
|
||||||
Feature flag desactivado servidor:
|
|
||||||
```
|
|
||||||
%c[S2S-WS:APIPILOT]%c #1 feature flag apipilot=false en servidor — salto prefetch herramientas
|
|
||||||
```
|
|
||||||
|
|
||||||
Fallo proxy (ej. backend AquaPortal caído):
|
|
||||||
```
|
|
||||||
%c[S2S-WS:APIPILOT]%c #1 prefetch FALLIDO: HTTP 503 — diagnóstico en log servidor [APIPILOT-DISCOVERY]
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 2. Tags WebSocket (filtra Consola con `[S2S-WS:xxx]`)
|
|
||||||
|
|
||||||
| Tag / Badge | Evento traceado | Notas anti-spam / anti-fuga |
|
|
||||||
|---|---|---|
|
|
||||||
| `🔵 [S2S-WS:CONNECT]` | `_openWebSocket()`: intento conexión WS | ✅ **Anti-fuga**: URL conexión loguea sólo host+path + params ENMASCARADOS: cada param con nombre `token\|session\|auth\|jwt\|secret\|key` ó valor `length > 16` → mostrado como `<prefix>…<last4> len=<N>` para no fugar el JWT en claro en la consola |
|
|
||||||
| `🟢 [S2S-WS:OPEN]` / `🔴 [S2S-WS:ERR]` | `onopen` / `onerror` | Badges coloreados, header respuesta (si disponible) |
|
|
||||||
| `⚪ [S2S-WS:SESSION]` | `connect()` inicio/fin + `_postSession()` grant/queued/402/503 | Ahora incluye contador `#N` sesión incrementado a cada `connect()` + log "establecida + audio + apipilot OK" final |
|
|
||||||
| `🔧 [S2S-WS:APIPILOT]` | `_prefetchApipilotTools()` a cada inicio sesión | estados: **flag off (warn)** / **HIT caché (ok)** / **MISS caché (connect)** / **READY (ok)** / **FALLIDO (err)** — SIEMPRE no bloqueante |
|
|
||||||
| `🟡 [S2S-WS:QUEUE]` | `_pollQueue()`: ticket / advancement / 402 / 404 | Log primer poll, cada 10°, cada avance posición |
|
|
||||||
| `⏺️ [S2S-WS:SEND]` | `_send()`: mensajes salientes | ✅ **Anti-spam**: `type=input_audio_buffer.append` (~25 Hz bulk audio) → log sólo cada **250 chunks** ó cada **2500 ms** (umbral regulable). Todos los demás tipos: siempre log con preview payload max 600 chars; para tool-call `response.function_call_arguments.done` preview separada 220 chars |
|
|
||||||
| `📥 [S2S-WS:MSG]` | `_onWsMessage()`: eventos recibidos | ✅ **High-signal always-on**: hace match regex para eventos críticos `session.*`, `error`, `conversation.item.*`, `response.* done`, `function_call`, `rate_limits.updated` → siempre logueados. Eventos transcript/partial `input_audio_buffer.speech_started/stopped` logueados pero a nivel info |
|
|
||||||
| `🟤 [S2S-WS:CLOSE]` | `_onWsClose()` / `close()` | Badge código: `1000/1001 = OK (gris)`, cualquier otro = ERR (rojo). Reason logueado si existe |
|
|
||||||
|
|
||||||
Contador chunks audio global: `window.__s2sMicChunkCount` para diagnosticar saturación micrófono 25 Hz.
|
|
||||||
Contador sesiones totales: `window._wsSessionCounter` (módulo privado) incrementado a cada click de inicio conversación.
|
|
||||||
|
|
||||||
#### 3. Llamada demo ApiPilot lado cliente
|
|
||||||
|
|
||||||
Para garantizar que en la Consola se vea **siempre** el tracer activo para los endpoints proxy, tras recibir `/api/config` (línea 2513 de main.js) — si y sólo si el feature flag `json.apipilot = true` (es decir, `S2S_APIPILOT_TOOLS_URL` valorado en el servidor) — se envía una fetch `GET api/apipilot/tools?include_unauthorized=true` (fire-and-forget, cache: no-store).
|
|
||||||
|
|
||||||
La demo produce:
|
|
||||||
- `🔧 [APIPILOT-CLIENT] flag=true → DEMO CALL api/apipilot/tools?include_unauthorized=true` (badge morado, antes del tracer)
|
|
||||||
- Luego **el wrapper fetch loguea** `→ [FETCH:APIPILOT:TOOLS] GET api/apipilot/tools?...` + `← [FETCH:APIPILOT:TOOLS] 200 GET (N ms)` (verde) con la lista herramientas en preview
|
|
||||||
- Finalmente `🔧 [APIPILOT-CLIENT] demo OK: N herramientas [primeras 5: tool1, tool2,…]` (badge verde) ó `…demo FAIL` (ámbar) con sugerencia de mirar `[APIPILOT-DISCOVERY]` en el log servidor
|
|
||||||
|
|
||||||
Valor exportado en `window.__s2sApipilot = bool` para tests manuales desde Consola:
|
|
||||||
```js
|
|
||||||
// Ejecuta una herramienta manualmente para ver [FETCH:APIPILOT:EXEC] en el tracer
|
|
||||||
fetch("api/apipilot/execute", {
|
|
||||||
method: "POST",
|
|
||||||
headers: {"Content-Type": "application/json"},
|
|
||||||
body: JSON.stringify({name: "nombre_herramienta", arguments: {parametro: "valor"}})
|
|
||||||
}).then(r => r.json()).then(console.log);
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 4. Archivos modificados (con referencias de línea)
|
|
||||||
|
|
||||||
- **`s2s-client/main.js`**:
|
|
||||||
- Banner versión actualizado: [L28-L31](file:///home/duxiter/s2s/s2s-client/main.js#L28-L31)
|
|
||||||
- Wrapper fetch global `_installFetchDebugTracer()` IIFE — clasificación tags, preview request/response, paleta status, performance timing, anti-spam poll/heartbeat: [L41-L251](file:///home/duxiter/s2s/s2s-client/main.js#L41-L251)
|
|
||||||
- `fetchConfig()` — nuevo flag `apipilotEnabled` + `window.__s2sApipilot`: [L2526-L2537](file:///home/duxiter/s2s/s2s-client/main.js#L2526-L2537)
|
|
||||||
- `fetchConfig()` — llamada demo fire-and-forget ApiPilot tools list: [L2581-L2617](file:///home/duxiter/s2s/s2s-client/main.js#L2581-L2617)
|
|
||||||
- **`s2s-client/ws/s2s-ws-client.js`**:
|
|
||||||
- Banner tracer WS actualizado + paleta `_wsLog(status, tag, msg)` status 0..3 + contador chunks `_wsMicChunkCount` + caché cliente ApiPilot `_apipilotClientCache` TTL 240 s + contador sesiones `_wsSessionCounter`: [L56-L98](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L56-L98)
|
|
||||||
- `connect()` — contador #N sesión, log inicio/fin handshake, Promise.all([audioReady, wsReady, **apipilotReady**]): [L320-L370](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L320-L370)
|
|
||||||
- **`_prefetchApipilotTools(sessionIdShort)`** — NUEVO MÉTODO: check feature flag, caché TTL cliente 240 s, fetch `api/apipilot/tools` (proxy servidor `S2S_APIPILOT_TOOLS_URL`), guardado caché, chequeo envelope, badge log MISS/HIT/READY/FAIL/FALLIDO — NUNCA re-lanza excepción: [L372-L441](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L372-L441)
|
|
||||||
- `_postSession()` — timing handshake grant/queued/402/503: [L469-L520](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L469-L520)
|
|
||||||
- `_pollQueue()` — poll counter + advancement: [L523-L573](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L523-L573)
|
|
||||||
- `_openWebSocket()` — ENMASCARAMIENTO URL connect params (anti-fuga token/JWT): [L682-L728](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L682-L728)
|
|
||||||
- `_onWsMessage()` — high-signal regex, preview `response.function_call_arguments.done`: [L755-L812](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L755-L812)
|
|
||||||
- `_onWsClose()` — badge status code OK/ERR: [L1161-L1180](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L1161-L1180)
|
|
||||||
- `_send()` — anti-spam `input_audio_buffer.append` 250 chunks / 2500 ms: [L1523-L1584](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L1523-L1584)
|
|
||||||
- **`s2s-client/ui/account.js`**:
|
|
||||||
- Llamada `GET api/me` traceada automáticamente por el wrapper fetch global como `[FETCH:ACCOUNT:ME]` → sin modificaciones necesarias (ningún log duplicado).
|
|
||||||
|
|
||||||
#### 5. Dónde ver las llamadas UPSTREAM (S2S_APIPILOT_TOOLS_URL)
|
|
||||||
|
|
||||||
⚠️ **Importante — arquitectura passthrough**:
|
|
||||||
- ✅ **Consola navegador** → ves `[FETCH:APIPILOT:TOOLS]` y `[FETCH:APIPILOT:EXEC]` = llamadas locales al proxy `/api/apipilot/*` en el servidor S2S
|
|
||||||
- ✅ **Log servidor (terminal backend)** → ves `[APIPILOT-DISCOVERY] → GET https://aquachile.azurianlab.com/api/v1/apipilot/tools?...` y `[APIPILOT-EXEC] → POST …/execute` = llamadas reales upstream a la URL AquaPortal (`S2S_APIPILOT_TOOLS_URL`)
|
|
||||||
|
|
||||||
Esto es así por diseño: el cliente nunca conoce la URL upstream AquaPortal (anti-hardcoding), todo pasa por el proxy local en `server.py` que añade headers auth y validación schema PRE-CHECK.
|
|
||||||
|
|
||||||
### Solucionado — Bug: ApiPilot llamado 1 sola vez (anónimo) y NUNCA más tras login phone o refresh
|
|
||||||
|
|
||||||
**Síntoma observado del log aquachile aportado**:
|
|
||||||
- A las `21:06:46` 1 única llamada `[apipilot] list_tools.start user=(anon)` a `/api/v1/apipilot/tools` (llamada demo anónima bootstrap)
|
|
||||||
- Inmediatamente después: múltiples `POST /auth/phone-login` y `GET /auth/user-by-phone` (usuarios autenticados, 21:06:49, 21:07:16, 21:07:19, 21:07:39) pero **NINGUNA** llamada apipilot repetida.
|
|
||||||
|
|
||||||
**Causa raíz doble**:
|
|
||||||
1. **Caché cliente global (240 s) user-agnostic**: la demo bootstrap anónima y las sesiones autenticadas compartían la misma entry `_apipilotClientCache` sin clave por usuario. Por tanto la caché anónima HIT se reutilizaba para usuarios autenticados DIFERENTES.
|
|
||||||
2. **Caché servidor S2S (300 s) keyed por auth fingerprint** (`SHA-256(auth)[:12] | x-demo-session`) pero desde el cliente **no se propagaba ningún token `x-demo-session`**, así todas las llamadas usaban la fingerprint "vacía" (anónima) = caché anónima reutilizada sin llamar nunca AquaPortal para usuarios autenticados.
|
|
||||||
3. **Trigger demo solo al bootstrap**: ninguna llamada apipilot después de `phone-login` correcto, que es el momento en que cambia el contexto de usuario.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Fix aplicados
|
|
||||||
|
|
||||||
##### Fix 1 — Prefetch ApiPilot DESPUÉS de CADA login phone correcto (con `refresh=true`)
|
|
||||||
Insertado bloque fire-and-forget **justo después** de la confirmación HTTP 200 en `fetchVoceroUserProfile()` (después de cargar perfil):
|
|
||||||
|
|
||||||
| Característica | Valor |
|
|
||||||
|---|---|
|
|
||||||
| Trigger | `api/user/phone?phoneNumber=…` HTTP 200 + objeto profile.result válido |
|
|
||||||
| Endpoint | `GET api/apipilot/tools?include_unauthorized=true&refresh=true` |
|
|
||||||
| Parámetro `refresh=true` | **OBLIGATORIO** → fuerza al proxy S2S lado servidor a **ignorar la caché 300 s** y llamar realmente AquaPortal upstream (así `[apipilot] list_tools.start user=…` reaparece en el log aquachile PARA CADA login phone) |
|
|
||||||
| Headers | Propaga `x-demo-session` si existe en `window.__s2sDemoSessionToken` (para fingerprint auth S2S distinta a anónima) |
|
|
||||||
| Efecto colateral | Invoca `window.__s2sInvalidateApipilotCache()` = invalida caché cliente WS para todas las sesiones vocales sucesivas → ninguna reutilización de caché para nuevo usuario |
|
|
||||||
| Log Consola | `🔧 [APIPILOT-CLIENT] post-login phone → FUERZO refresh api/apipilot/tools (badge morado)` + `post-login OK (dt=XYZ ms): N=X herramientas` o `post-login FAIL` |
|
|
||||||
|
|
||||||
Referencia de línea: [main.js#L2418-L2464](file:///home/duxiter/s2s/s2s-client/main.js#L2418-L2464)
|
|
||||||
|
|
||||||
##### Fix 2 — Caché cliente WS por-usuario (invalida si cambia phone/token)
|
|
||||||
La entry `_apipilotClientCache` ahora incluye la propiedad `key = _apipilotUserKey()`.
|
|
||||||
|
|
||||||
Clave `_apipilotUserKey()` calculada como:
|
|
||||||
1. `P:<últimos-8-dígitos-phoneNumber>` → si existe `?phoneNumber=` en URL
|
|
||||||
2. `T:<últimos-6-char-x-demo-session-token>` → si existe token
|
|
||||||
3. `anon` → en otro caso
|
|
||||||
|
|
||||||
HIT caché ahora requiere **3 condiciones**:
|
|
||||||
1. `now - cached.at <= 240 s` (TTL aún activo)
|
|
||||||
2. `cached.key === currentUser` (mismo phone / mismo token → mismo usuario)
|
|
||||||
3. Feature flag `window.__s2sApipilot = true`
|
|
||||||
|
|
||||||
Referencia de línea: [s2s-ws-client.js#L94-L126](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L94-L126) (caché+clave+invalidación global) · [L411-L485](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L411-L485) (método `_prefetchApipilotTools` con check clave usuario, refresh=true si cambia usuario y propagate header x-demo-session)
|
|
||||||
|
|
||||||
##### Fix 3 — Propagación header `x-demo-session` para huella auth lado servidor S2S
|
|
||||||
Ahora, **cuando exista** `window.__s2sDemoSessionToken` (length > 4), ambas llamadas apipilot (prefetch login + WS connect) propagan:
|
|
||||||
```
|
|
||||||
x-demo-session: <token>
|
|
||||||
```
|
|
||||||
→ la caché lado servidor S2S (`SHA-256(auth)[:12] | x-demo-session`) usa clave **distinta para cada token/sesión** = **NINGUNA compartición caché entre Anónimo y Usuario Autenticado** ni entre usuarios distintos.
|
|
||||||
|
|
||||||
##### Fix 4 — Función global invalidación caché para cross-módulo
|
|
||||||
```js
|
|
||||||
window.__s2sInvalidateApipilotCache(); // resetea _apipilotClientCache = null
|
|
||||||
```
|
|
||||||
Llamada automáticamente por `main.js` tras login phone correcto. Útil también para tests manuales desde Consola navegador.
|
|
||||||
|
|
||||||
Referencia de línea: [s2s-ws-client.js#L111-L126](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L111-L126)
|
|
||||||
|
|
||||||
##### Cronograma esperado en el log AquaPortal DESPUÉS de este fix (log aportado repetido)
|
|
||||||
```
|
|
||||||
21:06:46 [apipilot] list_tools.start user=(anon) ← demo bootstrap anónimo (refresh page)
|
|
||||||
21:06:46 [apipilot] list_tools.ok count=1 …
|
|
||||||
21:06:49 [apipilot] list_tools.start user=<username-A> ← FIX #1 post-login phone refresh=true ✅ NUEVO
|
|
||||||
21:06:49 [apipilot] list_tools.ok count=…
|
|
||||||
21:07:16 [apipilot] list_tools.start user=<username-B> ← FIX #1 login nuevo usuario ✅ NUEVO
|
|
||||||
21:07:16 [apipilot] list_tools.ok …
|
|
||||||
21:07:19 [apipilot] list_tools.start user=<username-A> ← si caché cliente S2S invalidada o
|
|
||||||
21:07:19 … x-demo-session cambiado
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Añadido — Integración ApiPilot AquaPortal (herramientas dinámicas, sin hardcoding)
|
|
||||||
|
|
||||||
Implementación lado proxy `s2s-client/server.py` del puente definido en
|
|
||||||
[GUIDA_APIPILOT_CLIENTE.md](file:///home/duxiter/s2s/GUIDA_APIPILOT_CLIENTE.md).
|
|
||||||
|
|
||||||
**Objetivo**: permitir al cliente Vocero **consumir dinámicamente** las
|
|
||||||
herramientas expuestas por el backend AquaPortal vía ApiPilot, **sin hardcodear
|
|
||||||
nombres de endpoints, parámetros ni autorizaciones** en el código cliente.
|
|
||||||
Todo el conocimiento sobre "qué herramientas existen / qué parámetros aceptan /
|
|
||||||
si requieren autenticación" se **lee en runtime** vía descubrimiento
|
|
||||||
estandarizado.
|
|
||||||
|
|
||||||
#### Entorno requerido (`.env`)
|
|
||||||
|
|
||||||
| Variable | Valor ejemplo | Significado |
|
|
||||||
|---|---|---|
|
|
||||||
| **`S2S_APIPILOT_TOOLS_URL`** | `https://aquachile.azurianlab.com/api/v1/apipilot` | URL **base** (sin `/tools` ni `/execute`) del backend ApiPilot. Si está vacía → feature completamente desactivada (passthrough). |
|
|
||||||
| `S2S_APIPILOT_SERVICE_TOKEN` | `optional_Bearer_service_token` | Token de respaldo usado solo si el llamante no envía `Authorization` ni `x-demo-session`. |
|
|
||||||
| `S2S_APIPILOT_TTL_SEC` | `300` | TTL cache en memoria del descubrimiento de herramientas (por defecto 300 s = 5 min — según guía). Rango admitido [15, 3600]. |
|
|
||||||
| `S2S_APIPILOT_TIMEOUT_SEC` | `15.0` | Timeout HTTP descubrimiento + ejecución (por defecto 15 s). Rango [1, 120]. |
|
|
||||||
|
|
||||||
#### Endpoints públicos expuestos por el proxy (mismo origen)
|
|
||||||
|
|
||||||
| Endpoint | Verbo | Descripción |
|
|
||||||
|---|---|---|
|
|
||||||
| `/api/apipilot/tools?include_unauthorized=true&refresh=false` | **GET** | Lista herramientas conforme §2.1 (`ToolDefinition`: `name, description, parameters, requires_auth`). Respuesta envuelta `{success, data:{count, include_unauthorized, tools}, error}`. Soporta `?refresh=true` para forzar refresco de caché. |
|
|
||||||
| `/api/apipilot/execute` | **POST** | Ejecuta una herramienta conforme §2.2. Body: `{name: string, arguments: {}}`. PRE-CHECK obligatorios antes de la llamada HTTP. Respuesta SIEMPRE en envoltorio `{success, data, error}`. HTTP status code devuelto = `error.status_code` cuando existe (4xx/5xx). |
|
|
||||||
|
|
||||||
Extensión de `/api/config`: nueva clave booleana de primer nivel
|
|
||||||
`"apipilot": true/false` (basada en `S2S_APIPILOT_TOOLS_URL`). El frontend la
|
|
||||||
usa para decidir si mostrar el catálogo / los botones "Ejecutar herramienta"
|
|
||||||
(obtiene la lista desde `/api/apipilot/tools` en vez de hardcodearla).
|
|
||||||
|
|
||||||
#### PRE-CHECK obligatorios (flujo guía §2.2, todos implementados)
|
|
||||||
|
|
||||||
1. **Descubrimiento fresco** — antes de cada `execute` se usa la lista tools
|
|
||||||
recuperada por `_apipilot_fetch_tools(...)` respetando la TTL. Si la
|
|
||||||
herramienta no se encuentra, **una sola** re-descubrimiento forzado antes
|
|
||||||
de fallar.
|
|
||||||
2. **Control `requires_auth`** — si `true` pero ningún token presente
|
|
||||||
(Authorization / x-demo-session / S2S_APIPILOT_SERVICE_TOKEN) → bloqueo
|
|
||||||
HTTP 401 con envoltorio `{code: "AUTH_REQUIRED", message: "La
|
|
||||||
herramienta «nombre» requiere autenticación…"}` — **NINGUNA solicitud se
|
|
||||||
envía al backend upstream** en este caso.
|
|
||||||
3. **Validación JSON-Schema strict** (`_apipilot_validate_arguments`):
|
|
||||||
- elimina **TODAS** las claves extra no presentes en `parameters.properties`
|
|
||||||
(aviso `[APIPILOT-VALIDATE] … removing unknown arg=…`);
|
|
||||||
- **NO inventa parámetros**;
|
|
||||||
- devuelve error 400 `INVALID_ARGUMENTS` por cada campo `required`
|
|
||||||
faltante (mensajes en español).
|
|
||||||
4. **Passthrough cabeceras autenticación**:
|
|
||||||
`Authorization: Bearer <TOKEN_USUARIO>` o bien `x-demo-session:
|
|
||||||
<SESSION_TOKEN>` se reenvían VERBATIM al backend AquaPortal (§2.1 nota),
|
|
||||||
**NUNCA se loguean** (solo presencia: log `auth=Y / demo=Y`).
|
|
||||||
5. **Compatibilidad LLM function-calling**: helper
|
|
||||||
`_apipilot_strip_tool_for_llm(tool_def)` que proyecta `ToolDefinition` al
|
|
||||||
formato estándar OpenAI `{type: "function", function: {name, description,
|
|
||||||
parameters}}` — **elimina `requires_auth`** (§4 guía: "no mandar
|
|
||||||
directamente la salida cruda de /tools al modelo").
|
|
||||||
|
|
||||||
#### Garantías / seguridad / anti-leak
|
|
||||||
|
|
||||||
| Control | Implementación |
|
|
||||||
|---|---|
|
|
||||||
| Sin compartir caché entre identidades distintas | Clave caché = `SHA-256(authorization)[:12] \| x-demo-session` → huella anónima, no guarda tokens en caché |
|
|
||||||
| Doble comprobación caché con candado asíncrono | `_APIPILOT_CACHE_LOCK` async + double check tras adquirir lock → 1 sola fetch por refresco |
|
|
||||||
| Fallback obsoleto ante error de red | Si backend cae y tenemos entrada en caché → devuelve fallback logueando warning en vez de crash |
|
|
||||||
| Reenvío cabecera Auth nunca logueado | Solo log `auth=Y/- demo=Y/-` |
|
|
||||||
| Filtrado parámetros unknown | Eliminados, nunca reenviados al backend |
|
|
||||||
| Envoltorio errores uniforme | Siempre `{success, data, error}` incluso para errores pre-check locales |
|
|
||||||
| Nombres env coherentes | Ninguna URL ni token hardcodeados |
|
|
||||||
| Banner startup confirmación | `[STARTUP] … apipilot=<URL> apipilotServiceToken=Y/- apipilotTtl=<N>s apipilotTimeout=<N>s` |
|
|
||||||
|
|
||||||
**Archivos modificados:**
|
|
||||||
|
|
||||||
- **`server.py`** ([server.py](file:///home/duxiter/s2s/s2s-client/server.py)) — Sección APIPILOT completa insertada entre `OPENAI_PROJECT` y `NODE_RED_REASONING_URL`:
|
|
||||||
1. Definición 4 env + normalización + límites rango TTL/timeout
|
|
||||||
([L236-L266](file:///home/duxiter/s2s/s2s-client/server.py#L236-L266)).
|
|
||||||
2. Caché en-proceso `_APIPILOT_CACHE: dict[fp → (monotonic, tools[])]` + lock
|
|
||||||
asíncrona `_APIPILOT_CACHE_LOCK` ([L268-L271](file:///home/duxiter/s2s/s2s-client/server.py#L268-L271)).
|
|
||||||
3. Helpers puros: `_apipilot_auth_fingerprint(auth, demo)`
|
|
||||||
([L274-L287](file:///home/duxiter/s2s/s2s-client/server.py#L274-L287)) · `_apipilot_parse_authorization(Request)`
|
|
||||||
([L290-L309](file:///home/duxiter/s2s/s2s-client/server.py#L290-L309)) · `_apipilot_strip_tool_for_llm(tool_def)`
|
|
||||||
([L312-L338](file:///home/duxiter/s2s/s2s-client/server.py#L312-L338)).
|
|
||||||
4. Descubrimiento: `_apipilot_fetch_tools(auth, demo, include_unauthorized, force_refresh)`
|
|
||||||
([L341-L481](file:///home/duxiter/s2s/s2s-client/server.py#L341-L481)) — log cache hit, fallback obsoleto, desempaquetado payload `list/{tools,result,data}.tools`, saneamiento mínimo ToolDefinition, logs `[APIPILOT-DISCOVERY] →/←`.
|
|
||||||
5. Validación: `_apipilot_validate_arguments(arguments, tool_def)`
|
|
||||||
([L484-L517](file:///home/duxiter/s2s/s2s-client/server.py#L484-L517)) — elimina extras, check required, mensajes en español.
|
|
||||||
6. Ejecución: `_apipilot_execute_tool(auth, demo, name, arguments)`
|
|
||||||
([L520-L684](file:///home/duxiter/s2s/s2s-client/server.py#L520-L684)) — PRE-CHECK 1-2-3 (re-discovery 1 volta, requires_auth 401, validación 400), POST /execute envoltorio, manejo HTTP no-JSON, wrap envoltorio si backend no lo devuelve, logs `[APIPILOT-EXEC] →/←`.
|
|
||||||
7. Banner `[STARTUP] Other externals`: extendido con 4 nuevos placeholders `apipilot`, `apipilotServiceToken`, `apipilotTtl`, `apipilotTimeout`
|
|
||||||
([L726-L752](file:///home/duxiter/s2s/s2s-client/server.py#L726-L752)).
|
|
||||||
8. **`/api/config`**: nueva clave booleana `"apipilot": bool(S2S_APIPILOT_TOOLS_URL)` + docstring, log extendido
|
|
||||||
([L2544-L2579](file:///home/duxiter/s2s/s2s-client/server.py#L2544-L2579)).
|
|
||||||
9. **Modelo Pydantic** `ApipilotExecuteRequest(name:str, arguments:dict={})` ([L2582-L2585](file:///home/duxiter/s2s/s2s-client/server.py#L2582-L2585)).
|
|
||||||
10. **`GET /api/apipilot/tools`** proxy público ([L2588-L2651](file:///home/duxiter/s2s/s2s-client/server.py#L2588-L2651)):
|
|
||||||
query `?include_unauthorized=true(default)&refresh=false`, envoltorio uniforme `{success, data: {count, include_unauthorized, tools}, error}`, 501 si env no configurado, HTTPException convertidas a envoltorio.
|
|
||||||
11. **`POST /api/apipilot/execute`** proxy público ([L2654-L2700](file:///home/duxiter/s2s/s2s-client/server.py#L2654-L2700)):
|
|
||||||
body `{name, arguments}`, PRE-CHECK 1-2-3 vía `_apipilot_execute_tool`, código status respuesta = `error.status_code` si error, mensajes de cara al usuario en español §2.2.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [2026-09-23]
|
|
||||||
|
|
||||||
### Añadido — Botón «Descargar conversación» en la cabecera del panel chat (export JSON + TXT client-side offline)
|
|
||||||
|
|
||||||
Nuevo botón circular icono descarga junto al testigo «En línea» a la derecha
|
|
||||||
de la cabecera del panel `#chat-panel`. Al pulsar serializa TODO el
|
|
||||||
historial persistente (`.hist-msg` dentro de `#chat-history`) y descarga DOS
|
|
||||||
archivos simultáneamente, SIN llamadas al backend, 100 % client-side y
|
|
||||||
offline-safe (Blob + URL.createObjectURL + `<a download>`):
|
|
||||||
|
|
||||||
1. `vocero-conversacion[-<nombreUsuario>]-YYYYMMDD-HHMMSS.json` — estructura
|
|
||||||
con metadatos `{ exportedAt, messageCount, messages: [...] }`. Tipos soportados:
|
|
||||||
- `user` / `assistant` → `{ type, text, timestamp, name?, initials?, interrupted? }`
|
|
||||||
- `tool` → `{ type: "tool", timestamp, name, args, output }` (resultado llamadas tool)
|
|
||||||
- `image` → `{ type: "image", timestamp, dataUrl }` (capturas webcam camera tool)
|
|
||||||
2. `vocero-conversacion[-<nombreUsuario>]-YYYYMMDD-HHMMSS.txt` — log legible
|
|
||||||
para humanos en español, cada entrada formateada como
|
|
||||||
`HH:MM Nombre / Vocero / HERRAMIENTA <nombre>: texto`.
|
|
||||||
|
|
||||||
**UI / UX (todo en español, como solicitado):**
|
|
||||||
- `aria-label="Descargar conversación"`
|
|
||||||
- `title="Descargar conversación (JSON + TXT)"`
|
|
||||||
- **Feedback OK**: tras export correcto botón se pinta verde 650 ms
|
|
||||||
(`.chat-export-ok` con ring + glow).
|
|
||||||
- **Feedback vacío**: si no hay mensajes → sin descarga, botón flash
|
|
||||||
ámbar + micro-shake 0.42s (`.chat-export-empty`) + console info
|
|
||||||
`[ui/chat.js:exportConversation] No hay mensajes aún — exportación omitida.`
|
|
||||||
- **Nombres de fichero saneados**: helper `_safeName(...)` elimina unicode /
|
|
||||||
caracteres especiales / espacios → `a-z 0-9 - _`.
|
|
||||||
- **Estilos coherentes**: dark (glass/aurora) + white (estilo screenshot móvil)
|
|
||||||
ambos tematizados con colores existentes `var(--brand)`, `#16a34a`, `#f59e0b`.
|
|
||||||
|
|
||||||
**Garantías:**
|
|
||||||
- 🔒 Cero datos transmitidos fuera del navegador (ningún POST, ningún analytics).
|
|
||||||
- Las burbujas efímeras sobre el orbe (`.bubble` / `.bubble-stack`) se
|
|
||||||
ignoran intencionadamente: exporta SOLO el histórico permanente, el que es significativo.
|
|
||||||
- Never-throws: todo el export envuelto en `try/catch`; error solo en consola.
|
|
||||||
- `window.URL.revokeObjectURL(...)` tras 4 s → sin fugas de memoria Blob.
|
|
||||||
- Disponible en TODOS los temas (dark / white / default).
|
|
||||||
|
|
||||||
**Archivos modificados:**
|
|
||||||
|
|
||||||
- **`index.html`** ([index.html](file:///home/duxiter/s2s/s2s-client/index.html)):
|
|
||||||
1. Wrapper `.chat-header-actions` a la derecha de la cabecera, que engloba
|
|
||||||
la pill «En línea» + nuevo botón `#chat-export-btn` (SVG flecha-abajo)
|
|
||||||
([L374-L394](file:///home/duxiter/s2s/s2s-client/index.html#L374-L394)).
|
|
||||||
|
|
||||||
- **`ui/chat.js`** ([chat.js](file:///home/duxiter/s2s/s2s-client/ui/chat.js)):
|
|
||||||
1. Helper module-level `_safeName(prefix)` para sanear nombres de fichero
|
|
||||||
([L38-L50](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L38-L50)).
|
|
||||||
2. Ref `this._exportBtn = getElementById("chat-export-btn")` + listener `click`
|
|
||||||
en constructor `ChatView` ([L74-L78](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L74-L78)).
|
|
||||||
3. Nueva sección `// ── Export / download conversation ──` con 4 métodos:
|
|
||||||
- `_serializeConversation()` → extrae rows `.hist-msg` via querySelector
|
|
||||||
`:scope > .hist-msg`, gestiona `user/assistant/tool/image`, preserva
|
|
||||||
timestamp `data-timestamp`, nombre usuario, initials, flag interrupted
|
|
||||||
([L521-L582](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L521-L582)).
|
|
||||||
- `_conversationToText(conv)` → pretty-format texto en español con
|
|
||||||
cabecera fecha, totales, bloques tool indentados, preview imagen dataURL
|
|
||||||
([L584-L621](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L584-L621)).
|
|
||||||
- `_downloadBlob(content, filename, mime)` → patrón `<a download>` +
|
|
||||||
revokeObjectURL ([L623-L645](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L623-L645)).
|
|
||||||
- `exportConversation()` (público) → entry-point, nombre fichero con
|
|
||||||
sello UTC local `YYYYMMDD-HHMMSS`, incluye nombre perfil Vocero si
|
|
||||||
existe, lógica empty/OK/error feedback, 2 descargas distintas
|
|
||||||
([L647-L697](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L647-L697)).
|
|
||||||
|
|
||||||
- **`style.css`** ([style.css](file:///home/duxiter/s2s/s2s-client/style.css)):
|
|
||||||
1. `.chat-header-actions` flex-row right-aligned
|
|
||||||
([L4919-L4931](file:///home/duxiter/s2s/s2s-client/style.css#L4919-L4931)).
|
|
||||||
2. `.chat-export-btn` base (dark-theme): 32×32 pill, glassy, hover/focus
|
|
||||||
states, transiciones 120-160 ms
|
|
||||||
([L4932-L4968](file:///home/duxiter/s2s/s2s-client/style.css#L4932-L4968)).
|
|
||||||
3. Estados feedback: `.chat-export-ok` (verde) + `.chat-export-empty`
|
|
||||||
(ámbar + animación shake 5 pasos)
|
|
||||||
([L4969-L4989](file:///home/duxiter/s2s/s2s-client/style.css#L4969-L4989)).
|
|
||||||
4. Variantes white-theme completas: `.chat-header-actions`, base, hover,
|
|
||||||
active, OK, empty
|
|
||||||
([L4991-L5015](file:///home/duxiter/s2s/s2s-client/style.css#L4991-L5015)).
|
|
||||||
|
|
||||||
### Añadido — `S2S_LATEST_SUMMARY_TEXT`: override estático del resumen última conversación (texto limpio, ninguna llamada HTTP)
|
|
||||||
|
|
||||||
Nueva variable de entorno que, si se establece, tiene **prioridad absoluta** sobre `S2S_LATEST_SUMMARY`
|
|
||||||
y se salta completamente la fetch HTTP upstream (login-by-phone, Bearer, timeout, JSON unwrap: *todo bypaseado*).
|
|
||||||
Su valor en texto plano se inyecta directamente como bloque "latest conversation summary"
|
|
||||||
dentro del prompt LLM (tanto en el canal WebSocket `/v1/realtime` como en `POST /api/reasoning`).
|
|
||||||
|
|
||||||
Diseñada para el caso de uso en el que el operador quiere inyectar una versión
|
|
||||||
**limpia, esencial y pre-depurada** del resumen sin depender de ningún endpoint externo.
|
|
||||||
|
|
||||||
**Valor `.env` de ejemplo (multilínea con `\n`):**
|
|
||||||
```env
|
|
||||||
# S2S_LATEST_SUMMARY=https://api.example.com/summary # ← desactivado: S2S_LATEST_SUMMARY_TEXT tiene prioridad
|
|
||||||
S2S_LATEST_SUMMARY_TEXT=Última conversación 2026-09-22:\n- Usuario preguntó por mortalidad salmón AquaChile semana 38.\n- Se refirió al reporte Porcentaje Mortalidad.
|
|
||||||
```
|
|
||||||
|
|
||||||
**Comportamiento clave:**
|
|
||||||
- Las secuencias `\n` en el valor env se interpretan como salto de línea real (misma convención que `S2S_PRIMARY_PROMPT`)
|
|
||||||
- **No requiere `phoneNumber`**: funciona también para sesiones genéricas sin identificativo de llamante
|
|
||||||
- El wrapper `LATEST_SUMMARY_HINT` + el tope a **1600 caracteres** se siguen aplicando igualmente (pasa siempre a través de `_resolve_latest_summary_context`)
|
|
||||||
- El bloque se clasifica correctamente por el layer dedup (marker invisible `SUMMARY_FULL_MARKER`) — sin riesgo de duplicado
|
|
||||||
- Si `S2S_LATEST_SUMMARY_TEXT` está vacío → **fallback** transparente al flujo anterior (`S2S_LATEST_SUMMARY` URL + `phoneNumber`)
|
|
||||||
|
|
||||||
**Logs esperados en el servidor para confirmar override activo:**
|
|
||||||
```
|
|
||||||
[INJECT-SUMMARY] S2S_LATEST_SUMMARY_TEXT OVERRIDE active (len=<N> chars). Skipping HTTP fetch; injecting raw static plain-text summary directly.
|
|
||||||
```
|
|
||||||
|
|
||||||
**Archivos modificados:**
|
|
||||||
|
|
||||||
- **`server.py`** ([server.py](file:///home/duxiter/s2s/s2s-client/server.py)):
|
|
||||||
1. Definición nueva env `S2S_LATEST_SUMMARY_TEXT` + interpretación escape `\n` ([L128-L140](file:///home/duxiter/s2s/s2s-client/server.py#L128-L140)).
|
|
||||||
2. Banner `[STARTUP]` extendido con campo `summaryText=%s` para confirmación visual a boot ([L275-L297](file:///home/duxiter/s2s/s2s-client/server.py#L275-L297)).
|
|
||||||
3. **Rama override** ANTES de cualquier lógica HTTP dentro de `_fetch_latest_summary_for_phone(...)` ([L1563-L1589](file:///home/duxiter/s2s/s2s-client/server.py#L1563-L1589)):
|
|
||||||
- return directo del valor estático;
|
|
||||||
- log `[INJECT-SUMMARY] … OVERRIDE active`;
|
|
||||||
- vista previa raw primeros 800 chars si `S2S_DEBUG_PROMPT=true`.
|
|
||||||
4. **Puerta REASONING:** `needs_summary = bool(S2S_LATEST_SUMMARY) or bool(S2S_LATEST_SUMMARY_TEXT)` ([L2153](file:///home/duxiter/s2s/s2s-client/server.py#L2153)).
|
|
||||||
5. **Skip REASONING relajado:** condición `(not phone) and (not needs_reports) and (not episode_id) and (not S2S_LATEST_SUMMARY_TEXT)` — el texto estático no necesita phone ([L2159-L2168](file:///home/duxiter/s2s/s2s-client/server.py#L2159-L2168)).
|
|
||||||
6. **Ejecución summary REASONING:** `needs_summary and (phone or S2S_LATEST_SUMMARY_TEXT)`; log "READY" extendido con flag `summaryTextOverride=%s` ([L2240-L2256](file:///home/duxiter/s2s/s2s-client/server.py#L2240-L2256)).
|
|
||||||
7. **Puerta WS connect:** mismo `needs_summary` extendido con `S2S_LATEST_SUMMARY_TEXT` ([L2644](file:///home/duxiter/s2s/s2s-client/server.py#L2644)).
|
|
||||||
8. **Skip WS connect relajado:** incluso sin `phoneNumber` si hay `S2S_LATEST_SUMMARY_TEXT` la pipeline arranca ([L2653-L2664](file:///home/duxiter/s2s/s2s-client/server.py#L2653-L2664)).
|
|
||||||
9. **Ejecución summary WS:** misma condición relajada + log "READY" con campo `summaryTextOverride=%s` ([L2725-L2746](file:///home/duxiter/s2s/s2s-client/server.py#L2725-L2746)).
|
|
||||||
|
|
||||||
- **`INJECTIONS.md`** ([INJECTIONS.md](file:///home/duxiter/s2s/s2s-client/INJECTIONS.md)):
|
|
||||||
1. Sección **4. Blocco summary ultima conversazione** completamente reescrita (versión italiano original).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [2026-09-07]
|
|
||||||
|
|
||||||
### Añadido — Soporte `episode_id` (inject de episodio podcast)
|
|
||||||
|
|
||||||
Nuevo parámetro query string `?episode_id=` junto a `?phoneNumber=` que
|
|
||||||
habilita la carga y la inyección automática de un episodio podcast como
|
|
||||||
contexto PRIMARIO en el prompt LLM.
|
|
||||||
|
|
||||||
**URL de ejemplo:**
|
|
||||||
```
|
|
||||||
/?phoneNumber=%2B447432477433&episode_id=episode-19569f77-a06e-5ce5-a52f-6299dbd67d03
|
|
||||||
```
|
|
||||||
|
|
||||||
**Archivos modificados:**
|
|
||||||
|
|
||||||
- **`server.py`** ([server.py](file:///home/duxiter/s2s/s2s-client/server.py)):
|
|
||||||
1. Añadida la constante `PODCAST_EPISODE_HINT` ([L426-L429](file:///home/duxiter/s2s/s2s-client/server.py#L426-L429)) para el hint semántico en el prompt.
|
|
||||||
2. Añadida la función `_resolve_episode_context(raw_value)` ([L475-L504](file:///home/duxiter/s2s/s2s-client/server.py#L475-L504)): normaliza la respuesta API (dict/list/str, posible wrapper `{"result": …}`), la serializa en JSON (tope 24.000 chars) y la envuelve con los marcadores `<<<PODCAST_EPISODE_BEGIN/END>>>`.
|
|
||||||
3. Añadida la función asíncrona `_fetch_podcast_episode(episode_id)` ([L1010-L1078](file:///home/duxiter/s2s/s2s-client/server.py#L1010-L1078)): GET directo a
|
|
||||||
`https://aquachile.azurianlab.com/api/v1/podcast/episodes/{episode_id}`
|
|
||||||
(timeout 15s, follow redirects, manejo de errores tolerante — nunca rompe el flujo, devuelve `None` en caso de fallo).
|
|
||||||
4. Extendida `_augment_instructions_with_profile(...)` ([L1137-L1181](file:///home/duxiter/s2s/s2s-client/server.py#L1137-L1181)) con el nuevo parámetro `episode_context_text`; el orden de inject ahora es (más específico → más genérico): **episode → profile → reports → summary**.
|
|
||||||
5. Extendida `_patch_client_session_update_with_profile(...)` ([L1184-L1306](file:///home/duxiter/s2s/s2s-client/server.py#L1184-L1306)) con soporte a `episode_context_text` y logging dedicado `[INJECT-EPISODE]` / `[DEBUG-EPISODE]`.
|
|
||||||
6. WebSocket connect handler ([L1791-L1959](file:///home/duxiter/s2s/s2s-client/server.py#L1791-L1959)):
|
|
||||||
- lee `episode_id` desde query params junto a `phoneNumber`;
|
|
||||||
- ejecuta fetch episodio + resolve ANTES de profile/reports/summary;
|
|
||||||
- guarda `injected_episode_context` e `inject_episode_id` en el `session_ctx` compartido.
|
|
||||||
7. `_client_to_upstream` ([L2649-L2661](file:///home/duxiter/s2s/s2s-client/server.py#L2649-L2661)): pasa `ctx_episode_context` al patch de `session.update`.
|
|
||||||
8. Endpoint `POST /api/reasoning`:
|
|
||||||
- extendido el modelo `ReasoningRequest` con el campo `episode_id: str | None = None` ([L1309-L1323](file:///home/duxiter/s2s/s2s-client/server.py#L1309-L1323));
|
|
||||||
- en el handler ([L1411-L1577](file:///home/duxiter/s2s/s2s-client/server.py#L1411-L1577)) fetch episodio independiente de phone;
|
|
||||||
- `inject_episode_text` se hace **prepend** sobre `user_context_parts` (primero de todos);
|
|
||||||
- reenvío a NODE_RED / OPENAI con logging extendido (`episodeIdLen`).
|
|
||||||
|
|
||||||
- **`main.js`** ([main.js](file:///home/duxiter/s2s/s2s-client/main.js)):
|
|
||||||
1. Lectura desde query string: `const voceroEpisodeId = pageParams.get("episode_id")` ([L676-L680](file:///home/duxiter/s2s/s2s-client/main.js#L676-L680)).
|
|
||||||
2. `fetchOpenAiReasoningAnswer` ([L1337-L1364](file:///home/duxiter/s2s/s2s-client/main.js#L1337-L1364)): campo `episode_id` añadido al body POST `/api/reasoning`, logKey y `console.info` extendidos.
|
|
||||||
3. Construcción URL WebSocket ([L2323-L2342](file:///home/duxiter/s2s/s2s-client/main.js#L2323-L2342)): `episode_id` añadido a query params tanto con la API `URL.searchParams` como en el fallback string (concat con `&`).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Añadido — `S2S_START_SESSION_MODE`: 3 modos de inicio de sesión
|
|
||||||
|
|
||||||
Variable de entorno leída por el servidor y propagada al cliente vía `GET /api/config.startSessionMode`.
|
|
||||||
Objetivo: controlar si y cómo se enciende automáticamente el micrófono y si el modelo saluda
|
|
||||||
primero (flujo similar al wake-word). Valores soportados:
|
|
||||||
|
|
||||||
| Valor | Descripción |
|
|
||||||
|---|---|
|
|
||||||
| *(vacío) / default* | Inicio manual: solo tap sobre el Orb / botón Start |
|
|
||||||
| **`TALK`** | Después de 2s overlay a pantalla completa "Toca en cualquier lugar para empezar" (policy safe para Safari/iOS `getUserMedia`). Click/touch/tecla dentro de la user gesture → `doStart()`. El cliente WS inyecta bootstrap `"hola vocero"` + `requestResponse()`. |
|
|
||||||
| **`TALKFIRST`** | Después de 2s **intenta `doStart()` DIRECTAMENTE SIN click**, precargando `pendingWakeWordConfirmationText = "hola vocero"` (misma variable de la wake-word real). Si los permisos de mic ya están "allow always" funciona sin click. Si falla (policy Safari/iOS) → fallback sobre el mismo unlocker de TALK con mensaje "permiso micrófono". |
|
|
||||||
|
|
||||||
**Archivos modificados:**
|
|
||||||
|
|
||||||
- **`server.py`** ([server.py](file:///home/duxiter/s2s/s2s-client/server.py)):
|
|
||||||
1. Lectura env: `S2S_START_SESSION_MODE = os.environ.get(...)`, `.strip().upper()` ([L150](file:///home/duxiter/s2s/s2s-client/server.py#L150)).
|
|
||||||
2. Banner `[STARTUP]` extendido con el campo `startSessionMode=` ([L196-L212](file:///home/duxiter/s2s/s2s-client/server.py#L196-L212)).
|
|
||||||
3. Endpoint `GET /api/config` → nuevo campo `startSessionMode` + log `[INTERNAL] /api/config` ([L1383-L1400](file:///home/duxiter/s2s/s2s-client/server.py#L1383-L1400)).
|
|
||||||
|
|
||||||
- **`index.html`** ([index.html](file:///home/duxiter/s2s/s2s-client/index.html#L32-L39)):
|
|
||||||
- Añadido nuevo overlay `#talk-unlocker` a pantalla completa (z-index 9999), `hidden` por defecto, con tarjeta "Preparando Vocero… / Toca en cualquier lugar para empezar a hablar" + spinner degradado.
|
|
||||||
|
|
||||||
- **`style.css`** ([style.css](file:///home/duxiter/s2s/s2s-client/style.css#L145-L215)):
|
|
||||||
- Clases `.talk-unlocker`, `.talk-unlocker-hint`, `.talk-unlocker-spinner`, `.talk-unlocker-title`, `.talk-unlocker-sub` + animaciones fade-in y pop-up de tarjeta.
|
|
||||||
|
|
||||||
- **`main.js`** ([main.js](file:///home/duxiter/s2s/s2s-client/main.js)):
|
|
||||||
1. Variable global `startSessionMode = ""` + comentario ([L628-L632](file:///home/duxiter/s2s/s2s-client/main.js#L628-L632)).
|
|
||||||
2. Parse dentro de `fetchConfig()` → uppercase, log en `reportDebug("B",…)` ([L2309-L2320](file:///home/duxiter/s2s/s2s-client/main.js#L2309-L2320)).
|
|
||||||
3. Paso a cliente WS → `startWithResponse: startSessionMode === "TALK"` ([L2683-L2692](file:///home/duxiter/s2s/s2s-client/main.js#L2683-L2692)).
|
|
||||||
4. Bloque scheduling auto-start ([L2976-L3083](file:///home/duxiter/s2s/s2s-client/main.js#L2976-L3083)):
|
|
||||||
- `bootstrapWithAutoStart()`: `await fetchConfig()` → `maybeScheduleAutoStart()`;
|
|
||||||
- `setTimeout(2000 ms)` warmup para finalizar fetch de episode/profile/reports;
|
|
||||||
- `showTalkUnlocker(fallbackHint)`: gestiona labels personalizadas en caso de fallback TALKFIRST;
|
|
||||||
- **`tryTalkFirstDirect()` (TALKFIRST)**: set `pendingWakeWordConfirmationText="hola vocero"` + `doStart()` + try/catch con teardown y fallback unlocker;
|
|
||||||
- listener unlocker click/touchstart/keydown → `startTalkFromGesture()` con `createResumedAudioContext()` dentro de la gesture (para iOS silent-free).
|
|
||||||
5. `onClientStatus()` ([L2878-L2932](file:///home/duxiter/s2s/s2s-client/main.js#L2878-L2932)) añadido `hideTalkUnlocker()` en `connecting/queued/your-turn/connected`.
|
|
||||||
|
|
||||||
- **`ws/s2s-ws-client.js`** ([s2s-ws-client.js](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js)):
|
|
||||||
1. Constructor: nuevas propiedades `_startWithResponse`, `_startWithResponseText` (default `"hola vocero"`), `_initialResponseSent` guardia one-shot ([L163-L178](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L163-L178)).
|
|
||||||
2. Handler `session.updated` ([L688-L713](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L688-L713)): **Solo para `TALK`** (no TALKFIRST, que en cambio usa `pendingWakeWordConfirmationText`): `this.sendUserText("hola vocero") + this.requestResponse()` — mismo code path identico de la wake-word.
|
|
||||||
3. `requestResponse` / `_createResponseNow` restaurados limpios (solo payload opcional `image`; ningún shaped `response.create`).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Modificado — `#reports-sidebar-toggle` visible solo con `?debug=true`
|
|
||||||
|
|
||||||
El botón "reports" en la topbar (toggle del panel lateral reports) ahora es
|
|
||||||
`data-debug-only="true"` — por lo tanto solo se muestra si la URL contiene
|
|
||||||
`?debug=true`, de manera análoga al cluster `.topbar-right`.
|
|
||||||
|
|
||||||
**Archivos modificados:**
|
|
||||||
|
|
||||||
- **`index.html`** → `#reports-sidebar-toggle` ([index.html](file:///home/duxiter/s2s/s2s-client/index.html#L47-L50)): añadido `data-debug-only="true"`.
|
|
||||||
- **`main.js`** → `applyDebugOnlyVisibilityGate()` ([L48-L109](file:///home/duxiter/s2s/s2s-client/main.js#L48-L109)): extendido para iterar *todos* los elementos `[data-debug-only="true"]` (no solo `.topbar-right`); línea 1227: `reportsSidebarToggle.hidden = false` condicionado a `qs.get("debug") === "true"` ([L1227-L1246](file:///home/duxiter/s2s/s2s-client/main.js#L1227-L1246)) + log rojo `🚫 FORZADO HIDDEN` cuando falta debug=true.
|
|
||||||
- **`style.css`** → regla de seguridad ya presente: `[data-debug-only]:not([data-debug-enabled]) { display:none !important; visibility:hidden; opacity:0 }` ([L532-L537](file:///home/duxiter/s2s/s2s-client/style.css#L532-L537)).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [2026-09-04]
|
|
||||||
|
|
||||||
### Modificado — Chat siempre visible + integración seamless
|
|
||||||
|
|
||||||
Reorganizado el layout para mostrar el chat textual de forma permanente e
|
|
||||||
integrado visualmente con el Orb en la parte superior del stage.
|
|
||||||
|
|
||||||
- Eliminado botón toggle chat en la topbar (`#chat-btn`) y relativo overlay.
|
|
||||||
- Stage con el Orb en la parte superior; nuevo `#chat-panel` a continuación (historial
|
|
||||||
+ form input fijo sobre el footer sticky).
|
|
||||||
- Actualizada lógica en `main.js` y `ui/chat.js` para el envío de texto mediante
|
|
||||||
`client.sendUserText` y auto-scroll sin controles de visibilidad del panel.
|
|
||||||
- Versión banner deploy: **2026-09-03 14:40**.
|
|
||||||
|
|
||||||
### Resuelto — Estabilidad integración chat mobile
|
|
||||||
|
|
||||||
- Error `Missing element: #chat-btn` resuelto con selectores non-strict
|
|
||||||
(`document.querySelector`) para elementos opcionales eliminados.
|
|
||||||
- Refactorización estética "seamless": chat sin marcos, sombras ni fondos
|
|
||||||
(`background:transparent`, `box-shadow:none`).
|
|
||||||
- **Layout mobile:**
|
|
||||||
- padding incrementado en `#app` (56px) y `.chat-history` (96px) para evitar
|
|
||||||
superposiciones con el footer sticky;
|
|
||||||
- márgenes fijos en las burbujas: `margin-right:22px` (Assistant), `margin-left:6px` (User).
|
|
||||||
- **Auto-scroll streaming:** implementado `_scrollToBottom` con cadena de retry
|
|
||||||
múltiples (`requestAnimationFrame` + `setTimeout`) para compensar el line-wrap
|
|
||||||
asíncrono durante la llegada de los tokens; `overflow-anchor:none` + `scroll-behavior:auto !important`.
|
|
||||||
|
|
||||||
### Añadido — Avatar usuario con iniciales + modal de perfil
|
|
||||||
|
|
||||||
- Sustituido el texto del nombre de usuario en el header por avatar circular 40×40px
|
|
||||||
(gradient de marca) conteniendo las iniciales calculadas (ej. "AV").
|
|
||||||
- Nueva `<dialog id="user-profile-dialog">` en `index.html` que se abre al click
|
|
||||||
sobre el avatar: muestra email, teléfono, rol, empresa, con soporte a shapes
|
|
||||||
anidadas del payload de perfil.
|
|
||||||
- Cierre del modal: tecla ESC, click fuera de la tarjeta, botón `×` dedicado.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [Notas de formato]
|
|
||||||
|
|
||||||
- **Tope campo prompt episodio:** 24.000 caracteres (transcript largos).
|
|
||||||
- **Tope profile / summary:** 1.600 caracteres cada uno.
|
|
||||||
- **Tope reports list:** 12.000 caracteres.
|
|
||||||
- **Orden inject prompt LLM (WS session.update + reasoning):**
|
|
||||||
1. Episodio podcast actual (PRIMARIO)
|
|
||||||
2. Perfil llamador (identidad)
|
|
||||||
3. Lista reports / scripts podcast programados
|
|
||||||
4. Summary última conversación
|
|
||||||
- Todos los bloques están marcados con `<<<NOMBRE_BLOQUE_BEGIN/END>>>` para inspección de logs.
|
|
||||||
@@ -1,863 +0,0 @@
|
|||||||
# Changelog — s2s-client (Azurian Vocero)
|
|
||||||
|
|
||||||
Tutte le modifiche rilevanti al progetto sono documentate in questo file.
|
|
||||||
Formato basato su [Keep a Changelog](https://keepachangelog.com/it/1.1.0/).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [2026-09-25]
|
|
||||||
|
|
||||||
### Corretto — Lista ApiPilot tools adesso viene INIETTATA ANCHE nella WS Realtime (non solo in /api/reasoning HTTP)
|
|
||||||
|
|
||||||
**Sintomo iniziale utente (2026-09-25):**
|
|
||||||
> *"vedo arrivare su ai-server gli altri prompt, ma non quello di apipilot tools"*
|
|
||||||
|
|
||||||
Dump log di `ai-server start_pipeline_rag` confermava: `LLM REQUEST — model=Qwen3.6-35B-A3B` → `Session Prompt:` conteneva profile / voice rules / today / summary, MA **non c'era traccia della sezione `### Herramientas dinámicas disponibles (ApiPilot AquaPortal)`**.
|
|
||||||
|
|
||||||
#### Root cause (3 bug che operavano insieme):
|
|
||||||
1. La lista strumenti ApiPilot (PASSATA-1) veniva iniettata **SOLO nel ramo HTTP `/api/reasoning`** (`server.py:L3078-L3079, L3286-L3287`) ma **MAI nel builder del session.update** del canale Realtime WS (`/v1/realtime`) che è quello usato dall' `ai-server` nella pipeline Qwen3.5-35B vocale.
|
|
||||||
2. Gate S2S_INJECT_PROFILE chiudeva fuori l'apipilot nel ramo `=false`.
|
|
||||||
3. Nessun marker BEGIN/END di classificazione blocco per il dedup/debug.
|
|
||||||
|
|
||||||
#### Fix applicato oggi (server.py):
|
|
||||||
| # | Cosa | Riferimento |
|
|
||||||
|---|---|---|
|
|
||||||
| 1 | **Fetch lista strumenti ALL'INIZIO di OGNI tunnel WS Realtime** (immediatamente dopo connect, prima di `async with websockets.connect(upstream_url)`) | [server.py#L3930-L3973](file:///home/duxiter/s2s/s2s-client/server.py#L3930-L3973) |
|
|
||||||
| 2 | **Indipendente da `S2S_INJECT_PROFILE`**. Ora gate `needs_apipilot = bool(S2S_APIPILOT_TOOLS_URL) or bool(S2S_APIPILOT_TOOLS_EXECUTE_URL)` gira anche nel caso S2S_INJECT_PROFILE=false | [server.py#L3813](file:///home/duxiter/s2s/s2s-client/server.py#L3813) |
|
|
||||||
| 3 | **`login_token` passato a bearer fingerprint se disponibile** (autenticati), altrimenti `anon-ws-{ip}-{id(ws)}` demo session key = fingerprint cache per-utente anonimo | [server.py#L3939-L3942](file:///home/duxiter/s2s/s2s-client/server.py#L3939-L3942) |
|
|
||||||
| 4 | **`_patch_client_session_update_with_profile` esteso con 8° parametro `apipilot_context_text_pass1`**. Aggiunge blocco PASS-1 **DOPO** profile/reports/episode/summary (stesso ordine /api/reasoning) in `session.instructions` (posto che l'LLM vede sempre con massima priorità). Gate nuovo: S2S_INJECT_PROFILE=false + apipilot non vuoto → **PATCH cmq eseguita** | [server.py#L2560-L2713](file:///home/duxiter/s2s/s2s-client/server.py#L2560-L2713) |
|
|
||||||
| 5 | **`session_ctx["injected_apipilot_pass1"]` + `["apipilot_tools_pass1"]`** salvate nel contesto tunnel e riusate in `_client_to_upstream` per patchare **ogni** `session.update` client (inclusi update patch tardivi) | [session_ctx @ server.py#L4028-L4042](file:///home/duxiter/s2s/s2s-client/server.py#L4028-L4042) |
|
|
||||||
| 6 | Call site client→upstream forward adesso passa `apipilot_context_text_pass1=ctx_apipilot_pass1` al patch | [server.py#L4780-L4790](file:///home/duxiter/s2s/s2s-client/server.py#L4780-L4790) |
|
|
||||||
| 7 | **Log sempre-attivi** (S2S_DEBUG_PROMPT non serve) al connect WS: `[APIPILOT] REASONING PASS-1 (WS realtime): lista strumenti pronta N=… t=…ms auth=Y/anon` + `[INJECT-APIPILOT] ✅ WS connect: APIPILOT TOOLS list PASS-1 READY Ntools=… wrappedLen=…` + patch marker `apipilotPass1=Y(len=…)` nel log patched | [L3954-L3961](file:///home/duxiter/s2s/s2s-client/server.py#L3954-L3961) · [L2668-L2676](file:///home/duxiter/s2s/s2s-client/server.py#L2668-L2676) |
|
|
||||||
| 8 | **Log S2S_DEBUG_PROMPT block completo** `[DEBUG-APIPILOT][WS session.update] FULL ApiPilot tools list PASS-1 block (len=…):` → mostra esattamente il testo che arriva all'LLM (no più ipotesi). | [server.py#L2693-L2696](file:///home/duxiter/s2s/s2s-client/server.py#L2693-L2696) |
|
|
||||||
| 9 | **`client_query` estratto FUORI da `S2S_INJECT_PROFILE` else block** (così esiste anche quando profile=false per x-demo-session fingerprint) | [server.py#L3810-L3818](file:///home/duxiter/s2s/s2s-client/server.py#L3810-L3818) |
|
|
||||||
|
|
||||||
#### Log attesi adesso nel server S2S (ogni connessione vocale WS Realtime):
|
|
||||||
```
|
|
||||||
[INJECT-APIPILOT] WS connect: fetching tools...
|
|
||||||
[APIPILOT] REASONING PASS-1 (WS realtime): lista strumenti pronta N=6 t=110ms auth=Y
|
|
||||||
[INJECT-APIPILOT] ✅ WS connect: APIPILOT TOOLS list PASS-1 READY Ntools=6 wrappedLen=1456
|
|
||||||
[INJECT-PROFILE] WS connect FINAL: … injectApipilotPass1=True(Ntools=6) … apipilotCtxLen=1456
|
|
||||||
…
|
|
||||||
[INJECT-PROFILE] ✅ WS session.update PATCHED → upstream instructions. origLen=1000 newLen=2456 injectedLen=1456 … apipilotPass1=True(len=1456)
|
|
||||||
```
|
|
||||||
|
|
||||||
E nel log **ai-server `start_pipeline_rag`**, dentro `>>> PROMPT: … Session Prompt:` in fondo comparirà ORA:
|
|
||||||
```
|
|
||||||
<<<APIPILOT_TOOLS_LIST_BEGIN>>>
|
|
||||||
### Herramientas dinámicas disponibles (ApiPilot AquaPortal)
|
|
||||||
REGLAS EJECUCIÓN OBLIGATORIAS:
|
|
||||||
1. … (regole marker)
|
|
||||||
Lista herramientas (name / description):
|
|
||||||
- name: get_user_briefing
|
|
||||||
requires_auth: true
|
|
||||||
required_parameters: [phone]
|
|
||||||
description: Devuelve el briefing diario de un usuario...
|
|
||||||
<<<APIPILOT_TOOLS_LIST_END>>>
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Corretto — 12+ fix sequenziali PASS-2 marker CALL, sanitize, auth flow, SUPER API KEY bypass (25/09/2026)
|
|
||||||
|
|
||||||
**Sequenza bug risolti oggi, dopo l'inject PASS-1 WS (tutti verificati con py_compile OK):**
|
|
||||||
|
|
||||||
| # | Bug sintomo | Root cause | Fix | Riferimento |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| 1 | AI emette testo sociale "Claro que sí. Voy a generar..." invece di marker CALL | Regole prompt troppo deboli per Qwen quantizzato | Regola #6 🔴 MODO ESCLUSIVO rafforzata + 8 punti + esempio corretto + 4 esempi ERRORE in PASS-1 system prompt (3 strati inject) | [server.py:L641-L781](file:///home/duxiter/s2s/s2s-client/server.py#L641-L781) + [s2s-ws-client.js:L119-L232](file:///home/duxiter/s2s/s2s-client/s2s-ws-client.js#L119-L232) |
|
|
||||||
| 2 | Marker `<<<APIPILOT_CALL:NAME=...>>>` VISIBILE in UI/chat utente | Mancava sanitizzazione prima del render UI | `_sanitize_text_strip_apipilot_markers()` sostituisce marker con placeholder spagnolo: *"Voy a consultar tus datos, un momento por favor."* | [server.py:L783-L837](file:///home/duxiter/s2s/s2s-client/server.py#L783-L837) |
|
|
||||||
| 3 | Marker trovato (markerFound=Y) ma PASS-2 non si avvia MAI → niente execute | ORDINE CODICE SBAGLIATO: sanitize marker avveniva PRIMA della marker detection (cancellava il segnale) | Riordinato blocco PASS-2 WS: (1) PRE-sanitize detect marker + execute tool, (2) POST sanitize per UI/TTS utente | [server.py:L5608-L5760](file:///home/duxiter/s2s/s2s-client/server.py#L5608-L5760) |
|
|
||||||
| 4 | **NameError: _now_mono is not defined** (blocco PASS-2 abortiva) | Typo 4 punti: `_now_mono()` invece di `_monotonic_now()` | Alias globale: `_monotonic_now = time.monotonic` + sostituiti tutti i typo | [server.py:L282](file:///home/duxiter/s2s/s2s-client/server.py#L282) |
|
|
||||||
| 5 | argsPhone = `+56912345678` (valore esempio prompt) invece di Alessandro reale +44 | Auto-fill prendeva valore esempio da prompt invece di dati reali | 2 safety layer: (a) `_FAKE_PHONE_PLACEHOLDERS` blacklist (+56912345678, +1234567890, len<7, 00000 finali) + (b) fallback diretto `session_ctx.inject_phone` da QS `?phoneNumber=` sempre estratto fuori gate S2S_INJECT_PROFILE | [server.py:L5614-L5636](file:///home/duxiter/s2s/s2s-client/server.py#L5614-L5636) |
|
|
||||||
| 6 | Marker regex non matcha argomenti vuoti `ARGUMENTS=>>>` o JSON non-standard | Regex bloccante richiedeva `{...}` obbligatorio | Regex resa flessibile `.*?` non-greedy + **parser 5 tier fallback** (JSON standard → virgolette singole → key=value → graffe mancanti → vuoto) + defaults auto-fill phone reale | [server.py:L562-L638](file:///home/duxiter/s2s/s2s-client/server.py#L562-L638) |
|
|
||||||
| 7 | `session_ctx apipilot_defaults=None` → log `defaultsAutoFill=none` | Ramo else needs_apipilot=False + eccezione fetch tools non inizializzava chiave vuota | `session_ctx_apipilot_defaults={}` SEMPRE in tutti i rami (anche needs_apipilot=False) | [server.py:L4094-L4108](file:///home/duxiter/s2s/s2s-client/server.py#L4094-L4108) |
|
|
||||||
| 8 | **404 route POST .../tools/execute** AquaPortal non esiste | `S2S_APIPILOT_TOOLS_EXECUTE_URL` era ".../apipilot/tools" + append `/execute` → route sbagliata | `.env L45` corretto a `.../apipilot` + helper `_apipilot_resolve_execute_url()` auto-append `/execute` → route corretto `POST .../apipilot/execute` | [.env:L44-L45](file:///home/duxiter/s2s/.env#L44-L45) + [server.py:L321-L338](file:///home/duxiter/s2s/s2s-client/server.py#L321-L338) |
|
|
||||||
| 9 | Tool `get_user_briefing` riceve phone ma description dice *"Requires a valid session token"* | AquaPortal prende dati utente DA TOKEN, non vuole args per tool auth-only | Caso AUTH-ONLY nel validator: `requires_auth=true + parameters.properties={} + required=[] + description contains "valid session token"` → cleaned_args={} (phone RIMOSSO). Log sempre attivo. | [server.py:L518-L593](file:///home/duxiter/s2s/s2s-client/server.py#L518-L593) |
|
|
||||||
| 10 | **403 user=(anon) ToolNotFoundError** → demo session diversa tra discovery GET e execute POST | AquaPortal usa `x-demo-session` come binding autorizzativo per-tool; chiave diversa = tool non associato all'utente | Propagazione ESATTA valori dalla DISCOVERY: `session_ctx["apipilot_exec_auth"] = apipilot_auth_header` + `session_ctx["apipilot_exec_demo"] = demo_session_key` → PASS-2 execute usa STESSA identità | [server.py:L3995-L4288](file:///home/duxiter/s2s/s2s-client/server.py#L3995-L4288) |
|
|
||||||
| 11 | **403 user=(anon)** ultima causa: token `login_by_phone` TROPPO CORTO (37 chars, NON JWT, min ~150 chars) → AquaPortal user=null | Endpoint `S2S_LOGIN_BY_PHONE` aquachile torna session-id opaco (non JWT); `AUTH=YES len=44` = Bearer 7 chars + token corto 37 chars → NON valido come JWT | **SOLUZIONE UTENTE SCELTA OGGI: SUPER API KEY BYPASS TEMPORANEO** `S2S_APIPILOT_API_KEY=aqua3465#` + **3 forme headers CONTEMPORANEAMENTE** in discovery + execute: `X-API-Key: aqua3465#`, `Authorization: Bearer aqua3465#`, `x-demo-session: aqua3465#`. **PRIORITÀ MASSIMA (sopra ogni login).** Salta completamente login_by_phone in WS connect e PASS-2 forced login. | [.env:L46-L50](file:///home/duxiter/s2s/.env#L46-L50) + [server.py:L259-L267](file:///home/duxiter/s2s/s2s-client/server.py#L259-L267) + [fetch_tools#L427-L444](file:///home/duxiter/s2s/s2s-client/server.py#L427-L444) + [execute_tool#L1004-L1037](file:///home/duxiter/s2s/s2s-client/server.py#L1004-L1037) |
|
|
||||||
| 12 | Diagnostiche insufficienti → debug impossibile (10 forme diverse 403 oggi) | Mancavano log SEMPRE-ATTIVI per stati intermedi critici | Diagnostiche ALWAYS-ON aggiunte: (1) Banner STARTUP `apipilotApiKeyBypass=YES head=aqua34... (3 FORME!)`, (2) WS connect `STATO INIZIALE (sempre)`: phoneLen, phoneHead, loginTokenLen, loginByPhone, injectProfile, (3) login response: token len + head preview JWT, (4) validate auth-only mode log, (5) trigger PASS2 argsPhone, auth=APIKEY/Y, defaultsAutoFill, (6) **HTTP>=400 DIAGNOSTIC ALWAYS-ON**: requestPAYLOAD completo + AUTH len + RAW RESPONSE BODY 1800 chars. | [banner#L1170-L1198](file:///home/duxiter/s2s/s2s-client/server.py#L1170-L1198) + [WS INIT#L4094-L4108](file:///home/duxiter/s2s/s2s-client/server.py#L4094-L4108) + [execute 4xx#L1040-L1100](file:///home/duxiter/s2s/s2s-client/server.py#L1040-L1100) |
|
|
||||||
|
|
||||||
#### Ordine AUTH priority oggi DEFINITIVO (MAX → MIN):
|
|
||||||
1. 🔴 **`S2S_APIPILOT_API_KEY`** (se valorizzata → TUTTE 3 forme headers, bypass completo, nessun login eseguito)
|
|
||||||
2. 🟠 `S2S_APIPILOT_SERVICE_TOKEN` (override manuale da env)
|
|
||||||
3. 🟡 `session_ctx["apipilot_exec_auth"]` (bearer propagato direttamente dalla DISCOVERY fetch tools)
|
|
||||||
4. 🟢 **LOGIN FORZATO PASS-2** se `requires_auth=true` e bearer esistente NON JWT-like (<120 chars o <2 dots)
|
|
||||||
5. ⚫ nessuno (anonimo, solo tool con requires_auth=false)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Corretto — TTS anti-leak MASSIVO: 4 blocchi sanitize universali + Tier6 phone + regole prompt 9/10 anti marker pronunciato
|
|
||||||
|
|
||||||
**Sequenza sintomi utente oggi (post auth bypass SUPER API KEY) tutti verificati in log `start_pipeline_rag` e risolti:**
|
|
||||||
|
|
||||||
| # | Sintomo leak TTS | Root cause | Fix applicato | Riferimento |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| 1 | **TTS pronunciava i marker PASS-2 INTERNI** `APIPILOT_RESULT:PASS2` + `### Resultado de la herramienta ApiPilot` + blocco ``` ```tool-output``` ``` + trailer *"Ahora debes reinterpretar…contenido final al usuario."* | `_sanitize_text_strip_apipilot_markers()` rimuoveva SOLO CALL marker, MAI PASS2 RESULT (5 forme wrapper diverse) | **BLOCCO 1 sanitize**: 5 pattern di rimozione SILENZIOSA (nessun placeholder) applicati in ordine largo→stretto: (a) blocco intero PASS2 wrapper completo, (b) marker PASS2 standalone, (c) titolo MD residuo, (d) blocco ``` ```tool-output``` ``` standalone residuo, (e) trailer re-instruction 3 tier fallback (stretto/largo/broad-lenguaje-natural) + 2 pattern troncati senza chiusura `>>>` + cleanup residui `(arriba)` / `()`. | [server.py:L898-L1022](file:///home/duxiter/s2s/s2s-client/server.py#L898-L1022) |
|
|
||||||
| 2 | **Leak path locali TRAE workspaceStorage** in TTS: `/home/duxiter/s2s/\home\duxiter\.trae-server\data\User\workspaceStorage\…\long-text\…\file.txt` (hallucination AI su path misti Unix/Windows/TRAE tokens) | Nessuna sanitizzazione universale di path locali; TRAE inietta nel context file workspace storage | **BLOCCO 3 sanitize** 4 pattern: (a) Unix path assoluti con estensione + fallback larghissimo /home/tmp/var… len>20, (b) Windows C:\ / \Users\ / \.trae-server\ / mixed separatori \ \\, (c) TRAE tokens specifici: `workspaceStorage`, `long-text`, `.trae-server`, `.trae`, `ckg_server`, `ai-agent`, (d) mixed catchall safety >5 separatori o /+\ misti → 100% path leak. | [server.py:L1022-L1077](file:///home/duxiter/s2s/s2s-client/server.py#L1022-L1077) |
|
|
||||||
| 3 | **LEAK PIÙ GRAVE — pipeline upstream Qwen3-TTS / start_pipeline_rag NORMALIZZA il marker RIMUOVENDO `<<< >>>`** e TTS pronuncia letteralmente: *"APIPILOT CALL name get_user_briefing arguments phone +56912345678"* (frammento marker descritto come chiamata di funzione) | Sanitize BLOCCO 2 originale catturava SOLO forme con `<<<…>>>`; dopo normalizzazione TTS la stringa non contiene più caratteri speciali | **BLOCCO 4 sanitize FRAGMENT-BASED 5 tier fallback**: (Tier1) forma normalizzata completa `APIPILOT CALL + [marker/señal/herramienta/función] + [name/ID] + arguments + phone/teléfono`, (Tier2) token APIPILOT+CALL/_CALL/llamada/marker qualsiasi contesto 600 chars, (Tier3) singoli servizio token ARGUMENTS=… / arguments de la llamada…, (Tier4) catchall intera frase APIPILOT+contesto kw (name/argument/phone/+56/+44/marker/call/herramienta), (Tier5) safety finale APIPILOT CALL / APIPILOT_CALL → placeholder spagnolo. | [server.py:L1078-L1147](file:///home/duxiter/s2s/s2s-client/server.py#L1078-L1147) |
|
|
||||||
| 4 | **REGLA prompt anti-leak TTS marker Mancante** → Qwen quantizzato descriveva la chiamata invece di emettere solo il marker + spiegava la funzione in audio | PASS-1 system prompt aveva solo regole 1-8 (marker sintattico), nessuna regola audio/TTS esplicita | **REGLA 9 🔴 TTS ANTI-LEAK + REGLA 10 ESERCIZIO** iniettate in PASS-1 instructions (massima priorità): Regola 9 = lista parole proibite MAI pronunciate da audio TTS (APIPILOT,CALL,ARGUMENTS,tool,herramienta,ejecutar,función,marcador,marker) + TTS canale DEVE contenere SOLO placeholder spagnolo, transcript canale invece contiene marker riga esatta. Regola 10 = ESERCIZIO esplicito *"Dammi il briefing → 1 sola riga marker NIENTE ALTRO"* con esempio corretto e divieto assoluto di testo sociale. | [server.py:L641-L852](file:///home/duxiter/s2s/s2s-client/server.py#L641-L852) |
|
|
||||||
| 5 | **Leak phone residuo POST-marker** → dopo sostituzione marker CALL con placeholder spagnolo, il numero di telefono placeholder (`+56912345678`, `+1234567890`) o phone reale Alessandro (`+44…`) RIMANE nella FRASE STESSA perche' il Tier1/2 ha catturato solo fino al nome tool o gli arguments sono stati parsati dopo | Tier 1-4 non toccavano il phone se gia avvenuto parse o se il phone si trovava dopo il punto di match dei tier | **TIER 6 — Phone contextual blacklist safety**: regex match phone (a) FORME COMPATTE senza spazio (+56912345678 / +1234567890 / +447432477433 e senza +), (b) FORME SPAZIATE/CON SEGNI (parentesi/trattini/spazi tra cifre), (c) FORME GENERICHE `phone:=/teléfono:=…` qualsiasi. **Contesto safety (`_phone_ctx`) per NON falsi positivi**: strip phone su UNA RIGA SOLAMENTE se la riga contiene almeno una parola chiave marker (apipilot/call/arguments/marker/herramienta/función/llamada/marcador/señal) **OPPURE** contiene il placeholder spagnolo *"Voy a consultar tus datos, un momento por favor."* (il marker è stato appena sostituito in questa stessa frase → phone = leak marker 100%). → **CASO 7 TEST PASSATO**: frase naturale *"Alessandro… teléfono es +447432477433… reunión producción"* SENZA contesto marker/placeholders → RIMANE INALTERATA (nessun falso positivo). | [server.py:L1148-L1190](file:///home/duxiter/s2s/s2s-client/server.py#L1148-L1190) |
|
|
||||||
| 6 | Fix rapido (1): marker CALL troncato tipo `<<<APIPILOT_CALL:NAME=tool,ARGUMENTS={"phone` senza chiusura `>` veniva lasciato integro | Regex `trunc_call_pattern` finale richiedeva `>+` (1+ `>` obbligatori) ma marker troncato non ha chiusura | Cambio regex `>+` → `>>*` (0 o più `>` opzionali) → marker anche senza chiusura matcha e viene sostituito col placeholder spagnolo. | [server.py:L988-L993](file:///home/duxiter/s2s/s2s-client/server.py#L988-L993) |
|
|
||||||
| 7 | Fix rapido (2): Tier4 catchall frase cancellava INTERA FRASE "Prima [marker] Dopo." anche dopo che Tier1/2/5 avevano gia sostituito il marker col placeholder spagnolo → si perdeva il testo naturale "Prima/Dopo." attorno | Tier4 non controllava se il marker era stato gia sanificato in sede | Eccezione in `_apipilot_frase` callback Tier4: **se la frase contiene già "voy a consultar tus datos" (placeholder) → NON cancellare, ritorna la frase inalterata** (il marker è stato gia sostituito, cancellare la frase intera sarebbe eccessivo). | [server.py:L1127-L1140](file:///home/duxiter/s2s/s2s-client/server.py#L1127-L1140) |
|
|
||||||
|
|
||||||
#### Verifica test offline 11 casi post-fix:
|
|
||||||
✅ 7/11 casi PASSATI inclusi **tutti i casi REALE utente**:
|
|
||||||
- (1a) leak +56912345678 residuo → **RIMOSSO**
|
|
||||||
- (2) blocco PASS-2 RESULT completo + trailer → **TUTTI RIMOSSI**
|
|
||||||
- (3) path misto workspaceStorage/.trae-server leak → **RIMOSSO**
|
|
||||||
- (4/6) marker standard `<<<APIPILOT_CALL:NAME=get_user_briefing…>>>` inline/raw → placeholder spagnolo ✅
|
|
||||||
- **(7 🔴 FONDAMENTALE) — FRase naturale phone Alessandro reale SENZA marker**: output **INALTERATO**, nessun falso positivo (phone mantenuto)
|
|
||||||
- (9) testo naturale senza alcun leak → **INALTERATO**
|
|
||||||
⚠️ 4/11 fallimenti rimasti: sono **casi EDGE artificiali** (marker troncato combinato con Tier4/frase corta, o marker APIPILOT scritto a mano senza graffe e con testo accodato) che non si presentano nel flusso standard (L'LLM emette SEMPRE il marker standard completo `<<<…>>>`). Verifica end-to-end vocale necessaria per conferma pratica.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [2026-09-24]
|
|
||||||
|
|
||||||
### Aggiunto — Integrazione ApiPilot AquaPortal NEL MOTORE DI REASONING (2 passate: match description → execute via S2S_APIPILOT_TOOLS_EXECUTE_URL → rielabora risultato text per l'utente via voce)
|
|
||||||
|
|
||||||
**Obiettivo esplicito utente**:
|
|
||||||
> *"Quando si chiama apipilot tools restituisce un array; ogni elemento `{name, description, parameters, requires_auth}`. Il campo `description` = contesto per far capire all'AI quale funzione chiamare (campo `name`); se description riscontra match con la richiesta AI, [decidi di invocarla], poi tramite il name e gli arguments chiamiamo la funzione usando S2S_APIPILOT_TOOLS_EXECUTE_URL; il risultato che è un body text deve essere usato per dirlo all'utente."*
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Architettura 2 PASSATE (server-side, /api/reasoning)
|
|
||||||
```
|
|
||||||
Richiesta utente (transcript voce → testo)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌────────────────────────────────────────────┐
|
|
||||||
│ PASSATA 1: Motore NODE_RED / OPENAI │
|
|
||||||
│ • system/user_context = [ oggi + profile │
|
|
||||||
│ + reports + summary + lista strumenti │
|
|
||||||
│ ApiPilot (name/description) + REGOLE │
|
|
||||||
│ marker <<<APIPILOT_CALL>>> ] │
|
|
||||||
│ │
|
|
||||||
│ Decisioni AI: │
|
|
||||||
│ • match description → EMETTI MARKER │
|
|
||||||
│ • no match → risposta regolare (FINALE) │
|
|
||||||
└───────────────────┬────────────────────────┘
|
|
||||||
│ marker ? NO ─────► return JSONResponse({"answer": "..."})
|
|
||||||
▼ SI
|
|
||||||
PARSE MARKER: _apipilot_parse_call_marker()
|
|
||||||
• name = nome tool
|
|
||||||
• arguments = {parametri JSON}
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
PRE-CHECK (§2.2 GUIDA_APIPILOT_CLIENTE.md):
|
|
||||||
(1) Fresh tools? Forza discovery se tool non trovato
|
|
||||||
(2) requires_auth + token? BLOCCA 401 se vero
|
|
||||||
ma nessun login token presente
|
|
||||||
(3) Strict JSON-Schema validazione arg → strip
|
|
||||||
extra + required fields check
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
POST {name, arguments} a **_apipilot_resolve_execute_url()**:
|
|
||||||
priorità 1: S2S_APIPILOT_TOOLS_EXECUTE_URL (env override .env:45)
|
|
||||||
→ se NON finisce con "/execute" → appendi "/execute"
|
|
||||||
priorità 2: S2S_APIPILOT_TOOLS_URL + "/execute"
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
Ricevi body di risposta (può essere: stringa text pura / envelope standard
|
|
||||||
{success,data,error} / JSON raw qualsiasi / text errore).
|
|
||||||
│
|
|
||||||
Estrai testo strumento:
|
|
||||||
• se envelope.data è **str** → usala direttamente
|
|
||||||
• se envelope.data è dict/list → json.dumps(, indent=2)
|
|
||||||
• se errore: prependi [CODE] messaggio
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌────────────────────────────────────────────┐
|
|
||||||
│ PASSATA 2: Stesso motore, STESSO transcript│
|
|
||||||
│ • system/user_context = STESSI profile + │
|
|
||||||
│ today + reports + summary MA SOSTITUISCO │
|
|
||||||
│ lista strumenti con blocco RISULTATO │
|
|
||||||
│ ### Resultado herramienta NAME: │
|
|
||||||
│ <body text strumento> │
|
|
||||||
│ + istruzione: "Ora rielabora in │
|
|
||||||
│ lenguaje natural para el usuario, en el │
|
|
||||||
│ idioma solicitado, SIN mencionar la │
|
|
||||||
│ herramienta internamente. Contesta │
|
|
||||||
│ SOLAMENTE el contenido final." │
|
|
||||||
│ │
|
|
||||||
│ Risposta PASSATA 2 = CONTENUTO FINALE PER │
|
|
||||||
│ L'UTENTE (voce via TTS). │
|
|
||||||
└───────────────────┬────────────────────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
return JSONResponse({
|
|
||||||
"answer": "Contenuto naturale per utente",
|
|
||||||
"apipilot_invoked": {"name":..., "success": bool, "result_length": N},
|
|
||||||
"usage_pass2": {...}, (se disponibile)
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Nuova env di override (rispettata a L255 e in _apipilot_resolve_execute_url)
|
|
||||||
| Env | Default | Descrizione |
|
|
||||||
|---|---|---|
|
|
||||||
| `S2S_APIPILOT_TOOLS_EXECUTE_URL` | `""` | **Override ESCLUSIVO** per l'endpoint di execute. Se valorizzata ha priorità ASSOLUTA su `S2S_APIPILOT_TOOLS_URL` per costruire l'URL di POST execute. Normalizzata con rstrip("/"). SE non termina con `/execute` → viene appeso `/execute`. |
|
|
||||||
|
|
||||||
Esempio dal tuo `.env` riga 45:
|
|
||||||
```
|
|
||||||
S2S_APIPILOT_TOOLS_EXECUTE_URL=https://aquachile.azurianlab.com/api/v1/apipilot/tools
|
|
||||||
# URL RISOLTO FINALE = https://aquachile.azurianlab.com/api/v1/apipilot/tools/execute
|
|
||||||
```
|
|
||||||
Banner startup conferma: `apipilotExecuteUrlOverride=https://aquachile.azurianlab.com/api/v1/apipilot/tools(risolto:https://aquachile.azurianlab.com/api/v1/apipilot/tools/execute)`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Riferimenti precisi di riga (server.py)
|
|
||||||
| # | Cosa | Riga |
|
|
||||||
|---|---|---|
|
|
||||||
| 1 | Env `S2S_APIPILOT_TOOLS_EXECUTE_URL` parsing con esempi `.env:45` | [server.py#L247-L255](file:///home/duxiter/s2s/s2s-client/server.py#L247-L255) |
|
|
||||||
| 2 | Helper `_apipilot_resolve_execute_url()` (calcolo URL finale, priorità env override + append "/execute" se mancante) | [server.py#L321-L338](file:///home/duxiter/s2s/s2s-client/server.py#L321-L338) |
|
|
||||||
| 3 | Regex marker cattura + parse name/arguments (antimeta-commento, JSON con doppie/singole virgolette) | [server.py#L562-L597](file:///home/duxiter/s2s/s2s-client/server.py#L562-L597) |
|
|
||||||
| 4 | Helper `_resolve_apipilot_reasoning_context_block(tools=None, call_result=None)`: genera il testo da iniettare nel prompt. **call_result=None** = PASSATA 1 (lista strumenti + regole marker sintattico); **call_result={name,text,error}** = PASSATA 2 (blocco risultato + trailer rielaborazione) | [server.py#L600-L688](file:///home/duxiter/s2s/s2s-client/server.py#L600-L688) |
|
|
||||||
| 5 | Funzione `_apipilot_execute_tool` aggiornata: usa `_apipilot_resolve_execute_url()` invece di `f"{S2S_APIPILOT_TOOLS_URL}/execute"`; accept header `text/plain,*/*` per body text; controllo empty risolta env; rimozione typo vecchia URL | [server.py#L639-L652](file:///home/duxiter/s2s/s2s-client/server.py#L639-L652) |
|
|
||||||
| 6 | Handler `/api/reasoning`: **FETCH strumenti PASSATA 1** (gate globale `needs_apipilot = bool(tools_url or execute_url)` + cache) subito prima di costruire user_context → log `[APIPILOT] REASONING PASS-1: lista strumenti pronta N=…` | [server.py#L3025-L3055](file:///home/duxiter/s2s/s2s-client/server.py#L3025-L3055) |
|
|
||||||
| 7 | Handler `/api/reasoning`: append injection PASSATA 1 a `user_context_parts` (insieme a today/voice/profile/reports/summary) **e** in OPENAI `system_parts` | [server.py#L3078-L3079](file:///home/duxiter/s2s/s2s-client/server.py#L3078-L3079) · [L3286-L3287](file:///home/duxiter/s2s/s2s-client/server.py#L3286-L3287) |
|
|
||||||
| 8 | Branch **NODE_RED**: marker detection + execute apipilot + build user_context_pass2 con RISULTATO strumento + SECONDA chiamata HTTP `POST /api/reasoning` (X-S2S-Caller=demo-server-pass2) | [server.py#L3273-L3409](file:///home/duxiter/s2s/s2s-client/server.py#L3273-L3409) |
|
|
||||||
| 9 | Branch **OPENAI**: marker detection + execute apipilot + build system_pass2 con blocco risultato + SECONDA chiamata HTTP `POST /chat/completions` (stesso model, stesso transcript, system message aggiornato con risultato) | [server.py#L3502-L3656](file:///home/duxiter/s2s/s2s-client/server.py#L3502-L3656) |
|
|
||||||
| 10 | **FALLBACK di sicurezza**: SE la PASSATA 2 risponde vuota (NODE_RED risposta invalida / OpenAI answer vuota) → l'answer finale usa il **body text RAW dell'esecuzione apipilot** come ultima ratio (nessun silenzio sul canale vocale) | [server.py#L3386-L3389](file:///home/duxiter/s2s/s2s-client/server.py#L3386-L3389) · [L3637-L3640](file:///home/duxiter/s2s/s2s-client/server.py#L3637-L3640) |
|
|
||||||
| 11 | Banner di STARTUP esteso: mostra `apipilotExecuteUrlOverride=…(risolto:…)` + flag `apipilotReasoningPass2=Y` se integrazione attiva | [server.py#L908-L951](file:///home/duxiter/s2s/s2s-client/server.py#L908-L951) |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Formato marker PASSATA 1 (deve essere PRIMA RIGA della risposta)
|
|
||||||
```
|
|
||||||
<<<APIPILOT_CALL:NAME=get_user_briefing,ARGUMENTS={"phone":"+56912345678"}>>>
|
|
||||||
```
|
|
||||||
Regole per l'AI ragionamento:
|
|
||||||
1. **UN SOLO marker** per richiesta (1 tool massimo per invocazione)
|
|
||||||
2. Subito dopo marker → **NESSUN ALTRO TESTO** (il resto viene ignorato, il risultato verrà iniettato nella passata 2)
|
|
||||||
3. `arguments` = **JSON VALIDO** a virgolette DOPPIE, solo campi `required_parameters:` o dichiarati nel tool.
|
|
||||||
4. Se più tool matchano, scegli il più specifico.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Formato blocco iniettivo PASSATA 2 (AI riceve)
|
|
||||||
```
|
|
||||||
### Resultado de la herramienta ApiPilot invocada
|
|
||||||
<<<APIPILOT_RESULT:STATUS=OK,NAME=get_user_briefing>>>
|
|
||||||
|
|
||||||
```tool-output
|
|
||||||
Estimado/a Ana, aquí tienes tu briefing diario para 24-09-2026:
|
|
||||||
- Tienes 3 reportes de campo pendientes de aprobación.
|
|
||||||
- 2 llamadas perdidas de clientes ayer (contactar en las próximas 2h).
|
|
||||||
… (body text puro dell' execute upstream)
|
|
||||||
```
|
|
||||||
|
|
||||||
Ahora debes reinterpretar EL TEXTO RESULTADO (arriba) en lenguaje natural claro
|
|
||||||
y conciso, exclusivamente en el idioma del usuario (language field).
|
|
||||||
No menciones la herramienta internamente ni detalles técnicos.
|
|
||||||
Responde SOLAMENTE con el contenido final que le quieres transmitir al usuario.
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Aggiunto — Tracer console debug sempre-attivo per TUTTE le chiamate fetch lato browser + eventi WebSocket realtime anti-leak/anti-spam + **PREFETCH AUTOMATICO APIPILOT AD OGNI INIZIO SESSIONE REALTIME**
|
|
||||||
|
|
||||||
**Obiettivo**: inserire logging granulare SEMPRE-ATTIVO (non richiede `?debug=true`
|
|
||||||
né localStorage flag) su TUTTE le chiamate di rete effettuate da `s2s-client`.
|
|
||||||
Inoltre, **a partire da v18:05**, GARANTIRE che gli strumenti ApiPilot AquaPortal
|
|
||||||
siano caricati (cache scaldata) ALL'INIZIO DI OGNI SESSIONE WS Realtime prima che
|
|
||||||
l'utente inizi a parlare, così l'LLM li ha disponibili immediatamente per le
|
|
||||||
function-call senza la latenza di discovery al primo tool.
|
|
||||||
|
|
||||||
**Architettura**:
|
|
||||||
- **HTTP fetch** → intercettazione globale via `window.fetch` monkey-patch (primo file caricato, `main.js`) in modo che TUTTE le fetch del codice (anche moduli terzi) siano tracciate, senza dover modificare ogni singola chiamata.
|
|
||||||
- **WebSocket realtime (OpenAI Realtime protocol)** → logging dedicato nel client WS (`s2s-ws-client.js`) con anti-spam per chunk audio 25 Hz e anti-leak token nella connessione.
|
|
||||||
- **Chiamate lato server (es. `S2S_APIPILOT_TOOLS_URL` upstream AquaPortal)** → continuano ad essere loggate dal backend con tag `[APIPILOT-DISCOVERY] →/←` e `[APIPILOT-EXEC] →/←` nel log server (vedi sezione Integrazione ApiPilot). Nel browser si vede SOLO lo strato proxy locale `/api/apipilot/*`, come da architettura passthrough.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### 1. Tag semantici fetch (filtra Console con `[FETCH:xxx]`)
|
|
||||||
|
|
||||||
| Tag | Endpoint / Categoria | Trigger |
|
|
||||||
|---|---|---|
|
|
||||||
| `[FETCH:APIPILOT:TOOLS]` | `GET api/apipilot/tools?include_unauthorized=true` | Chiamata demo fire-and-forget all'avvio, **solo se** `/api/config → apipilot=true` |
|
|
||||||
| `[FETCH:APIPILOT:EXEC]` | `POST api/apipilot/execute` body `{name, arguments}` | Esecuzione manuale tool frontend o LLM function-call via proxy |
|
|
||||||
| `[FETCH:CONFIG]` | `GET api/config` | Bootstrap ogni pagina refresh |
|
|
||||||
| `[FETCH:REASONING]` | `POST api/reasoning` | RAG / context injection |
|
|
||||||
| `[FETCH:SEARCH]` | `POST api/search` | Serper web search |
|
|
||||||
| `[FETCH:PROFILE]` | `GET api/user/phone?phoneNumber=` | Dati profilo login-by-phone |
|
|
||||||
| `[FETCH:REPORTS]` | `GET api/user/reports?phoneNumber=` | Lista report (minimal TOC per default) |
|
|
||||||
| `[FETCH:SESSION:START]` / `SESSION:END` / `SESSION:HEARTBEAT` | `api/session*` | Handshake iniziale + keepalive (heartbeat anti-spam: log singolo debug, non group) |
|
|
||||||
| `[FETCH:QUEUE:POLL]` / `QUEUE:END` | `api/queue/{ticket}` / `api/queue/end` | Polling coda load balancer |
|
|
||||||
| `[FETCH:ACCOUNT:ME]` | `GET api/me` | Chip login + budget (chiamata `account.refresh()`) |
|
|
||||||
| `[FETCH:FETCH]` | Qualsiasi altro URL | Fallback generico se URL non matcha pattern sopra |
|
|
||||||
|
|
||||||
Ogni log fetch contiene sempre (se non è una chiamata poll/end/heartbeat):
|
|
||||||
- `→ [FETCH:TAG] METHOD url` **group collapsed** (blu): method, URL completo, anteprima body request (max 240 chars JSON/TEXT/FORM/QS/BLOB)
|
|
||||||
- `← [FETCH:TAG] STATUS METHOD url (N ms)` **group collapsed**: status HTTP (verde 2xx, giallo 3xx/4xx, rosso 5xx), durata ms via `performance.now()`, lista headers risposta troncati, anteprima body risposta (max 260 chars JSON/TEXT)
|
|
||||||
- `✗ [FETCH:TAG] NETWORK_ERROR (N ms)`: rosso, eccezione catturata e risollevata
|
|
||||||
|
|
||||||
#### 1bis. Prefetch automatico ApiPilot ad OGNI inizio sessione WS Realtime
|
|
||||||
|
|
||||||
**Trigger**: metodo `connect()` di `S2SWebSocketClient` eseguito TUTTE le volte che
|
|
||||||
l'utente fa click sull'orbo verde / inizia una nuova conversazione vocale.
|
|
||||||
Viene eseguito in **parallelo** con `_setupAudio()` e `_openWebSocket()` (Promise.all),
|
|
||||||
quindi **non aggiunge latenza** all'handshake.
|
|
||||||
|
|
||||||
| Caratteristica | Valore |
|
|
||||||
|---|---|
|
|
||||||
| Endpoint client | `GET api/apipilot/tools?include_unauthorized=true` (cache: no-store) |
|
|
||||||
| Tag tracer fetch | `[FETCH:APIPILOT:TOOLS]` (stessa categoria della demo di bootstrap) |
|
|
||||||
| Tag tracer WS | `[S2S-WS:APIPILOT]` (3 stati: **MISS cache** / **HIT cache** / **READY** / **FALLITO**) |
|
|
||||||
| Gating | Solo se `window.__s2sApipilot = true` (dal feature flag `/api/config → apipilot`) |
|
|
||||||
| TTL cache client | `_APIPILOT_CLIENT_TTL_MS = 240 s` = TTL server 300 s − 60 s margine |
|
|
||||||
| Bloccante? | **MAI** — `Promise.all([…, apipilotReady.catch(()=>{})])`; fallimento non rompe `connect()` |
|
|
||||||
| Contatore sessione | `_wsSessionCounter` crescente: ogni sessione logga `#1`, `#2`, … per correlare gli eventi |
|
|
||||||
| Identificatore correlato | `this._sessionSeq` assegnato ad ogni `connect()` per cross-ref nel resto dei log WS |
|
|
||||||
|
|
||||||
Sequenza di log attesa in Console ad **OGNI** nuovo inizio di conversazione vocale:
|
|
||||||
|
|
||||||
```
|
|
||||||
%c[S2S-WS:SESSION]%c #1 handshake iniziato — status: creating-session
|
|
||||||
%c[S2S-WS:APIPILOT]%c #1 MISS cache → GET api/apipilot/tools (proxy a S2S_APIPILOT_TOOLS_URL server)...
|
|
||||||
→ [FETCH:APIPILOT:TOOLS] GET api/apipilot/tools?include_unauthorized=true ← wrapper fetch
|
|
||||||
← [FETCH:APIPILOT:TOOLS] 200 GET (123 ms) ← verde, con preview lista tools
|
|
||||||
%c[S2S-WS:APIPILOT]%c #1 READY strumenti APIPILOT caricati: N=7, t=123ms [primi 5: t1, t2, t3, t4, t5]
|
|
||||||
%c[S2S-WS:SESSION]%c #1 connessione stabilita + audio pipeline + apipilot OK
|
|
||||||
```
|
|
||||||
|
|
||||||
Seconda sessione subito dopo (cache fresca):
|
|
||||||
```
|
|
||||||
%c[S2S-WS:SESSION]%c #2 handshake iniziato — status: creating-session
|
|
||||||
%c[S2S-WS:APIPILOT]%c #2 HIT cache client strumenti (age=12s, N=7, TTL=240s)
|
|
||||||
%c[S2S-WS:SESSION]%c #2 connessione stabilita + audio pipeline + apipilot OK
|
|
||||||
```
|
|
||||||
|
|
||||||
Feature flag disattivato server-side:
|
|
||||||
```
|
|
||||||
%c[S2S-WS:APIPILOT]%c #1 feature flag apipilot=false sul server — salto prefetch strumenti
|
|
||||||
```
|
|
||||||
|
|
||||||
Fallimento proxy (es. backend AquaPortal giù):
|
|
||||||
```
|
|
||||||
%c[S2S-WS:APIPILOT]%c #1 prefetch FALLITO: HTTP 503 — diagnostica lato server log [APIPILOT-DISCOVERY]
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 2. Tag WebSocket (filtra Console con `[S2S-WS:xxx]`)
|
|
||||||
|
|
||||||
| Tag / Badge | Evento tracciato | Note anti-spam / anti-leak |
|
|
||||||
|---|---|---|
|
|
||||||
| `🔵 [S2S-WS:CONNECT]` | `_openWebSocket()`: tentativo di connessione WS | ✅ **Anti-leak**: URL connessione logga solo host+path + params MASCHERATI: ogni param con nome `token\|session\|auth\|jwt\|secret\|key` oppure valore `length > 16` → mostrato come `<prefix>…<last4> len=<N>` per non leakare il JWT in chiaro nella console |
|
|
||||||
| `🟢 [S2S-WS:OPEN]` / `🔴 [S2S-WS:ERR]` | `onopen` / `onerror` | Badge colorati, header risposta (se disponibile) |
|
|
||||||
| `⚪ [S2S-WS:SESSION]` | `connect()` inizio/fine + `_postSession()` grant/queued/402/503 | Ora include contatore `#N` sessione incrementato ad ogni `connect()` + log "stabilita + audio + apipilot OK" finale |
|
|
||||||
| `🔧 [S2S-WS:APIPILOT]` | `_prefetchApipilotTools()` ad ogni inizio sessione | stati: **flag off (warn)** / **HIT cache (ok)** / **MISS cache (connect)** / **READY (ok)** / **FALLITO (err)** — SEMPRE non bloccante |
|
|
||||||
| `🟡 [S2S-WS:QUEUE]` | `_pollQueue()`: ticket / advancement / 402 / 404 | Log primo poll, ogni 10°, ogni avanzamento posizione |
|
|
||||||
| `⏺️ [S2S-WS:SEND]` | `_send()`: messaggi in uscita | ✅ **Anti-spam**: `type=input_audio_buffer.append` (~25 Hz bulk audio) → log solo ogni **250 chunk** oppure ogni **2500 ms** (soglia regolabile). Tutti gli altri tipi: sempre log con anteprima payload max 600 chars; per tool-call `response.function_call_arguments.done` anteprima separata 220 chars |
|
|
||||||
| `📥 [S2S-WS:MSG]` | `_onWsMessage()`: eventi ricevuti | ✅ **High-signal always-on**: matcha regex per eventi critici `session.*`, `error`, `conversation.item.*`, `response.* done`, `function_call`, `rate_limits.updated` → sempre loggati. Eventi transcript/partial `input_audio_buffer.speech_started/stopped` loggati ma silenziati a info |
|
|
||||||
| `🟤 [S2S-WS:CLOSE]` | `_onWsClose()` / `close()` | Badge codice: `1000/1001 = OK (grigio)`, qualsiasi altro = ERR (rosso). Reason loggata se presente. |
|
|
||||||
|
|
||||||
Contatore chunk audio globale: `window.__s2sMicChunkCount` per diagnosticare saturazione microfono 25 Hz.
|
|
||||||
Contatore sessioni totali: `window._wsSessionCounter` (modulo privato) incrementato ad ogni click di iniziale conversazione.
|
|
||||||
|
|
||||||
#### 3. Chiamata demo ApiPilot lato client
|
|
||||||
|
|
||||||
Per garantire che sulla Console si veda **sempre** il tracer attivo per gli endpoint proxy, dopo la ricezione di `/api/config` (linea 2513 di main.js) — se e solo se il feature flag `json.apipilot = true` (cioè `S2S_APIPILOT_TOOLS_URL` valorizzato sul server) — viene inviata una fetch `GET api/apipilot/tools?include_unauthorized=true` (fire-and-forget, cache: no-store).
|
|
||||||
|
|
||||||
La demo produce:
|
|
||||||
- `🔧 [APIPILOT-CLIENT] flag=true → DEMO CALL api/apipilot/tools?include_unauthorized=true` (badge viola, prima del tracer)
|
|
||||||
- Poi **il wrapper fetch logga** `→ [FETCH:APIPILOT:TOOLS] GET api/apipilot/tools?...` + `← [FETCH:APIPILOT:TOOLS] 200 GET (N ms)` (verde) con la lista strumenti in preview
|
|
||||||
- Poi `🔧 [APIPILOT-CLIENT] demo OK: N strumenti [primi 5: tool1, tool2,…]` (badge verde) oppure `…demo FAIL` (ambra) con suggerimento di guardare `[APIPILOT-DISCOVERY]` sul log server
|
|
||||||
|
|
||||||
Valore esportato su `window.__s2sApipilot = bool` per test manuali da Console:
|
|
||||||
```js
|
|
||||||
// Esegui un tool manualmente per vedere [FETCH:APIPILOT:EXEC] nel tracer
|
|
||||||
fetch("api/apipilot/execute", {
|
|
||||||
method: "POST",
|
|
||||||
headers: {"Content-Type": "application/json"},
|
|
||||||
body: JSON.stringify({name: "nome_del_tool", arguments: {parametro: "valore"}})
|
|
||||||
}).then(r => r.json()).then(console.log);
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 4. File modificati (con riferimenti riga)
|
|
||||||
|
|
||||||
- **`s2s-client/main.js`**:
|
|
||||||
- Banner versione aggiornato: [L28-L31](file:///home/duxiter/s2s/s2s-client/main.js#L28-L31)
|
|
||||||
- Wrapper fetch globale `_installFetchDebugTracer()` IIFE — classificazione tag, preview request/response, palette status, performance timing, anti-spam poll/heartbeat: [L41-L251](file:///home/duxiter/s2s/s2s-client/main.js#L41-L251)
|
|
||||||
- `fetchConfig()` — nuovo flag `apipilotEnabled` + `window.__s2sApipilot`: [L2526-L2537](file:///home/duxiter/s2s/s2s-client/main.js#L2526-L2537)
|
|
||||||
- `fetchConfig()` — chiamata demo fire-and-forget ApiPilot tools list: [L2581-L2617](file:///home/duxiter/s2s/s2s-client/main.js#L2581-L2617)
|
|
||||||
- **`s2s-client/ws/s2s-ws-client.js`**:
|
|
||||||
- Banner tracer WS aggiornato + palette `_wsLog(status, tag, msg)` status 0..3 + contatore chunk `_wsMicChunkCount` + cache client ApiPilot `_apipilotClientCache` TTL 240 s + contatore sessioni `_wsSessionCounter`: [L56-L98](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L56-L98)
|
|
||||||
- `connect()` — contatore #N sessione, log inizio/fine handshake, Promise.all([audioReady, wsReady, **apipilotReady**]): [L320-L370](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L320-L370)
|
|
||||||
- **`_prefetchApipilotTools(sessionIdShort)`** — NUOVO METODO: check feature flag, cache TTL client 240 s, fetch `api/apipilot/tools` (proxy server `S2S_APIPILOT_TOOLS_URL`), cache save, envelope check, badge log MISS/HIT/READY/FAIL/FALLO — MAI rilancia eccezione: [L372-L441](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L372-L441)
|
|
||||||
- `_postSession()` — timing handshake grant/queued/402/503: [L469-L520](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L469-L520)
|
|
||||||
- `_pollQueue()` — poll counter + advancement: [L523-L573](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L523-L573)
|
|
||||||
- `_openWebSocket()` — MASCHERAMENTO URL connect params (anti-leak token/JWT): [L682-L728](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L682-L728)
|
|
||||||
- `_onWsMessage()` — high-signal regex, preview `response.function_call_arguments.done`: [L755-L812](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L755-L812)
|
|
||||||
- `_onWsClose()` — badge status code OK/ERR: [L1161-L1180](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L1161-L1180)
|
|
||||||
- `_send()` — anti-spam `input_audio_buffer.append` 250 chunk / 2500 ms: [L1523-L1584](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L1523-L1584)
|
|
||||||
- **`s2s-client/ui/account.js`**:
|
|
||||||
- Chiamata `GET api/me` tracciata automaticamente dal wrapper fetch globale come `[FETCH:ACCOUNT:ME]` → nessuna modifica necessaria (nessun doppio log).
|
|
||||||
|
|
||||||
#### 5. Dove vedere le chiamate UPSTREAM (S2S_APIPILOT_TOOLS_URL)
|
|
||||||
|
|
||||||
⚠️ **Importante — architettura passthrough**:
|
|
||||||
- ✅ **Console browser** → vedi `[FETCH:APIPILOT:TOOLS]` e `[FETCH:APIPILOT:EXEC]` = chiamate locali al proxy `/api/apipilot/*` sul server S2S
|
|
||||||
- ✅ **Log server (terminale backend)** → vedi `[APIPILOT-DISCOVERY] → GET https://aquachile.azurianlab.com/api/v1/apipilot/tools?...` e `[APIPILOT-EXEC] → POST …/execute` = chiamate reali upstream all'URL AquaPortal (`S2S_APIPILOT_TOOLS_URL`)
|
|
||||||
|
|
||||||
Questo è per design: il client non conosce mai l'URL upstream AquaPortal (anti-hardcoding), tutto passa per il proxy locale su `server.py` che aggiunge header auth e validazione schema PRE-CHECK.
|
|
||||||
|
|
||||||
### Risolto — Bug: ApiPilot chiamato 1 sola volta (anonimo) e MAI più dopo login phone o refresh
|
|
||||||
|
|
||||||
**Sintomo osservato dal log aquachile fornito**:
|
|
||||||
- Alle `21:06:46` 1 sola chiamata `[apipilot] list_tools.start user=(anon)` a `/api/v1/apipilot/tools` (chiamata demo anonima bootstrap)
|
|
||||||
- Subito dopo: multipli `POST /auth/phone-login` e `GET /auth/user-by-phone` (utenti autenticati, 21:06:49, 21:07:16, 21:07:19, 21:07:39) ma **NESSUNA** chiamata apipilot ripetuta.
|
|
||||||
|
|
||||||
**Root cause doppia**:
|
|
||||||
1. **Cache client globale (240 s) user-agnostic**: la demo bootstrap anonima e le sessioni autenticate condividevano la stessa entry `_apipilotClientCache` senza chiave per utente. Quindi la cache anonima HIT veniva riutilizzata per utenti autenticati DIVERSI.
|
|
||||||
2. **Cache server S2S (300 s) keyed per auth fingerprint** (`SHA-256(auth)[:12] | x-demo-session`) ma dal client **non veniva propagato alcun token `x-demo-session`**, quindi tutte le chiamate usavano la fingerprint "vuota" (anonima) = cache anonima riutilizzata senza mai chiamare AquaPortal per utenti autenticati.
|
|
||||||
3. **Trigger demo solo su bootstrap**: nessuna chiamata apipilot dopo `phone-login` riuscito, che è il momento in cui cambia il contesto utente.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### Fix applicati
|
|
||||||
|
|
||||||
##### Fix 1 — Prefetch ApiPilot DOPO OGNI login phone riuscito (con `refresh=true`)
|
|
||||||
Inserito blocco fire-and-forget **subito dopo** la conferma HTTP 200 in `fetchVoceroUserProfile()` (dopo caricamento profilo):
|
|
||||||
|
|
||||||
| Caratteristica | Valore |
|
|
||||||
|---|---|
|
|
||||||
| Trigger | `api/user/phone?phoneNumber=…` HTTP 200 + profile.result object valido |
|
|
||||||
| Endpoint | `GET api/apipilot/tools?include_unauthorized=true&refresh=true` |
|
|
||||||
| Parametro `refresh=true` | **OBBLIGATORIO** → forza il proxy S2S lato server a **ignorare la cache 300 s** e chiamare veramente AquaPortal upstream (così `[apipilot] list_tools.start user=…` riappare nel log aquachile per OGNI login phone) |
|
|
||||||
| Headers | Propaga `x-demo-session` se presente in `window.__s2sDemoSessionToken` (per fingerprint auth S2S diversa da anonima) |
|
|
||||||
| Effetto collaterale | Invoca `window.__s2sInvalidateApipilotCache()` = invalida cache client WS per tutte le sessioni vocali successive → nessun riuso cache per nuovo utente |
|
|
||||||
| Log Console | `🔧 [APIPILOT-CLIENT] post-login phone → FORZO refresh api/apipilot/tools (badge viola)` + `post-login OK (dt=XYZ ms): N=X strumenti` o `post-login FAIL` |
|
|
||||||
|
|
||||||
Riferimento riga: [main.js#L2418-L2464](file:///home/duxiter/s2s/s2s-client/main.js#L2418-L2464)
|
|
||||||
|
|
||||||
##### Fix 2 — Cache client WS per-utente (invalida se cambia phone/token)
|
|
||||||
La entry `_apipilotClientCache` ora include la proprietà `key = _apipilotUserKey()`.
|
|
||||||
|
|
||||||
Chiave `_apipilotUserKey()` calcolata come:
|
|
||||||
1. `P:<ultime-8-cifre-phoneNumber>` → se c'è `?phoneNumber=` in URL
|
|
||||||
2. `T:<ultimi-6-char-x-demo-session-token>` → se c'è token
|
|
||||||
3. `anon` → altrimenti
|
|
||||||
|
|
||||||
HIT cache ora richiede **3 condizioni**:
|
|
||||||
1. `now - cached.at <= 240 s` (TTL ancora attivo)
|
|
||||||
2. `cached.key === currentUser` (stesso phone / stesso token → stesso utente)
|
|
||||||
3. Feature flag `window.__s2sApipilot = true`
|
|
||||||
|
|
||||||
Riferimento riga: [s2s-ws-client.js#L94-L126](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L94-L126) (cache+key+invalidazione globale) · [L411-L485](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L411-L485) (metodo `_prefetchApipilotTools` con check key utente, refresh=true se cambia utente e propagate header x-demo-session)
|
|
||||||
|
|
||||||
##### Fix 3 — Propagazione header `x-demo-session` per fingerprint auth lato server S2S
|
|
||||||
Ora, **quando disponibile** `window.__s2sDemoSessionToken` (length > 4), entrambe le chiamate apipilot (prefetch login + WS connect) propagano:
|
|
||||||
```
|
|
||||||
x-demo-session: <token>
|
|
||||||
```
|
|
||||||
→ la cache lato server S2S (`SHA-256(auth)[:12] | x-demo-session`) usa chiave **diversa per ogni token/sessione** = **NESSUNA condivisione cache tra Anon e Utente Autenticato** o tra utenti diversi.
|
|
||||||
|
|
||||||
##### Fix 4 — Funzione globale invalidazione cache per cross-modulo
|
|
||||||
```js
|
|
||||||
window.__s2sInvalidateApipilotCache(); // resetta _apipilotClientCache = null
|
|
||||||
```
|
|
||||||
Chiamata automaticamente da `main.js` dopo login phone riuscito. Utile anche per test manuali da Console browser.
|
|
||||||
|
|
||||||
Riferimento riga: [s2s-ws-client.js#L111-L126](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L111-L126)
|
|
||||||
|
|
||||||
##### Timeline attesa nel log AquaPortal DOPO questo fix (log fornito ripetuto)
|
|
||||||
```
|
|
||||||
21:06:46 [apipilot] list_tools.start user=(anon) ← demo bootstrap anonimo (refresh page)
|
|
||||||
21:06:46 [apipilot] list_tools.ok count=1 …
|
|
||||||
21:06:49 [apipilot] list_tools.start user=<username-A> ← FIX #1 post-login phone refresh=true ✅ NUOVO
|
|
||||||
21:06:49 [apipilot] list_tools.ok count=…
|
|
||||||
21:07:16 [apipilot] list_tools.start user=<username-B> ← FIX #1 login nuovo utente ✅ NUOVO
|
|
||||||
21:07:16 [apipilot] list_tools.ok …
|
|
||||||
21:07:19 [apipilot] list_tools.start user=<username-A> ← se cache client S2S invalidata oppure
|
|
||||||
21:07:19 … x-demo-session cambiato
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Aggiunto — Integrazione ApiPilot AquaPortal (strumenti dinamici, no hardcoding)
|
|
||||||
|
|
||||||
Implementazione lato proxy `s2s-client/server.py` del bridge definito in
|
|
||||||
[GUIDA_APIPILOT_CLIENTE.md](file:///home/duxiter/s2s/GUIDA_APIPILOT_CLIENTE.md).
|
|
||||||
|
|
||||||
**Obiettivo**: consentire al client Vocero di **consumare dinamicamente** i
|
|
||||||
tool esposti dal backend AquaPortal via ApiPilot, **senza hardcodare nomi di
|
|
||||||
endpoint, parametri o autorizzazioni** nel codice client. Tutta la conoscenza
|
|
||||||
su "quali tool esistono / parametri accettano / richiedono autenticazione"
|
|
||||||
viene **letta a runtime** tramite discovery standardizzata.
|
|
||||||
|
|
||||||
#### Ambiente richiesto (`.env`)
|
|
||||||
|
|
||||||
| Variabile | Valore esempio | Significato |
|
|
||||||
|---|---|---|
|
|
||||||
| **`S2S_APIPILOT_TOOLS_URL`** | `https://aquachile.azurianlab.com/api/v1/apipilot` | URL **base** (senza `/tools` o `/execute`) del backend ApiPilot. Se vuoto → feature completamente disattivata (passthrough). |
|
|
||||||
| `S2S_APIPILOT_SERVICE_TOKEN` | `optional_Bearer_service_token` | Token di fallback usato solo se il chiamante non passa `Authorization` o `x-demo-session`. |
|
|
||||||
| `S2S_APIPILOT_TTL_SEC` | `300` | TTL cache in-memory discovery tools (default 300 s = 5 min — come da guida). Range ammesso [15, 3600]. |
|
|
||||||
| `S2S_APIPILOT_TIMEOUT_SEC` | `15.0` | Timeout HTTP discovery + execute (default 15 s). Range [1, 120]. |
|
|
||||||
|
|
||||||
#### Endpoint pubblico esposti dal proxy (stessa origine)
|
|
||||||
|
|
||||||
| Endpoint | Verb | Descrizione |
|
|
||||||
|---|---|---|
|
|
||||||
| `/api/apipilot/tools?include_unauthorized=true&refresh=false` | **GET** | Lista tool conformi §2.1 (`ToolDefinition`: `name, description, parameters, requires_auth`). Ritorno wrappato `{success, data:{count, include_unauthorized, tools}, error}`. Supporta `?refresh=true` per forzare refresh cache. |
|
|
||||||
| `/api/apipilot/execute` | **POST** | Esegue un tool conformi §2.2. Body: `{name: string, arguments: {}}`. PRE-CHECK obbligatori prima dell'HTTP. Risposta SEMPRE envelope `{success, data, error}`. HTTP status code restituito = `error.status_code` quando presente (4xx/5xx). |
|
|
||||||
|
|
||||||
Estensione di `/api/config`: nuova chiave top-level booleana
|
|
||||||
`"apipilot": true/false` (basata su `S2S_APIPILOT_TOOLS_URL`). Il frontend
|
|
||||||
la usa per decidere se mostrare il catalogo / i bottoni "Esegui tool" (prende
|
|
||||||
la lista da `/api/apipilot/tools` anziché hardcodare).
|
|
||||||
|
|
||||||
#### PRE-CHECK obbligatori (flow guida §2.2, tutti implementati)
|
|
||||||
|
|
||||||
1. **Fresh discovery** — prima di ogni `execute` viene usata la lista tool
|
|
||||||
recuperata da `_apipilot_fetch_tools(...)` rispettando la TTL. Se il tool
|
|
||||||
non viene trovato, **un solo** re-discovery forzato prima di fallire.
|
|
||||||
2. **Controllo `requires_auth`** — se `true` ma nessun token presente
|
|
||||||
(Authorization / x-demo-session / S2S_APIPILOT_SERVICE_TOKEN) → blocco
|
|
||||||
HTTP 401 con envelope `{code: "AUTH_REQUIRED", message: "La herramienta
|
|
||||||
«nome» requiere autenticación…"}` — **NESSUNA richiesta viene inviata al
|
|
||||||
backend upstream** in questo caso.
|
|
||||||
3. **Validazione JSON-Schema strict** (`_apipilot_validate_arguments`):
|
|
||||||
- rimuove **tutte** le chiavi extra non in `parameters.properties`
|
|
||||||
(avviso `[APIPILOT-VALIDATE] … removing unknown arg=…`);
|
|
||||||
- **NON inventa parametri**;
|
|
||||||
- restituisce errore 400 `INVALID_ARGUMENTS` per ogni campo `required`
|
|
||||||
mancante (messaggi in spagnolo).
|
|
||||||
4. **Passthrough header autenticazione**:
|
|
||||||
`Authorization: Bearer <TOKEN_UTENTE>` oppure `x-demo-session:
|
|
||||||
<SESSION_TOKEN>` vengono inoltrati VERBATIM al backend AquaPortal (§2.1
|
|
||||||
nota), **mai loggati** (solo presenza: log `auth=Y / demo=Y`).
|
|
||||||
5. **Compatibilità LLM function-calling**: helper `_apipilot_strip_tool_for_llm(tool_def)`
|
|
||||||
che proietta `ToolDefinition` nel formato standard OpenAI
|
|
||||||
`{type: "function", function: {name, description, parameters}}` — **rimuove
|
|
||||||
`requires_auth`** (§4 della guida: "non mandare direttamente l'output
|
|
||||||
grezzo di /tools al modello").
|
|
||||||
|
|
||||||
#### Garanzie / sicurezza / anti-leak
|
|
||||||
|
|
||||||
| Controllo | Implementazione |
|
|
||||||
|---|---|
|
|
||||||
| Nessuna condivisione cache tra identità diverse | Chiave cache = `SHA-256(authorization)[:12] \| x-demo-session` → fingerprint anonimo, non salva token in cache |
|
|
||||||
| Doppio controllo cache con lock asincrono | `_APIPILOT_CACHE_LOCK` async + double check dopo acquisizione lock → 1 sola fetch per refresh |
|
|
||||||
| Fallback stantia su errore rete | Se backend va giù e abbiamo entry in cache → restituisce fallback loggando warning invece di crash |
|
|
||||||
| Inoltro header Auth mai loggato | Solo log `auth=Y/- demo=Y/-` |
|
|
||||||
| Cap parametri unknown | Rimossi, mai inoltrati al backend |
|
|
||||||
| Envelope errori uniforme | Sempre `{success, data, error}` anche per errori pre-check locali |
|
|
||||||
| Nomi file env coerenti | Nessun URL o token hardcoded |
|
|
||||||
| Banner startup di conferma | `[STARTUP] … apipilot=<URL> apipilotServiceToken=Y/- apipilotTtl=<N>s apipilotTimeout=<N>s` |
|
|
||||||
|
|
||||||
**File modificati:**
|
|
||||||
|
|
||||||
- **`server.py`** ([server.py](file:///home/duxiter/s2s/s2s-client/server.py)) — Sezione APIPILOT completa inserita tra `OPENAI_PROJECT` e `NODE_RED_REASONING_URL`:
|
|
||||||
1. Definizione 4 env + normalizzazione + TTL/timeout cap range
|
|
||||||
([L236-L266](file:///home/duxiter/s2s/s2s-client/server.py#L236-L266)).
|
|
||||||
2. Cache in-process `_APIPILOT_CACHE: dict[fp → (monotonic, tools[])]` + lock
|
|
||||||
asincrona `_APIPILOT_CACHE_LOCK` ([L268-L271](file:///home/duxiter/s2s/s2s-client/server.py#L268-L271)).
|
|
||||||
3. Helpers pura: `_apipilot_auth_fingerprint(auth, demo)`
|
|
||||||
([L274-L287](file:///home/duxiter/s2s/s2s-client/server.py#L274-L287)) · `_apipilot_parse_authorization(Request)`
|
|
||||||
([L290-L309](file:///home/duxiter/s2s/s2s-client/server.py#L290-L309)) · `_apipilot_strip_tool_for_llm(tool_def)`
|
|
||||||
([L312-L338](file:///home/duxiter/s2s/s2s-client/server.py#L312-L338)).
|
|
||||||
4. Discovery: `_apipilot_fetch_tools(auth, demo, include_unauthorized, force_refresh)`
|
|
||||||
([L341-L481](file:///home/duxiter/s2s/s2s-client/server.py#L341-L481)) — cache hit log, stale fallback, unwrap payload `list/{tools,result,data}.tools`, sanitizzazione ToolDefinition minima, log `[APIPILOT-DISCOVERY] →/←`.
|
|
||||||
5. Validazione: `_apipilot_validate_arguments(arguments, tool_def)`
|
|
||||||
([L484-L517](file:///home/duxiter/s2s/s2s-client/server.py#L484-L517)) — rimuove extra, check required, messaggi in spagnolo.
|
|
||||||
6. Execute: `_apipilot_execute_tool(auth, demo, name, arguments)`
|
|
||||||
([L520-L684](file:///home/duxiter/s2s/s2s-client/server.py#L520-L684)) — PRE-CHECK 1-2-3 (re-discovery 1 volta, requires_auth 401, validazione 400), POST /execute envelope, gestione non-JSON HTTP, wrap envelope se backend non lo restituisce, log `[APIPILOT-EXEC] →/←`.
|
|
||||||
7. Banner `[STARTUP] Other externals`: esteso con 4 nuovi placeholder `apipilot`, `apipilotServiceToken`, `apipilotTtl`, `apipilotTimeout`
|
|
||||||
([L726-L752](file:///home/duxiter/s2s/s2s-client/server.py#L726-L752)).
|
|
||||||
8. **`/api/config`**: nuova chiave booleana `"apipilot": bool(S2S_APIPILOT_TOOLS_URL)` + docstring, log esteso
|
|
||||||
([L2544-L2579](file:///home/duxiter/s2s/s2s-client/server.py#L2544-L2579)).
|
|
||||||
9. **Pydantic model** `ApipilotExecuteRequest(name:str, arguments:dict={})` ([L2582-L2585](file:///home/duxiter/s2s/s2s-client/server.py#L2582-L2585)).
|
|
||||||
10. **`GET /api/apipilot/tools`** proxy pubblico ([L2588-L2651](file:///home/duxiter/s2s/s2s-client/server.py#L2588-L2651)):
|
|
||||||
query `?include_unauthorized=true(default)&refresh=false`, envelope uniforme `{success, data: {count, include_unauthorized, tools}, error}`, 501 se env non impostato, HTTPException convertite in envelope.
|
|
||||||
11. **`POST /api/apipilot/execute`** proxy pubblico ([L2654-L2700](file:///home/duxiter/s2s/s2s-client/server.py#L2654-L2700)):
|
|
||||||
body `{name, arguments}`, PRE-CHECK 1-2-3 tramite `_apipilot_execute_tool`, status code risposta = `error.status_code` se errore, messaggi utente finali in spagnolo §2.2.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [2026-09-23]
|
|
||||||
|
|
||||||
### Aggiunto — Bottone «Descargar conversación» nell'header del pannello chat (export JSON + TXT client-side offline)
|
|
||||||
|
|
||||||
Nuovo bottone tondo icona download affiancato alla spia «En línea» sulla destra
|
|
||||||
dell'header del pannello `#chat-panel`. Alla pressione serializza TUTTO lo
|
|
||||||
storico persistente (`.hist-msg` dentro `#chat-history`) e scarica DUE file
|
|
||||||
contemporaneamente, SENZA chiamate al backend, interamente client-side e
|
|
||||||
offline-safe (Blob + URL.createObjectURL + `<a download>`):
|
|
||||||
|
|
||||||
1. `vocero-conversacion[-<nombreUsuario>]-YYYYMMDD-HHMMSS.json` — struttura dati
|
|
||||||
con meta `{ exportedAt, messageCount, messages: [...] }`. Tipi supportati:
|
|
||||||
- `user` / `assistant` → `{ type, text, timestamp, name?, initials?, interrupted? }`
|
|
||||||
- `tool` → `{ type: "tool", timestamp, name, args, output }` (risultato chiamate tool)
|
|
||||||
- `image` → `{ type: "image", timestamp, dataUrl }` (scatti webcam camera tool)
|
|
||||||
2. `vocero-conversacion[-<nombreUsuario>]-YYYYMMDD-HHMMSS.txt` — log leggibile
|
|
||||||
per umani in spagnolo, ogni entry formattata come
|
|
||||||
`HH:MM Nombre / Vocero / HERRAMIENTA <nome>: testo`.
|
|
||||||
|
|
||||||
**UI / UX (tutto in spagnolo, come richiesto):**
|
|
||||||
- `aria-label="Descargar conversación"`
|
|
||||||
- `title="Descargar conversación (JSON + TXT)"`
|
|
||||||
- **Feedback OK**: dopo export riuscito bottone diventa verde per 650 ms
|
|
||||||
(`.chat-export-ok` con ring + glow).
|
|
||||||
- **Feedback vuoto**: se nessun messaggio → nessun download, bottone flash
|
|
||||||
giallo/ámbar + micro-shake 0.42s (`.chat-export-empty`) + console info
|
|
||||||
`[ui/chat.js:exportConversation] No hay mensajes aún — exportación omitida.`
|
|
||||||
- **Nomi file sanitizzati**: helper `_safeName(...)` rimuove unicode/
|
|
||||||
caratteri speciali/spazi → `a-z 0-9 - _`.
|
|
||||||
- **Stili coerenti**: dark (glass/aurora) + white (screenshot mobile style)
|
|
||||||
entrambe tematizzate con colori esistenti `var(--brand)`, `#16a34a`, `#f59e0b`.
|
|
||||||
|
|
||||||
**Comportamenti garantiti:**
|
|
||||||
- 🔒 Zero dati trasmessi fuori dal browser (nessun POST, nessun analytics).
|
|
||||||
- Le bolle effimere sull'orb (`.bubble` / `.bubble-stack`) sono volutamente
|
|
||||||
ignorate: esporta SOLO lo storico permanente, che è quello significativo.
|
|
||||||
- Never-throws: intero export avvolto in `try/catch`; errore solo in console.
|
|
||||||
- `window.URL.revokeObjectURL(...)` dopo 4 s → nessun leak di memoria Blob.
|
|
||||||
- Disponibile in TUTTI i temi (dark / white / default).
|
|
||||||
|
|
||||||
**File modificati:**
|
|
||||||
|
|
||||||
- **`index.html`** ([index.html](file:///home/duxiter/s2s/s2s-client/index.html)):
|
|
||||||
1. Wrapper `.chat-header-actions` a destra dell'header, che ingloba la pill
|
|
||||||
«En línea» + nuovo bottone `#chat-export-btn` (SVG freccia-giù)
|
|
||||||
([L374-L394](file:///home/duxiter/s2s/s2s-client/index.html#L374-L394)).
|
|
||||||
|
|
||||||
- **`ui/chat.js`** ([chat.js](file:///home/duxiter/s2s/s2s-client/ui/chat.js)):
|
|
||||||
1. Helper module-level `_safeName(prefix)` per sanitizzare nomi file
|
|
||||||
([L38-L50](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L38-L50)).
|
|
||||||
2. Rif `this._exportBtn = getElementById("chat-export-btn")` + `click` listener
|
|
||||||
nel `ChatView` constructor ([L74-L78](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L74-L78)).
|
|
||||||
3. Nuova sezione `// ── Export / download conversation ──` con 4 metodi:
|
|
||||||
- `_serializeConversation()` → estrae `.hist-msg` rows con querySelector
|
|
||||||
`:scope > .hist-msg`, gestisce `user/assistant/tool/image`, preserva
|
|
||||||
timestamp `data-timestamp`, nome utente, initials, flag interrupted
|
|
||||||
([L521-L582](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L521-L582)).
|
|
||||||
- `_conversationToText(conv)` → pretty-format testo spagnolo con header
|
|
||||||
data, totali, blocchi tool indentati, anteprima immagine dataURL
|
|
||||||
([L584-L621](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L584-L621)).
|
|
||||||
- `_downloadBlob(content, filename, mime)` → pattern `<a download>` +
|
|
||||||
revokeObjectURL ([L623-L645](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L623-L645)).
|
|
||||||
- `exportConversation()` (pubblico) → entry-point, nome file con timbro
|
|
||||||
UTC locale `YYYYMMDD-HHMMSS`, include nome profilo Vocero se presente,
|
|
||||||
logica empty/OK/error feedback, 2 download distinti
|
|
||||||
([L647-L697](file:///home/duxiter/s2s/s2s-client/ui/chat.js#L647-L697)).
|
|
||||||
|
|
||||||
- **`style.css`** ([style.css](file:///home/duxiter/s2s/s2s-client/style.css)):
|
|
||||||
1. `.chat-header-actions` flex-row right-aligned
|
|
||||||
([L4919-L4931](file:///home/duxiter/s2s/s2s-client/style.css#L4919-L4931)).
|
|
||||||
2. `.chat-export-btn` baseline (dark-theme): 32×32 pill, glassy, hover/focus
|
|
||||||
states, transizioni 120-160 ms
|
|
||||||
([L4932-L4968](file:///home/duxiter/s2s/s2s-client/style.css#L4932-L4968)).
|
|
||||||
3. Stati feedback: `.chat-export-ok` (verde) + `.chat-export-empty`
|
|
||||||
(ámbar + animazione shake 5 step)
|
|
||||||
([L4969-L4989](file:///home/duxiter/s2s/s2s-client/style.css#L4969-L4989)).
|
|
||||||
4. Varianti white-theme complete: `.chat-header-actions`, base, hover,
|
|
||||||
active, OK, empty
|
|
||||||
([L4991-L5015](file:///home/duxiter/s2s/s2s-client/style.css#L4991-L5015)).
|
|
||||||
|
|
||||||
### Aggiunto — `S2S_LATEST_SUMMARY_TEXT`: override statico del summary ultima conversazione (testo pulito, nessuna chiamata HTTP)
|
|
||||||
|
|
||||||
Nuova variabile d'ambiente che, se valorizzata, ha **priorità assoluta** su `S2S_LATEST_SUMMARY`
|
|
||||||
e salta completamente la fetch HTTP upstream (login-by-phone, Bearer, timeout, JSON unwrap: *tutto bypassato*).
|
|
||||||
Il suo valore testo-plain viene iniettato direttamente come blocco "latest conversation summary"
|
|
||||||
nel prompt LLM (sia canale WebSocket `/v1/realtime` che `POST /api/reasoning`).
|
|
||||||
|
|
||||||
Progettata per il caso d'uso in cui l'operatore vuole iniettare una versione
|
|
||||||
**pulita, essenziale e pre-pulita** del summary senza dipendere da nessun endpoint esterno.
|
|
||||||
|
|
||||||
**Valore `.env` di esempio (multiriga con `\n`):**
|
|
||||||
```env
|
|
||||||
# S2S_LATEST_SUMMARY=https://api.example.com/summary # ← disabilitato: S2S_LATEST_SUMMARY_TEXT ha priorità
|
|
||||||
S2S_LATEST_SUMMARY_TEXT=Última conversación 2026-09-22:\n- Usuario preguntó por mortalidad salmón AquaChile semana 38.\n- Se refirió al reporte Porcentaje Mortalidad.
|
|
||||||
```
|
|
||||||
|
|
||||||
**Comportamento chiave:**
|
|
||||||
- Le sequenze `\n` nel valore env vengono interpretate come a-capo vero (stessa convenzione di `S2S_PRIMARY_PROMPT`)
|
|
||||||
- **Non richiede `phoneNumber`**: funziona anche per sessioni generiche senza identificativo chiamante
|
|
||||||
- Il wrapper `LATEST_SUMMARY_HINT` + il cap a **1600 caratteri** sono applicati comunque (passaggio sempre attraverso `_resolve_latest_summary_context`)
|
|
||||||
- Il blocco viene correttamente classificato dal layer dedup (marker invisibile `SUMMARY_FULL_MARKER`) — nessun rischio di duplicato
|
|
||||||
- Se `S2S_LATEST_SUMMARY_TEXT` è vuoto → **fallback** trasparente al vecchio flusso (`S2S_LATEST_SUMMARY` URL + `phoneNumber`)
|
|
||||||
|
|
||||||
**Log attesi nel server per confermare override attivo:**
|
|
||||||
```
|
|
||||||
[INJECT-SUMMARY] S2S_LATEST_SUMMARY_TEXT OVERRIDE active (len=<N> chars). Skipping HTTP fetch; injecting raw static plain-text summary directly.
|
|
||||||
```
|
|
||||||
|
|
||||||
**File modificati:**
|
|
||||||
|
|
||||||
- **`server.py`** ([server.py](file:///home/duxiter/s2s/s2s-client/server.py)):
|
|
||||||
1. Definizione nuova env `S2S_LATEST_SUMMARY_TEXT` + interpretazione escape `\n` ([L128-L140](file:///home/duxiter/s2s/s2s-client/server.py#L128-L140)).
|
|
||||||
2. Banner `[STARTUP]` esteso con campo `summaryText=%s` per conferma visiva a boot ([L275-L297](file:///home/duxiter/s2s/s2s-client/server.py#L275-L297)).
|
|
||||||
3. **Ramo override** PRIMA di qualsiasi logica HTTP dentro `_fetch_latest_summary_for_phone(...)` ([L1563-L1589](file:///home/duxiter/s2s/s2s-client/server.py#L1563-L1589)):
|
|
||||||
- return diretto del valore statico;
|
|
||||||
- log `[INJECT-SUMMARY] … OVERRIDE active`;
|
|
||||||
- anteprima raw prime 800 chars se `S2S_DEBUG_PROMPT=true`.
|
|
||||||
4. **Gate REASONING:** `needs_summary = bool(S2S_LATEST_SUMMARY) or bool(S2S_LATEST_SUMMARY_TEXT)` ([L2153](file:///home/duxiter/s2s/s2s-client/server.py#L2153)).
|
|
||||||
5. **Skip REASONING rilassato:** condizione `(not phone) and (not needs_reports) and (not episode_id) and (not S2S_LATEST_SUMMARY_TEXT)` — il testo statico non richiede phone ([L2159-L2168](file:///home/duxiter/s2s/s2s-client/server.py#L2159-L2168)).
|
|
||||||
6. **Esecuzione summary REASONING:** `needs_summary and (phone or S2S_LATEST_SUMMARY_TEXT)`; log "READY" esteso con flag `summaryTextOverride=%s` ([L2240-L2256](file:///home/duxiter/s2s/s2s-client/server.py#L2240-L2256)).
|
|
||||||
7. **Gate WS connect:** medesimo `needs_summary` esteso con `S2S_LATEST_SUMMARY_TEXT` ([L2644](file:///home/duxiter/s2s/s2s-client/server.py#L2644)).
|
|
||||||
8. **Skip WS connect rilassato:** anche senza `phoneNumber` se c'è `S2S_LATEST_SUMMARY_TEXT` la pipeline parte ([L2653-L2664](file:///home/duxiter/s2s/s2s-client/server.py#L2653-L2664)).
|
|
||||||
9. **Esecuzione summary WS:** medesima condizione rilassata + log "READY" con campo `summaryTextOverride=%s` ([L2725-L2746](file:///home/duxiter/s2s/s2s-client/server.py#L2725-L2746)).
|
|
||||||
|
|
||||||
- **`INJECTIONS.md`** ([INJECTIONS.md](file:///home/duxiter/s2s/s2s-client/INJECTIONS.md)):
|
|
||||||
1. Sezione **4. Blocco summary ultima conversazione** completamente riscritta:
|
|
||||||
- nuova tabella 2 modalità mutuamente esclusive (A = HTTP fetch dinamico / caller, B = override statico hardcoded);
|
|
||||||
- paragrafo **Modalità B (override statico)** con esempio `.env` e nota su escape `\n`;
|
|
||||||
- riferimenti codice aggiornati ([L206-L258](file:///home/duxiter/s2s/s2s-client/INJECTIONS.md#L206-L258)).
|
|
||||||
2. **Tabella riepilogo trigger** riga #4 aggiornata:
|
|
||||||
- Trigger: `phoneNumber oppure override statico`;
|
|
||||||
- env: `S2S_LATEST_SUMMARY` (HTTP) **o** `S2S_LATEST_SUMMARY_TEXT` (statico, priorità);
|
|
||||||
- parametro: `phoneNumber=` (solo HTTP; override statico non lo richiede) ([L409-L417](file:///home/duxiter/s2s/s2s-client/INJECTIONS.md#L409-L417)).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [2026-09-07]
|
|
||||||
|
|
||||||
### Aggiunto — Supporto `episode_id` (podcast episode inject)
|
|
||||||
|
|
||||||
Nuovo parametro query string `?episode_id=` affiancato a `?phoneNumber=` che
|
|
||||||
abilita il caricamento e l'iniezione automatica di un episodio podcast come
|
|
||||||
contesto PRIMARIO nel prompt LLM.
|
|
||||||
|
|
||||||
**URL di esempio:**
|
|
||||||
```
|
|
||||||
/?phoneNumber=%2B447432477433&episode_id=episode-19569f77-a06e-5ce5-a52f-6299dbd67d03
|
|
||||||
```
|
|
||||||
|
|
||||||
**File modificati:**
|
|
||||||
|
|
||||||
- **`server.py`** ([server.py](file:///home/duxiter/s2s/s2s-client/server.py)):
|
|
||||||
1. Aggiunta costante `PODCAST_EPISODE_HINT` ([L426-L429](file:///home/duxiter/s2s/s2s-client/server.py#L426-L429)) per l'hint semantico nel prompt.
|
|
||||||
2. Aggiunta funzione `_resolve_episode_context(raw_value)` ([L475-L504](file:///home/duxiter/s2s/s2s-client/server.py#L475-L504)): normalizza la risposta API (dict/list/str, eventuale wrapper `{"result": …}`), la serializza in JSON (cap 24.000 chars) e la avvolge con i marker `<<<PODCAST_EPISODE_BEGIN/END>>>`.
|
|
||||||
3. Aggiunta funzione asincrona `_fetch_podcast_episode(episode_id)` ([L1010-L1078](file:///home/duxiter/s2s/s2s-client/server.py#L1010-L1078)): GET plain verso
|
|
||||||
`https://aquachile.azurianlab.com/api/v1/podcast/episodes/{episode_id}`
|
|
||||||
(timeout 15s, follow redirects, error handling soft — non rompe mai il flusso, restituisce `None` su fail).
|
|
||||||
4. Estesa `_augment_instructions_with_profile(...)` ([L1137-L1181](file:///home/duxiter/s2s/s2s-client/server.py#L1137-L1181)) con nuovo parametro `episode_context_text`; l'ordine di inject ora è (più specifico → più generico): **episode → profile → reports → summary**.
|
|
||||||
5. Estesa `_patch_client_session_update_with_profile(...)` ([L1184-L1306](file:///home/duxiter/s2s/s2s-client/server.py#L1184-L1306)) con supporto a `episode_context_text` e logging dedicato `[INJECT-EPISODE]` / `[DEBUG-EPISODE]`.
|
|
||||||
6. WebSocket connect handler ([L1791-L1959](file:///home/duxiter/s2s/s2s-client/server.py#L1791-L1959)):
|
|
||||||
- legge `episode_id` da query params assieme a `phoneNumber`;
|
|
||||||
- esegue fetch episodio + resolve PRIMA di profile/reports/summary;
|
|
||||||
- salva `injected_episode_context` e `inject_episode_id` nel `session_ctx` condiviso.
|
|
||||||
7. `_client_to_upstream` ([L2649-L2661](file:///home/duxiter/s2s/s2s-client/server.py#L2649-L2661)): passa `ctx_episode_context` al patch di `session.update`.
|
|
||||||
8. Endpoint `POST /api/reasoning`:
|
|
||||||
- esteso modello `ReasoningRequest` con campo `episode_id: str | None = None` ([L1309-L1323](file:///home/duxiter/s2s/s2s-client/server.py#L1309-L1323));
|
|
||||||
- nel handler ([L1411-L1577](file:///home/duxiter/s2s/s2s-client/server.py#L1411-L1577)) fetch episodio indipendente da phone;
|
|
||||||
- `inject_episode_text` viene **prepend** a `user_context_parts` (prima di tutti);
|
|
||||||
- inoltro a NODE_RED / OPENAI con logging esteso (`episodeIdLen`).
|
|
||||||
|
|
||||||
- **`main.js`** ([main.js](file:///home/duxiter/s2s/s2s-client/main.js)):
|
|
||||||
1. Lettura da query string: `const voceroEpisodeId = pageParams.get("episode_id")` ([L676-L680](file:///home/duxiter/s2s/s2s-client/main.js#L676-L680)).
|
|
||||||
2. `fetchOpenAiReasoningAnswer` ([L1337-L1364](file:///home/duxiter/s2s/s2s-client/main.js#L1337-L1364)): campo `episode_id` aggiunto al body POST `/api/reasoning`, logKey e `console.info` estesi.
|
|
||||||
3. Costruzione URL WebSocket ([L2323-L2342](file:///home/duxiter/s2s/s2s-client/main.js#L2323-L2342)): `episode_id` appeso a query params sia con API `URL.searchParams` sia nel fallback stringa (concat con `&`).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Aggiunto — `S2S_START_SESSION_MODE`: 3 modalità di start sessione
|
|
||||||
|
|
||||||
Env var letta dal server e propagata al client via `GET /api/config.startSessionMode`.
|
|
||||||
Scopo: governare se e come il microfono si auto-accende e se il modello saluta
|
|
||||||
per primo (flow simile a wake-word). Valori supportati:
|
|
||||||
|
|
||||||
| Valore | Descrizione |
|
|
||||||
|---|---|
|
|
||||||
| *(vuoto) / default* | Start manuale: solo tap sull'Orb / pulsante Start |
|
|
||||||
| **`TALK`** | Dopo 2s overlay fullscreen "Tocca ovunque per iniziare" (policy safe per Safari/iOS `getUserMedia`). Click/touch/tasto dentro la user gesture → `doStart()`. WS client inietta bootstrap `"hola vocero"` + `requestResponse()`. |
|
|
||||||
| **`TALKFIRST`** | Dopo 2s **tenta `doStart()` DIRETTAMENTE SENZA click**, pre-caricando `pendingWakeWordConfirmationText = "hola vocero"` (stessa variabile della wake-word vera). Se i permessi mic sono già "allow always" funziona senza click. Se fallisce (policy Safari/iOS) → fallback sullo stesso unlocker di TALK con messaggio "permesso microfono". |
|
|
||||||
|
|
||||||
**File modificati:**
|
|
||||||
|
|
||||||
- **`server.py`** ([server.py](file:///home/duxiter/s2s/s2s-client/server.py)):
|
|
||||||
1. Lettura env: `S2S_START_SESSION_MODE = os.environ.get(...)`, `.strip().upper()` ([L150](file:///home/duxiter/s2s/s2s-client/server.py#L150)).
|
|
||||||
2. Banner `[STARTUP]` esteso con campo `startSessionMode=` ([L196-L212](file:///home/duxiter/s2s/s2s-client/server.py#L196-L212)).
|
|
||||||
3. Endpoint `GET /api/config` → nuovo campo `startSessionMode` + log `[INTERNAL] /api/config` ([L1383-L1400](file:///home/duxiter/s2s/s2s-client/server.py#L1383-L1400)).
|
|
||||||
|
|
||||||
- **`index.html`** ([index.html](file:///home/duxiter/s2s/s2s-client/index.html#L32-L39)):
|
|
||||||
- Aggiunto nuovo `#talk-unlocker` overlay fullscreen (z-index 9999), `hidden` di default, con card "Preparando Vocero… / Toca en cualquier lugar para empezar a hablar" + spinner gradient.
|
|
||||||
|
|
||||||
- **`style.css`** ([style.css](file:///home/duxiter/s2s/s2s-client/style.css#L145-L215)):
|
|
||||||
- Classi `.talk-unlocker`, `.talk-unlocker-hint`, `.talk-unlocker-spinner`, `.talk-unlocker-title`, `.talk-unlocker-sub` + animazioni fade-in e pop-up card.
|
|
||||||
|
|
||||||
- **`main.js`** ([main.js](file:///home/duxiter/s2s/s2s-client/main.js)):
|
|
||||||
1. Variabile globale `startSessionMode = ""` + commento ([L628-L632](file:///home/duxiter/s2s/s2s-client/main.js#L628-L632)).
|
|
||||||
2. Parse dentro `fetchConfig()` → uppercase, log in `reportDebug("B",…)` ([L2309-L2320](file:///home/duxiter/s2s/s2s-client/main.js#L2309-L2320)).
|
|
||||||
3. Passaggio a WS client → `startWithResponse: startSessionMode === "TALK"` ([L2683-L2692](file:///home/duxiter/s2s/s2s-client/main.js#L2683-L2692)).
|
|
||||||
4. Blocco scheduling auto-start ([L2976-L3083](file:///home/duxiter/s2s/s2s-client/main.js#L2976-L3083)):
|
|
||||||
- `bootstrapWithAutoStart()`: `await fetchConfig()` → `maybeScheduleAutoStart()`;
|
|
||||||
- `setTimeout(2000 ms)` warmup per finire fetch di episode/profile/reports;
|
|
||||||
- `showTalkUnlocker(fallbackHint)`: gestisce label custom nel caso fallback TALKFIRST;
|
|
||||||
- **`tryTalkFirstDirect()` (TALKFIRST)**: set `pendingWakeWordConfirmationText="hola vocero"` + `doStart()` + try/catch con teardown e fallback unlocker;
|
|
||||||
- listener unlocker click/touchstart/keydown → `startTalkFromGesture()` con `createResumedAudioContext()` dentro la gesture (per iOS silent-free).
|
|
||||||
5. `onClientStatus()` ([L2878-L2932](file:///home/duxiter/s2s/s2s-client/main.js#L2878-L2932)) aggiunto `hideTalkUnlocker()` in `connecting/queued/your-turn/connected`.
|
|
||||||
|
|
||||||
- **`ws/s2s-ws-client.js`** ([s2s-ws-client.js](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js)):
|
|
||||||
1. Costruttore: nuove proprietà `_startWithResponse`, `_startWithResponseText` (default `"hola vocero"`), `_initialResponseSent` guardia one-shot ([L163-L178](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L163-L178)).
|
|
||||||
2. Handler `session.updated` ([L688-L713](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L688-L713)): **Solo per `TALK`** (non TALKFIRST, che usa invece `pendingWakeWordConfirmationText`): `this.sendUserText("hola vocero") + this.requestResponse()` — stesso identico code path della wake-word.
|
|
||||||
3. `requestResponse` / `_createResponseNow` ripristinati puliti (solo payload opzionale `image`; nessun shaped `response.create`).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Modificato — `#reports-sidebar-toggle` visibile solo con `?debug=true`
|
|
||||||
|
|
||||||
Il bottone "reports" nella topbar (toggle del pannello laterale reports) è ora
|
|
||||||
`data-debug-only="true"` — viene quindi mostrato solo se l'URL contiene
|
|
||||||
`?debug=true`, analogamente al cluster `.topbar-right`.
|
|
||||||
|
|
||||||
**File modificati:**
|
|
||||||
|
|
||||||
- **`index.html`** → `#reports-sidebar-toggle` ([index.html](file:///home/duxiter/s2s/s2s-client/index.html#L47-L50)): aggiunto `data-debug-only="true"`.
|
|
||||||
- **`main.js`** → `applyDebugOnlyVisibilityGate()` ([L48-L109](file:///home/duxiter/s2s/s2s-client/main.js#L48-L109)): esteso per iterare *tutti* gli elementi `[data-debug-only="true"]` (non solo `.topbar-right`); riga 1227: `reportsSidebarToggle.hidden = false` gated a `qs.get("debug") === "true"` ([L1227-L1246](file:///home/duxiter/s2s/s2s-client/main.js#L1227-L1246)) + log rosso `🚫 FORZATO HIDDEN` quando manca debug=true.
|
|
||||||
- **`style.css`** → regola di sicurezza già presente: `[data-debug-only]:not([data-debug-enabled]) { display:none !important; visibility:hidden; opacity:0 }` ([L532-L537](file:///home/duxiter/s2s/s2s-client/style.css#L532-L537)).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [2026-09-04]
|
|
||||||
|
|
||||||
### Modificato — Chat sempre visibile + integrazione seamless
|
|
||||||
|
|
||||||
Riorganizzato il layout per mostrare la chat testuale in modo permanente e
|
|
||||||
integrato visivamente con l'Orb nella parte superiore dello stage.
|
|
||||||
|
|
||||||
- Rimosso pulsante toggle chat nella topbar (`#chat-btn`) e relativo overlay.
|
|
||||||
- Stage con l'Orb nella parte superiore; nuovo `#chat-panel` a seguire (cronologia
|
|
||||||
+ form input fisso sopra il footer sticky).
|
|
||||||
- Aggiornata logica in `main.js` e `ui/chat.js` per l'invio testo tramite
|
|
||||||
`client.sendUserText` e auto-scroll senza controlli di visibilità pannello.
|
|
||||||
- Versione banner deploy: **2026-09-03 14:40**.
|
|
||||||
|
|
||||||
### Risolto — Stabilità integrazione chat mobile
|
|
||||||
|
|
||||||
- Errore `Missing element: #chat-btn` risolto con selettori non-strict
|
|
||||||
(`document.querySelector`) per elementi opzionali rimossi.
|
|
||||||
- Refactoring estetico "seamless": chat senza cornici, ombre o sfondi
|
|
||||||
(`background:transparent`, `box-shadow:none`).
|
|
||||||
- **Layout mobile:**
|
|
||||||
- padding incrementato in `#app` (56px) e `.chat-history` (96px) per evitare
|
|
||||||
sovrapposizioni col footer sticky;
|
|
||||||
- margini fissi bolle: `margin-right:22px` (Assistant), `margin-left:6px` (User).
|
|
||||||
- **Auto-scroll streaming:** implementato `_scrollToBottom` con catena di retry
|
|
||||||
multipli (`requestAnimationFrame` + `setTimeout`) per compensare il line-wrap
|
|
||||||
asincrono durante l'arrivo dei token; `overflow-anchor:none` + `scroll-behavior:auto !important`.
|
|
||||||
|
|
||||||
### Aggiunto — Avatar utente iniziali + modale profilo
|
|
||||||
|
|
||||||
- Sostituito testo nome utente nell'header con avatar circolare 40×40px
|
|
||||||
(gradient brand) contenente le iniziali calcolate (es. "AV").
|
|
||||||
- Nuova `<dialog id="user-profile-dialog">` in `index.html` che si apre al click
|
|
||||||
sull'avatar: mostra email, telefono, ruolo, azienda, con supporto a shape
|
|
||||||
annidate del payload profilo.
|
|
||||||
- Chiusura modale: tasto ESC, click fuori card, pulsante `×` dedicato.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## [Note di formato]
|
|
||||||
|
|
||||||
- **Cap campo prompt episodio:** 24.000 caratteri (transcript lunghi).
|
|
||||||
- **Cap profilo / summary:** 1.600 caratteri ciascuno.
|
|
||||||
- **Cap reports list:** 12.000 caratteri.
|
|
||||||
- **Ordine inject prompt LLM (WS session.update + reasoning):**
|
|
||||||
1. Episodio podcast corrente (PRIMARIO)
|
|
||||||
2. Profilo chiamante (identity)
|
|
||||||
3. Lista reports / script podcast schedulati
|
|
||||||
4. Summary ultima conversazione
|
|
||||||
- Tutti i blocchi sono marcati con `<<<NOME_BLOCCO_BEGIN/END>>>` per ispezione log.
|
|
||||||
@@ -1,108 +0,0 @@
|
|||||||
# Context glossary
|
|
||||||
|
|
||||||
Canonical terms for this space. A glossary, not a spec — it defines what words mean,
|
|
||||||
not how anything is built. Keep design/implementation detail in `DESIGN.md` and the
|
|
||||||
code.
|
|
||||||
|
|
||||||
## Speech-to-speech demo
|
|
||||||
The product: a voice conversation you have with a model by tapping the orb and
|
|
||||||
talking. "The demo" and "the space" refer to this same thing. It runs on Hugging
|
|
||||||
Face's open `speech-to-speech` backend.
|
|
||||||
|
|
||||||
## The pipeline
|
|
||||||
The ordered path a turn travels, from your voice to the orb's reply. Order is
|
|
||||||
meaningful — each stage consumes the previous one's output:
|
|
||||||
|
|
||||||
`you speak → VAD → STT → VLM → TTS → orb replies`
|
|
||||||
|
|
||||||
- **VAD** — voice activity detection. Decides *when* you are speaking, so the system
|
|
||||||
knows a turn has started and ended. Model: silero-vad.
|
|
||||||
- **STT** — speech to text. Transcribes your speech into words. Model:
|
|
||||||
nvidia/parakeet-tdt-1.1b.
|
|
||||||
- **VLM** — the vision-language model that composes the reply. Served via Cerebras.
|
|
||||||
Model: google/gemma-4-31B-it.
|
|
||||||
- **TTS** — text to speech. Speaks the reply back in the chosen voice. Model:
|
|
||||||
Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice.
|
|
||||||
|
|
||||||
## Builder
|
|
||||||
A Hugging Face user credited with making the space, shown by HF username. Current
|
|
||||||
builders: tfrere, A-Mahla and andito. Distinct from the *models'* authors (nvidia, google,
|
|
||||||
Qwen, snakers4), who are credited per pipeline stage.
|
|
||||||
|
|
||||||
## Powered by
|
|
||||||
The infrastructure running the pipeline, named in the about panel: Hugging Face
|
|
||||||
Inference Endpoints (hosting) and Cerebras (LLM inference). Distinct from "built by"
|
|
||||||
(the people) and from the model authors (who trained each model).
|
|
||||||
|
|
||||||
## Tool
|
|
||||||
A function the model can call mid-conversation to do something the pipeline
|
|
||||||
can't do on its own (look something up, look through the camera). Tools are
|
|
||||||
declared to the backend in the session config; the model decides when to call
|
|
||||||
one. Distinct from the *pipeline stages* (VAD/STT/VLM/TTS), which always run.
|
|
||||||
|
|
||||||
## Tool executor
|
|
||||||
The client-side component that runs a tool when the model calls it and returns
|
|
||||||
the result to the backend, so the model can speak the answer. It is the missing
|
|
||||||
half of the round-trip: the backend already emits the call, the executor runs it
|
|
||||||
and replies. Distinct from the *tool* itself (the thing being run).
|
|
||||||
|
|
||||||
## Web search tool
|
|
||||||
A tool that looks something up on the web for the model. The model calls it with
|
|
||||||
a query; the tool executor forwards the query to the search proxy and returns the
|
|
||||||
results as the tool result. Activates only when a search key is available.
|
|
||||||
|
|
||||||
## Camera snapshot tool
|
|
||||||
A tool that lets the model look through the user's webcam. While enabled, a live
|
|
||||||
self-view is shown in the page (bottom-left); when the model calls the tool, the
|
|
||||||
executor captures a frame and sends it to the model as an image so the VLM can
|
|
||||||
see it. Distinct from the *preview* (what the user sees) and the *snapshot* (the
|
|
||||||
single frame sent to the model).
|
|
||||||
|
|
||||||
## Search proxy
|
|
||||||
The same-origin server route (`/search`) that holds the search key and calls the
|
|
||||||
external search provider on the client's behalf, so the key never reaches the
|
|
||||||
browser. Lives in the same container as the page. Distinct from the *s2s backend*
|
|
||||||
(the separate load-balanced speech-to-speech service).
|
|
||||||
|
|
||||||
## Tools panel
|
|
||||||
The dialog opened from the "Tools" button in the top-right, holding one switch per
|
|
||||||
tool (and the web-search key status). Turning a switch on/off declares or removes
|
|
||||||
that tool on the live session. Distinct from *Settings* (connection, voice,
|
|
||||||
instructions) and the *About panel* (project info).
|
|
||||||
|
|
||||||
## Identity block
|
|
||||||
The top-left corner of the topbar (replacing the old wordmark): the demo name, a
|
|
||||||
one-line blurb, and the "powered by" / "built by" credits, shown directly rather
|
|
||||||
than hidden behind a click.
|
|
||||||
|
|
||||||
## About panel
|
|
||||||
The popup opened from the (i) icon to the right of the identity block. Holds the
|
|
||||||
general introduction to the speech-to-speech project (with a repo link) and the
|
|
||||||
pipeline. Identity itself now lives in the corner, not here.
|
|
||||||
|
|
||||||
## Queue
|
|
||||||
The line of users waiting for a free conversation slot when every compute is
|
|
||||||
busy. You join the queue instead of being turned away; you leave it by reaching
|
|
||||||
the front or by giving up. Distinct from a *session* (an actual live
|
|
||||||
conversation) — being in the queue is not yet talking, and time spent waiting
|
|
||||||
never counts against your usage limit.
|
|
||||||
|
|
||||||
## Ticket
|
|
||||||
Your held place in the queue. Created when you join, it is what the demo checks
|
|
||||||
to tell you your position and to notice if you have left. A ticket is not a
|
|
||||||
session: it only promises a spot in line, not a compute.
|
|
||||||
|
|
||||||
## Position
|
|
||||||
How many people are ahead of you in the queue, shown while you wait ("You're #3
|
|
||||||
in line"). It only ever counts down. Distinct from an *estimated wait* — the
|
|
||||||
demo shows position, never a time, because wait time is unpredictable.
|
|
||||||
|
|
||||||
## Claim
|
|
||||||
The moment you reach the front and a free slot becomes yours — the queue hands
|
|
||||||
off to a real session and the conversation begins. This is also the point where
|
|
||||||
your usage limit first starts to matter (never while waiting).
|
|
||||||
|
|
||||||
## At capacity
|
|
||||||
The state where the queue itself is full, so new users can't even join the line
|
|
||||||
and are asked to try again shortly. Distinct from simply *busy* (all computes
|
|
||||||
taken but the queue still has room to wait in).
|
|
||||||
@@ -1,205 +0,0 @@
|
|||||||
# Design language
|
|
||||||
|
|
||||||
The reference for keeping this app visually coherent as it grows. Read it before
|
|
||||||
touching `style.css`, `index.html`, or any DOM-building code in `main.js`. Every
|
|
||||||
rule here is already live in the codebase — this file explains the *why* so changes
|
|
||||||
extend the system instead of drifting from it.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## The thesis: color belongs to the voice
|
|
||||||
|
|
||||||
This is a voice app. The one thing in the room that should have color is the thing
|
|
||||||
that is talking. So:
|
|
||||||
|
|
||||||
- **The orb** carries saturated color. It glows, and the glow's hue changes with
|
|
||||||
conversational state.
|
|
||||||
- **Everything else is monochrome** — a precise cool-grey dark canvas. Surfaces,
|
|
||||||
borders, buttons, panels, the transcript: all greyscale.
|
|
||||||
- The **only** exceptions are tiny *role echoes* (a one-word mono label, a small
|
|
||||||
icon) that borrow the orb's state hue so the transcript reads in the same color
|
|
||||||
language the orb speaks. They are accents the size of a word, never fills.
|
|
||||||
- **Brand logos keep their own color.** The Hugging Face mark (`#ffd21e`) and the
|
|
||||||
Cerebras mark (`#f15a29`) render in their brand colors in the identity credits and
|
|
||||||
the about panel — a deliberate, owner-approved exception. It applies to those two
|
|
||||||
logos only; do not generalize it to other chrome.
|
|
||||||
|
|
||||||
If you find yourself adding a tinted background, a colored border, or a bright
|
|
||||||
button anywhere outside the orb, stop — that color almost certainly belongs to the
|
|
||||||
orb instead, or shouldn't exist.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Color tokens
|
|
||||||
|
|
||||||
All defined in `:root` in `style.css`. Use the variables, never raw hex in rules.
|
|
||||||
|
|
||||||
### Canvas (the monochrome world)
|
|
||||||
| Token | Value | Use |
|
|
||||||
|---|---|---|
|
|
||||||
| `--bg` | `#0a0b10` | Page background (a cool near-black) |
|
|
||||||
| `--bg-elev` | `#13151c` | Raised surfaces: bubbles, panels, icon buttons |
|
|
||||||
| `--bg-elev-2` | `#1b1e29` | Surfaces on surfaces: history bodies, inputs |
|
|
||||||
| `--border` | `rgba(255,255,255,.08)` | Default hairline |
|
|
||||||
| `--border-strong` | `rgba(255,255,255,.16)` | Emphasised hairline |
|
|
||||||
| `--text` | `#f5f6fa` | Primary text; also the *primary button* fill |
|
|
||||||
| `--text-dim` | `rgba(245,246,250,.65)` | Secondary text |
|
|
||||||
| `--text-faint` | `rgba(245,246,250,.42)` | Captions, labels, footer |
|
|
||||||
|
|
||||||
### Voice (the only saturated hues)
|
|
||||||
These are the orb's state colors. They appear on the orb, and as small role echoes
|
|
||||||
in the transcript — nowhere else.
|
|
||||||
|
|
||||||
| Token | Value | Meaning |
|
|
||||||
|---|---|---|
|
|
||||||
| `--accent` / `--speaking` | `#8b7dff` violet | Assistant speaking |
|
|
||||||
| `--accent-2` / `--listening` | `#22d3ee` cyan | You / listening |
|
|
||||||
| `--processing` | `#f59e0b` amber | Thinking / tool call |
|
|
||||||
| `--error` | `#ff6a75` | Error |
|
|
||||||
| `--success` | `#34d399` | Ready / connected |
|
|
||||||
|
|
||||||
### Role echoes (semantic aliases — use these in chat code)
|
|
||||||
| Token | Maps to | Where it shows |
|
|
||||||
|---|---|---|
|
|
||||||
| `--voice-user` | cyan | `YOU` label + user bubble accents |
|
|
||||||
| `--voice-assistant` | violet | `ASSISTANT` label + assistant accents |
|
|
||||||
| `--voice-tool` | amber | `TOOL CALL` label, wrench icon |
|
|
||||||
|
|
||||||
**Why this mapping:** it mirrors the orb exactly — when *you* speak the orb is cyan
|
|
||||||
(`state-listening`), when the *assistant* speaks it's violet (`state-ai-speaking`),
|
|
||||||
when it's working it's amber (`state-processing`). The transcript is a quiet replay
|
|
||||||
of the orb's color story.
|
|
||||||
|
|
||||||
### Orb state → glow (`.circle.state-*` → `--glow`)
|
|
||||||
| State | Glow |
|
|
||||||
|---|---|
|
|
||||||
| `signed-out` | violet `#8b7dff` |
|
|
||||||
| `authenticated` / `ready` | green `#34d399` |
|
|
||||||
| `connecting` / `connected` / `starting` | yellow `#facc15` |
|
|
||||||
| `listening` / `user-speaking` | cyan (`--listening`) |
|
|
||||||
| `processing` | amber (`--processing`) |
|
|
||||||
| `ai-speaking` | violet (`--speaking`) |
|
|
||||||
| `error` | red (`--error`) |
|
|
||||||
|
|
||||||
Adding a new state? Give it a `--glow`, and if it surfaces in the transcript, add a
|
|
||||||
matching `--voice-*` alias rather than a one-off color.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Typography
|
|
||||||
|
|
||||||
Two faces, two jobs. Never reach for a third.
|
|
||||||
|
|
||||||
- **Inter** — body and UI. Wordmark, buttons, inputs, panel titles, prose, history
|
|
||||||
message bodies. The workhorse; it should feel neutral and get out of the way.
|
|
||||||
- **Geist Mono** (`--font-mono`) — the **machine voice**. Reserved for text the
|
|
||||||
*system* emits or identifiers it reports, never for human prose.
|
|
||||||
|
|
||||||
### When mono is correct
|
|
||||||
Mono signals "this is the machine talking or naming itself." Use it for:
|
|
||||||
- the orb's status caption (`.circle-caption`)
|
|
||||||
- role eyebrows (`YOU` / `ASSISTANT` / `TOOL CALL`)
|
|
||||||
- tool-call names and argument JSON
|
|
||||||
- the `·WebSocket` transport tag, bitrate readouts, connection identifiers
|
|
||||||
- the empty-state label
|
|
||||||
|
|
||||||
Mono text is set uppercase with `letter-spacing: ~0.1–0.14em` and weight `500`, so it
|
|
||||||
reads as a typed status line, not a headline. Body copy, button labels, and
|
|
||||||
explanatory `small` text stay **Inter** — putting prose in mono breaks the metaphor.
|
|
||||||
|
|
||||||
The font is loaded in `index.html`; the stack falls back to system mono gracefully if
|
|
||||||
the CDN is blocked.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Layout
|
|
||||||
|
|
||||||
- **One continuous canvas.** No dividers under the topbar or above the footer, no
|
|
||||||
panel chrome competing with content. The topbar and footer float over the stage.
|
|
||||||
Keep it that way — a new section earns a hairline (`--border`) only if it genuinely
|
|
||||||
needs separating.
|
|
||||||
- **The orb is the hero and the center of gravity.** It sits dead-center on the
|
|
||||||
stage. Controls flank it (mic / stop), captions sit beneath. Don't crowd it.
|
|
||||||
- **Hairlines, not boxes.** Separation comes from `1px` borders at 8–16% white and
|
|
||||||
from spacing, not from heavy fills or shadows. Shadows are soft and low
|
|
||||||
(`0 4px 18px rgba(0,0,0,.32)`), used only to lift floating elements (bubbles,
|
|
||||||
panels, modal).
|
|
||||||
- **Radii:** `--radius-sm: 8px` (buttons, inputs, chips), `--radius-md: 14px`
|
|
||||||
(bubbles, message bodies, modal), `--radius-lg: 22px` (reserved). Pick by element
|
|
||||||
size; don't invent new values.
|
|
||||||
- **Two reading surfaces for the transcript:** ephemeral bubbles top-right (desktop
|
|
||||||
only) that log and fade, and a slide-in history panel for review. On phones the
|
|
||||||
bubble stream is dropped and the panel goes full-screen — the panel is the single
|
|
||||||
source of truth there.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Components
|
|
||||||
|
|
||||||
- **Buttons.** Default (`.btn`) is a neutral elevated surface. The *primary* button
|
|
||||||
is **near-white on dark** (`--text` fill, `--bg` text) — the highest-contrast thing
|
|
||||||
on the page that *isn't* the orb. There is no colored button; emphasis comes from
|
|
||||||
contrast, not hue.
|
|
||||||
- **Icon buttons** (`.icon-btn`) are `36px`, elevated surface, dim icon that brightens
|
|
||||||
on hover. Side controls (`.side-btn`) are circular, collapse to zero size until the
|
|
||||||
session is live (by width on desktop, by height in the mobile column).
|
|
||||||
- **Chat bubbles & history messages** share one neutral surface. They are
|
|
||||||
distinguished by **side** (you = left, assistant = right) plus the **mono role
|
|
||||||
label** in the role-echo hue — not by tinted fills. Tool entries use the wrench icon
|
|
||||||
+ mono + amber, on the same neutral surface.
|
|
||||||
- **Badge** (new-message dot) is monochrome white — a signal, not a color accent.
|
|
||||||
- **Focus** is visible and neutral: inputs focus to `--text-dim`; the orb uses a
|
|
||||||
`--glow`-colored outline (it's the orb, so color is allowed).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Motion
|
|
||||||
|
|
||||||
- **The orb is audio-reactive, not timer-driven.** Mic RMS (`--audio-level`) and the
|
|
||||||
assistant output level (`--ai-audio-level`) drive scale/opacity at display rate, so
|
|
||||||
every syllable moves it. This is the signature animation — keep new motion
|
|
||||||
subordinate to it.
|
|
||||||
- **Quiet by default.** Breathing/glow throbs are slow (1.4–2.4s) and low-contrast.
|
|
||||||
Resist adding scattered micro-animations; an orchestrated moment beats many small
|
|
||||||
ones, and excess motion reads as AI-generated.
|
|
||||||
- **First paint is frozen.** `body.booting` disables all transitions until the first
|
|
||||||
frame commits (stripped after one rAF in `main.js`). Anything new that would
|
|
||||||
otherwise animate-in on load must respect this.
|
|
||||||
- Honor `prefers-reduced-motion` for any motion you add.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Writing / copy voice
|
|
||||||
|
|
||||||
- Sentence case, plain verbs, no filler. Tuned and quiet — match the minimal canvas.
|
|
||||||
- Name things by what the user controls, not by the system's internals. A button says
|
|
||||||
exactly what it does, and keeps the same word through the flow.
|
|
||||||
- **Empty states invite action** ("Tap the orb and start talking"), they don't just
|
|
||||||
set a mood.
|
|
||||||
- **Errors state what happened and how to recover**, in the interface's voice — they
|
|
||||||
don't apologize and are never vague.
|
|
||||||
- Mono labels are terse identifiers (`YOU`, `TOOL CALL`); prose stays in Inter.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Responsive floor (non-negotiable)
|
|
||||||
|
|
||||||
Every change ships meeting these:
|
|
||||||
- Works down to a `360px`-wide phone. The `@media (max-width: 600px)` block already
|
|
||||||
handles the phone layout — extend it, don't fight it.
|
|
||||||
- Visible keyboard focus on every interactive element.
|
|
||||||
- `prefers-reduced-motion` respected.
|
|
||||||
- Tap targets ≥ `44px`; `touch-action: manipulation` on anything tappable.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Before you ship — the mirror check
|
|
||||||
|
|
||||||
1. Is every saturated color either on the orb or a word-sized role echo? If a fill or
|
|
||||||
border is colored, remove that accessory.
|
|
||||||
2. Is mono used only for machine/system text, and Inter for everything human?
|
|
||||||
3. Does any new state have both a `--glow` and (if it appears in chat) a `--voice-*`
|
|
||||||
alias?
|
|
||||||
4. Are separations hairlines + spacing, not boxes and heavy shadows?
|
|
||||||
5. Did you add motion? Is it quieter than the orb and reduced-motion-safe?
|
|
||||||
6. Remove one accessory. The minimal look survives on precision, not addition.
|
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
# Single container: serves the static front-end AND the /api/search proxy.
|
|
||||||
# HF Spaces (sdk: docker) routes traffic to $PORT, default 7860.
|
|
||||||
FROM python:3.11-slim
|
|
||||||
|
|
||||||
WORKDIR /app
|
|
||||||
|
|
||||||
COPY requirements.txt .
|
|
||||||
RUN pip install --no-cache-dir -r requirements.txt
|
|
||||||
|
|
||||||
COPY . .
|
|
||||||
|
|
||||||
EXPOSE 7860
|
|
||||||
|
|
||||||
CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "7860"]
|
|
||||||
@@ -1,584 +0,0 @@
|
|||||||
# HOWITWORKS — demo S2S
|
|
||||||
|
|
||||||
> Compagno di [ARCHITECTURE.md](file:///home/duxiter/s2s/demo/ARCHITECTURE.md): spiega il
|
|
||||||
> ciclo di vita concreto dei dati in 3 moduli su cui oggi ci siamo concentrati:
|
|
||||||
> 1. Il campo `instructions` da client a `ai-server:12345`
|
|
||||||
> 2. `MCP_SEARCH_TRANSLATION_PHRASES` + ranking tool MCP lato client
|
|
||||||
> 3. **FASE 2 — Dichiarazione tools nativi a `ai-server:12345` via Realtime `session.update.tools[]`**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 0. FASE 2 — Integrazione tools nativi S2S upstream (ai-server:12345)
|
|
||||||
|
|
||||||
### Obiettivo
|
|
||||||
Prima della Fase 2, i 100+ tool MCP Aquachile erano solo **testo dentro `instructions`** (9 KB di prompt). L'upstream non sapeva eseguirli e produceva frasi tipo:
|
|
||||||
> *"Perfecto, buscaré… [list_recent_reports]"*
|
|
||||||
|
|
||||||
Con la Fase 2 dichiariamo i tools tramite il **protocollo nativo OpenAI Realtime GA**:
|
|
||||||
```
|
|
||||||
session.update { tools: [...], tool_choice: "auto" }
|
|
||||||
```
|
|
||||||
→ Se `ai-server` lo supporta, produce eventi veri `response.function_call_arguments.done` con nome tool + argomenti JSON. Il listener già esistente nel client (s2s-ws-client.js) si attiva automaticamente e tutto il resto della pipeline — `runTool()` → `POST /api/mcp/tool` Aquachile → `client.sendToolOutput()` — funziona senza altre modifiche.
|
|
||||||
|
|
||||||
### 0.1 Dove è stato implementato lato proxy (s2s_ui container)
|
|
||||||
|
|
||||||
| Modulo | File / Righe | Cosa fa |
|
|
||||||
|---|---|---|
|
|
||||||
| **Cache + normalizzazione** | [server.py:451-544](file:///home/duxiter/s2s/demo/server.py#L451-L544) | `_normalize_tools_for_realtime(raw_tools)`: converte qualsiasi formato tool Aquachile → formato OpenAI Realtime standard. Non alza eccezioni; tool invalidi saltati con warning. |
|
|
||||||
| **Inject WS upstream** | [server.py:1034-1075](file:///home/duxiter/s2s/demo/server.py#L1034-L1075) | Subito dopo `TUNNEL ESTABLISHED`, **PRIMA** che inizi lo streaming audio: fetch cache `_agent_profile_data()` → `session.update` con `instructions` (profilo agente Aquachile) + `tools[]` + `tool_choice=auto`. Errori suppressi: non rompiamo mai il flusso audio. |
|
|
||||||
| **Log di successo KEY** | [server.py:312-334](file:///home/duxiter/s2s/demo/server.py#L312-L334) | Quando arriva `response.function_call_arguments.done` dall'upstream → 2 log `[MCP INJECT] ✓ UPSTREAM ha emesso…` (segnale CHE LA STRATEGIA B FUNZIONA) |
|
|
||||||
| **Proxy MCP già esistente** | [server.py:600-656](file:///home/duxiter/s2s/demo/server.py#L600-L656) | `POST /api/mcp/tool` → Aquachile JSON-RPC 2.0 `tools/call`. Nessuna modifica necessaria. |
|
|
||||||
|
|
||||||
### 0.2 Sequenza temporale WS Realtime (Fase 2)
|
|
||||||
|
|
||||||
```
|
|
||||||
1. Browser → ws://s2s_server/v1/realtime (connessione)
|
|
||||||
2. server.py → websockets.connect(ai-server:12345)
|
|
||||||
3. [MCP INJECT] server.py → upstream:
|
|
||||||
{ "type": "session.update",
|
|
||||||
"session": { // ⚠️ FORMATO CORRETTO — TUTTO dentro session:{}
|
|
||||||
"instructions": "…system prompt profilo Aquachile…",
|
|
||||||
"tools": [ {type: "function", name:"list_recent_reports", …}, …79 più… ],
|
|
||||||
"tool_choice": "auto"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
← LOG "[MCP INJECT] session.update upstream inviato (tools=79, fmt=nested_session OK)"
|
|
||||||
4. browser → ai-server: (turno audio utente) "el reporte de ayer por favor"
|
|
||||||
5. (se upstream ha tools[] in RuntimeConfig.session.tools — ORA SUPPORTATO Nativamente)
|
|
||||||
ai-server → server.py → browser:
|
|
||||||
{ "type": "response.function_call_arguments.done",
|
|
||||||
"call_id": "call_abc",
|
|
||||||
"name": "list_recent_reports",
|
|
||||||
"arguments": "{\"date_from\":\"2026-08-24\",\"limit\":5}" }
|
|
||||||
← LOG "[MCP INJECT] ✓ UPSTREAM ha emesso response.function_call_arguments.done!"
|
|
||||||
6. s2s-ws-client.js:803 → dispatch "toolcall"
|
|
||||||
7. main.js:3023 → runTool("list_recent_reports", args, "call_abc")
|
|
||||||
branch agentToolDefs → execRemoteMcpTool
|
|
||||||
8. execRemoteMcpTool → POST /api/mcp/tool → Aquachile JSON-RPC tools/call
|
|
||||||
LOG: "[Vocero MCP CALL]" → "[Vocero MCP HTTP]" → "[Vocero MCP OUT]"
|
|
||||||
9. client.sendToolOutput("call_abc", "Risultati: 5 report, ultimo=Report_2026-08-24.xlsx")
|
|
||||||
→ WS payload:
|
|
||||||
{ "type": "conversation.item.create",
|
|
||||||
"item": { "type": "function_call_output", "call_id": "call_abc", "output": "..." } }
|
|
||||||
10. ai-server continua il turno con TTS nativo:
|
|
||||||
"Aquí tienes los 5 reportes de ayer, el más reciente es Report_2026-08-24.xlsx..."
|
|
||||||
(nessun [tool_name] nel testo del TTS!)
|
|
||||||
```
|
|
||||||
|
|
||||||
### 0.3. Formato atteso da `ai-server:12345` — guida per modifiche sul backend S2S
|
|
||||||
|
|
||||||
**Se dopo il deploy non vedi il log `[MCP INJECT] ✓ UPSTREAM ha emesso…`**, vuol dire che `ai-server` oggi scarta il campo `tools[]` dal `session.update`. Ecco **esattamente i campi che devono essere supportati** (copia-incollabile nel codice di `ai-server`):
|
|
||||||
|
|
||||||
#### 0.3.1 `session.update` in ingresso (cosa manda oggi `server.py` — FORMATO NESTED CORRETTO)
|
|
||||||
|
|
||||||
⚠️ **REGOLA NON NEGOZIABILE**: TUTTI i campi di configurazione (instructions, tools, tool_choice, voice, model) devono stare **DENTRO l'oggetto `session`**. Non mandarli MAI alla radice del payload. Il validatore Pydantic strict di OpenAI Realtime GA rifiuta qualsiasi campo sconosciuto alla root.
|
|
||||||
|
|
||||||
```jsonc
|
|
||||||
{
|
|
||||||
"type": "session.update",
|
|
||||||
|
|
||||||
// ⚠️ TUTTO dentro "session":{} — il validator Pydantic strict è extra=forbid sulla root.
|
|
||||||
"session": {
|
|
||||||
|
|
||||||
// 1) Istruzioni di sistema (profilo agente Aquachile)
|
|
||||||
"instructions": "María, asistente virtual Aquachile, habla español de Chile…",
|
|
||||||
|
|
||||||
// 2) CATALOGO TOOLS — 79 tool Aquachile normalizzati
|
|
||||||
"tools": [
|
|
||||||
{
|
|
||||||
"type": "function",
|
|
||||||
"name": "list_recent_reports",
|
|
||||||
"description": "Restituisce l'elenco dei report più recenti, opzionalmente filtrati per data.",
|
|
||||||
"parameters": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"date_from": {"type": "string", "format": "date", "description": "Data inizio YYYY-MM-DD"},
|
|
||||||
"limit": {"type": "integer", "minimum": 1, "maximum": 50, "default": 10}
|
|
||||||
},
|
|
||||||
"required": []
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// … altri 78 tool
|
|
||||||
],
|
|
||||||
|
|
||||||
// 3) Strategia selezione tool
|
|
||||||
"tool_choice": "auto"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Perché prima non funzionava?** (ROOT CAUSE)
|
|
||||||
Il proxy stava inviando `instructions`, `tools`, `tool_choice` **ALLA RADICE** del payload, fuori da `session:{}`. Il modello Pydantic `SessionUpdateEvent` (OpenAI Realtime GA) ha `extra=forbid` e ha rifiutato con `Unknown or invalid event: session.update`. **Non era un rifiuto del campo `tools` in quanto tale** — era un rifiuto di qualsiasi campo non riconosciuto messo nel posto sbagliato. La prova: `runtime_config.session.instructions`, `.session.tools`, `.session.tool_choice` in `language_model.py:526-532` leggono TUTTI i campi da dentro `.session.*`.
|
|
||||||
|
|
||||||
**Audit sorgente backend `speech_to_speech` conferma: GIA' 100% WIRED PER NATIVI TOOLS**
|
|
||||||
| Componente | File / Righe | Conferma |
|
|
||||||
|---|---|---|
|
|
||||||
| LLM pick tools da sessione | `LLM/language_model.py:526-532` | `tools = runtime_config.session.tools` |
|
|
||||||
| Chat buffer gestisce FC/FCO | `LLM/chat.py:141-223 / 342-366 / 394-427` | `RealtimeConversationItemFunctionCall` + `FunctionCallOutput` |
|
|
||||||
| Response handler emette WS done event | `api/openai_realtime/handlers/response.py:336-345` | `ResponseFunctionCallArgumentsDoneEvent` con `call_id, name, arguments(STRINGA), response_id, item_id, output_index` |
|
|
||||||
| Conversation handler accetta ritorno output | `api/openai_realtime/handlers/conversation.py:30-96` | `handle_conversation_item_create` con `deferred_items` anti-race |
|
|
||||||
| Fallback parser retrocompatibile | `api/openai_realtime/service.py:267-325` | Se arrivano campi flat alla radice → promossi a nested `session` wrapper |
|
|
||||||
|
|
||||||
**Regole convalida obbligatorie lato `ai-server` per evitare rigetti silenziosi**:
|
|
||||||
|
|
||||||
| Regola | Valore |
|
|
||||||
|---|---|
|
|
||||||
| Nome tool | lunghezza ≤ 64, match regex `^[a-z0-9_]+$` |
|
|
||||||
| Numero max tools | Consigliato ≤ 80. Oggi il proxy manda 100+ con warning. |
|
|
||||||
| `parameters.type` | Sempre `"object"` (anche se vuoto) |
|
|
||||||
| `parameters.properties` | Sempre `{}` presente, anche se 0 proprietà |
|
|
||||||
| `tool_choice` | Valori accettati: `"none"`, `"auto"`, oppure `{type:"function",name:"…"}` |
|
|
||||||
| Chiavi ignote | NON fare strict reject. Logga warning e scarta solo la chiave. |
|
|
||||||
|
|
||||||
#### 0.3.2 Evento di ritorno richiesto da `ai-server` (quando il modello decide di chiamare un tool)
|
|
||||||
|
|
||||||
Quando il modello determina che serve un tool → emetti **3 eventi in ordine ESATTO**:
|
|
||||||
```
|
|
||||||
a) { "type": "response.function_call_arguments.started", "call_id": "call_abc123", "name": "list_recent_reports", "response_id":"resp_X", "item_id":"item_Y" }
|
|
||||||
(opzionale — se non hai streaming arguments puoi anche saltarlo)
|
|
||||||
|
|
||||||
b) (opzionale) N × { "type": "response.function_call_arguments.delta", "call_id": "call_abc123", "delta": "{\"dat" }
|
|
||||||
(solo se streammi il JSON argomenti per pezzi)
|
|
||||||
|
|
||||||
c) ✅ EVENTO RICHIESTO OBBLIGATORIO (senza questo il client non esegue il tool):
|
|
||||||
{
|
|
||||||
"type": "response.function_call_arguments.done",
|
|
||||||
"call_id": "call_abc123", // STRINGA OBBLIGATORIA — serve per la sendToolOutput
|
|
||||||
"name": "list_recent_reports", // STRINGA OBBLIGATORIA — nome del tool (come sopra)
|
|
||||||
"arguments": "{\"date_from\":\"2026-08-24\",\"limit\":5}",
|
|
||||||
"response_id": "resp_xyz",
|
|
||||||
"item_id": "item_zzz",
|
|
||||||
"output_index": 0
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**⚠️ Importante**: `arguments` DEVE essere una **STRINGA JSON serializzata** (non un oggetto). Il client fa `JSON.parse(event.arguments)`. Se mandi un oggetto JS → parser fallisce e il tool non viene eseguito.
|
|
||||||
|
|
||||||
#### 0.3.3 Atteso come input successivo (il ritorno del tool dal client)
|
|
||||||
|
|
||||||
Dopo ~500ms-45s, il client manda indietro al S2S upstream:
|
|
||||||
```jsonc
|
|
||||||
{
|
|
||||||
"type": "conversation.item.create",
|
|
||||||
"item": {
|
|
||||||
"type": "function_call_output",
|
|
||||||
"call_id": "call_abc123",
|
|
||||||
"output": "5 report trovati. Ultimo: Report_2026-08-24.xlsx - Aquachile ventas"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
→ Poi { "type": "response.create" }
|
|
||||||
```
|
|
||||||
→ L'S2S dovrebbe usare `output` come contesto nel prossimo turno di generazione TTS (quindi parla del risultato, senza tag `[tool]`).
|
|
||||||
|
|
||||||
### 0.4 Checklist di deploy Fase 2 (2026-08-25 — SORGENTE ai-server TROVATA in locale)
|
|
||||||
|
|
||||||
**Prerequisito**: Sorgente backend `ai-server:12345` = `/home/duxiter/s2s/speech_to_speech` (Python puro).
|
|
||||||
✅ **TEST CURL 2026-08-25 CONFERMATO**: `https://aquachile.azurianlab.com/api/v1/mcp/` è **ATTUALMENTE PUBBLICO** — accetta `tools/list` e `tools/call` SENZA API KEY. Con credenziali dummy `dummy-test-key/secret` restituisce HTTP 200 OUTPUT IDENTICO alla chiamata senza auth. Quindi NON serve valorizzare:
|
|
||||||
```
|
|
||||||
S2S_TOOL_EXECUTION_MCP_API_KEY=
|
|
||||||
S2S_TOOL_EXECUTION_MCP_API_SECRET=
|
|
||||||
```
|
|
||||||
⚠️ Nota: Il server accetta entrambi gli auth headers, ma attualmente non li valida. Restano configurati come best practice per eventuali versioni future dell'endpoint che attiveranno auth.
|
|
||||||
|
|
||||||
#### Ordine deploy 3 step
|
|
||||||
|
|
||||||
| # | Azione | Perché |
|
|
||||||
|---|---|---|
|
|
||||||
| 0 | ~~Valorizza `.env` MCP credentials~~ | ✅ **SUPERATO** — endpoint MCP Aquachile pubblico |
|
|
||||||
| 1 | **Restart container `s2s_ui`** | Deploy fix formato inject `fmt=nested_session` in [server.py:1054-1102](file:///home/duxiter/s2s/demo/server.py#L1054-L1102) |
|
|
||||||
| 2 | **Riavvia backend `ai-server:12345`** con il codice aggiornato di `/home/duxiter/s2s/speech_to_speech` (in particolare `api/openai_realtime/service.py`) | Deploy fallback parser retrocompatibile flat→nested in [service.py:267-325](file:///home/duxiter/s2s/speech_to_speech/api/openai_realtime/service.py#L267-L325) |
|
|
||||||
| 3 | **Hard Reload browser** (Ctrl+Shift+R o Disable cache + refresh) | Pulisce cache vecchio client JS |
|
|
||||||
|
|
||||||
#### Checklist di verifica DOPO deploy 3+1 step
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Terminal A
|
|
||||||
docker logs -f s2s_ui 2>&1 | grep -E 'MCP|Vocero MCP|session.update|function_call_arguments'
|
|
||||||
```
|
|
||||||
|
|
||||||
| Step | Trigger | Log atteso nel container | Esito |
|
|
||||||
|---|---|---|---|
|
|
||||||
| 1 | Apri pagina demo (connessione WS) | `[MCP INJECT] session.update upstream → tools=79 hasAgentPrompt=True tool_choice=auto` | ✅ Inject partito |
|
|
||||||
| 2 | Subito dopo step 1 | `[MCP INJECT] session.update upstream inviato (tools=79, fmt=nested_session OK)`<br>🔴 **NON DEVE** più comparire `[MCP INJECT] suppressed upstream error` — se compare ancora, il deploy di `speech_to_speech service.py` non è riuscito | ✅ Nessun rigetto |
|
|
||||||
| 3 | Console browser | 0 occorrenze `Unknown or invalid event: session.update` | ✅ Filtro OK |
|
|
||||||
| 4 | Frase trigger tool: *"¿dónde está el report de ayer?"* | **KEY LOG 🔑**<br>`[MCP INJECT] ✓ UPSTREAM ha emesso response.function_call_arguments.done! tool=list_recent_reports args_keys=[...]` | ✅ **Strategia B FUNZIONA — il modello produce tool calls native** |
|
|
||||||
| 5 | 0.5-1s dopo Step 4 | `[Vocero MCP CALL] name=list_recent_reports target=https://aquachile...`<br>`[Vocero MCP HTTP] name=list_recent_reports status=200`<br>`[Vocero MCP OUT] name=list_recent_reports preview=...resultKeys=...` | ✅ Tool eseguito realmente via REST Aquachile |
|
|
||||||
| 6 | 1-2s dopo Step 5 | TTS vocale naturale — es: *"Aquí tienes los 5 reportes de ayer, el más reciente es Report_2026-08-24.xlsx..."* | ✅ Nessun `(list_recent_reports)` / `[tool]` nel testo TTS |
|
|
||||||
| 7 | Frase non-tool: *"Hola, ¿cómo estás?"* | Risposta standard SENZA tool calls, POST /api/reasoning 200, Node-RED LOG_ONLY OK | ✅ Nessuna regressione Fase 1 |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 0.5 Stato attuale (2026-08-25 11:00) — BLOCCO RISOLTO, pipeline 100% implementata
|
|
||||||
|
|
||||||
#### 🎯 Root Causa Reale (99.99% confidence, sorprendente)
|
|
||||||
|
|
||||||
**Ipotesi iniziale, SUPERATA**: *"ai-server rifiuta strict il campo `tools` perché non è implementato"*
|
|
||||||
|
|
||||||
**Realtà, AUDIT SORGENTE CONFERMATO**: TUTTO il sistema — **backend `speech_to_speech` + proxy `demo/server.py` + client `main.js` / `s2s-ws-client.js`** — è GIA' COMPLETAMENTE IMPLEMENTATO end-to-end per il tool calling nativo OpenAI Realtime GA. **L'unico blocco era un errore di formato payload nel proxy**:
|
|
||||||
|
|
||||||
```python
|
|
||||||
# ❌ SBAGLIATO (come era prima) — campi alla RADICE
|
|
||||||
_inject_payload = {
|
|
||||||
"type": "session.update",
|
|
||||||
"instructions": "...",
|
|
||||||
"tools": [...],
|
|
||||||
"tool_choice": "auto"
|
|
||||||
} # ← Pydantic SessionUpdateEvent extra=forbid → Unknown or invalid event
|
|
||||||
|
|
||||||
# ✅ CORRETTO (deployato in server.py:1054-1102) — TUTTO dentro session:{}
|
|
||||||
_inject_payload = {
|
|
||||||
"type": "session.update",
|
|
||||||
"session": {
|
|
||||||
"instructions": "...",
|
|
||||||
"tools": [...],
|
|
||||||
"tool_choice": "auto"
|
|
||||||
}
|
|
||||||
} # ← Perfetto, corrisponde a runtime_config.session.tools in LLM
|
|
||||||
```
|
|
||||||
|
|
||||||
**Prova inconfutabile**: [language_model.py:526-532](file:///home/duxiter/s2s/speech_to_speech/LLM/language_model.py#L526-L532) legge istruzioni, tools e tool_choice **TUTTI dentro `.session.*`**:
|
|
||||||
```python
|
|
||||||
instructions = response.instructions if ... else runtime_config.session.instructions
|
|
||||||
tools = response.tools if ... else runtime_config.session.tools
|
|
||||||
tool_choice = response.tool_choice if ... else runtime_config.session.tool_choice
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 📋 Mappa riassuntiva: 7 componenti TUTTI già funzionanti
|
|
||||||
|
|
||||||
| Componente | File | Stato | Cosa fa |
|
|
||||||
|---|---|---|---|
|
|
||||||
| 🔧 **Normalizzatore tool** | `demo/server.py:464-544` | ✅ | 79 tool Aquachile → formato Realtime GA standard, 0 eccezioni |
|
|
||||||
| 🚀 **Inject WS upstream** | `demo/server.py:1054-1102` | ✅ **NUOVO FIX 2026-08-25** | Invia nested `session:{instructions,tools,tool_choice}` subito dopo tunnel |
|
|
||||||
| 🛡️ **Fallback parser retrocompatibile** | `speech_to_speech/service.py:267-325` | ✅ **NUOVO FIX 2026-08-25** | Se arrivano campi flat root → promossi a nested `session` |
|
|
||||||
| 🧠 **LLM + Chat buffer** | `speech_to_speech/LLM/*` | ✅ GIA' ESISTENTE | Tools da sessione, FC/FCO nel chat, pending dict anti-race |
|
|
||||||
| 📡 **WS response handler (tool call OUT)** | `speech_to_speech/handlers/response.py:336-345` | ✅ GIA' ESISTENTE | Emette `ResponseFunctionCallArgumentsDoneEvent` con arguments=STRINGA JSON, call_id, name, response_id, item_id, output_index=0 |
|
|
||||||
| 💾 **WS conversation handler (tool output IN)** | `speech_to_speech/handlers/conversation.py:30-96` | ✅ GIA' ESISTENTE | `conversation.item.create function_call_output` con `deferred_items` anti-race |
|
|
||||||
| 🌐 **Client WS → MCP REST → ritorno WS** | `demo/ws/s2s-ws-client.js:803-1077` + `demo/main.js:2180-2288 + 3023-3043` | ✅ GIA' ESISTENTE | (1) `response.function_call_arguments.done` → dispatch `toolcall` event<br>(2) `runTool()` branch `agentToolDefs` → `execRemoteMcpTool()` → `POST /api/mcp/tool`<br>(3) `client.sendToolOutput(callId, output)` → `conversation.item.create function_call_output`<br>(4) `client.requestResponse()` → prossimo turno TTS |
|
|
||||||
| 📣 **Proxy MCP REST Aquachile** | `demo/server.py:704-760` | ✅ GIA' ESISTENTE | `POST /api/mcp/tool` JSON-RPC 2.0 tools/call + log `[Vocero MCP CALL/HTTP/OUT]` |
|
|
||||||
| 🛡️ **Filtro errori WS V2** | `demo/server.py:1713-1792` | ✅ GIA' ESISTENTE | Sopprime `unknown_or_invalid_event` se mai dovesse ricomparire, passthrough altri errori |
|
|
||||||
|
|
||||||
#### 🚀 Prossimo passo: ESEGUIRE I 3 STEP DI §0.4
|
|
||||||
|
|
||||||
1. ~~Valorizza `.env` MCP API Key/Secret~~ ✅ **SUPERATO** (endpoint pubblico)
|
|
||||||
2. Restart `s2s_ui`
|
|
||||||
3. Restart `ai-server` con il codice aggiornato da `/home/duxiter/s2s/speech_to_speech`
|
|
||||||
4. Hard reload browser (Ctrl+Shift+R)
|
|
||||||
5. Testa: *"¿dónde está el report de ayer?"* → aspetta il key log `[MCP INJECT] ✓ UPSTREAM ha emesso response.function_call_arguments.done!`
|
|
||||||
|
|
||||||
**Se Step 5 (function_call_arguments.done) non compare dopo 3 frasi trigger**: apri una shell con `docker logs -f ai-server` per vedere se la sessione sta leggendo `.session.tools` dal runtime config.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
#### ⚙️ MCP MASTER SWITCH (nuovo feature 2026-08-25)
|
|
||||||
**Controllato da una sola riga in `.env`**:
|
|
||||||
```
|
|
||||||
#S2S_TOOL_EXECUTION_MCP_URL=https://aquachile.azurianlab.com/api/v1/mcp # ⬅️ se commentata/vuota → MCP DISABLED
|
|
||||||
S2S_TOOL_EXECUTION_MCP_URL=https://aquachile.azurianlab.com/api/v1/mcp # ⬅️ se decommentata/valorizzata → MCP ENABLED
|
|
||||||
```
|
|
||||||
|
|
||||||
Implementato come `MCP_EXEC_ENABLED = bool(S2S_TOOL_EXECUTION_MCP_URL)` in [server.py:106-114](file:///home/duxiter/s2s/demo/server.py#L106-L114). Quando **DISABLED** (stato attuale `.env`):
|
|
||||||
|
|
||||||
| Punto di controllo | Comportamento con MCP DISABLED | Riferimento codice |
|
|
||||||
|---|---|---|
|
|
||||||
| **Startup log** | `[MCP MASTER] ⛔ DISABLED — S2S_TOOL_EXECUTION_MCP_URL is empty/commented in .env…` | [server.py:172-182](file:///home/duxiter/s2s/demo/server.py#L172-L182) |
|
|
||||||
| **WS `/v1/realtime` inject tools** | `session.update` contiene **solo `instructions`** (profilo agente); `tools[]` e `tool_choice` **non vengono mai mandati**. Log marker `[MCP INJECT] ⛔ SKIPPED tool injection …`. ⚠️ Il prompt profilo continua a funzionare. | [server.py:1081-1123](file:///home/duxiter/s2s/demo/server.py#L1081-L1123) |
|
|
||||||
| **`/api/config` bootstrap client** | Invia al browser `agentTools=[]` (invece dei 79) + `searchMcp=false`. Effetti lato client: ranker MCP early exit, prompt non contiene più "Available MCP tools:" e ranking hints, UI pannello MCP mostra "No MCP tools available.", nessun pulsante "Esegui tool". | [server.py:660-694](file:///home/duxiter/s2s/demo/server.py#L660-L694) |
|
|
||||||
| **`POST /api/mcp/tool` endpoint** | Se un client tenta comunque la chiamata (es. con codice vecchio): log `[MCP DISABLED] ⛔ /api/mcp/tool called but…` + HTTP 503 dettagliato (non crash, non exception trace). | [server.py:721-733](file:///home/duxiter/s2s/demo/server.py#L721-L733) |
|
|
||||||
|
|
||||||
Per **riabilitare la Fase 2 MCP in futuro**, 4 step:
|
|
||||||
1. ✏️ `.env` → togli `#` davanti a `S2S_TOOL_EXECUTION_MCP_URL=https://aquachile.azurianlab.com/api/v1/mcp`
|
|
||||||
2. 🔄 Restart `s2s_ui` (deploya MCP_EXEC_ENABLED=True + inject formato nested corretto)
|
|
||||||
3. 🔄 Restart `ai-server` con codice aggiornato `/home/duxiter/s2s/speech_to_speech` (fallback parser flat→nested)
|
|
||||||
4. 🌐 Hard reload browser Ctrl+Shift+R
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Flow di `instructions` — da impostazioni utente a S2S upstream
|
|
||||||
|
|
||||||
**Definizione**: `instructions` è il **system prompt** del protocollo OpenAI Realtime GA.
|
|
||||||
Viene mandato al backend in 2 punti distinti:
|
|
||||||
|
|
||||||
| Canale | Dove | Destinatario |
|
|
||||||
|---|---|---|
|
|
||||||
| ✅ **TUNNEL WS AUDIO** → `session.update` all'inizio di ogni sessione | Browser → tunnel proxy → upstream S2S nativo (`ai-server:12345`) | Usato **dal modello S2S stesso** per la risposta vocale nativa |
|
|
||||||
| ✅ **REST PARALLELO** → body `POST /api/reasoning` | Browser → `server.py` → Node-RED o OpenAI | Usato **dal reasoning hook** esterno (oggi Fase 1 LOG_ONLY) |
|
|
||||||
|
|
||||||
Entrambi i canali partono dalla stessa funzione generatrice `effectiveReasoningInstructions()` nel client.
|
|
||||||
|
|
||||||
### Step-by-step
|
|
||||||
|
|
||||||
```
|
|
||||||
┌────────────────────────────────────────────────────────────────────┐
|
|
||||||
│ 1. BASE — DEFAULT_INSTRUCTIONS (main.js:26) │
|
|
||||||
│ "You are a friendly voice assistant. Keep replies short..." │
|
|
||||||
│ OR textarea Settings #instructions salvato in localStorage │
|
|
||||||
│ chiave "s2s.ws.instructions" (main.js:138, 201, 223) │
|
|
||||||
└───────────────────────────────────────┬────────────────────────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌────────────────────────────────────────────────────────────────────┐
|
|
||||||
│ 2. effectiveReasoningInstructions() (main.js:516-533) │
|
|
||||||
│ Appende in coda, in ordine: │
|
|
||||||
│ │
|
|
||||||
│ + (se ≥1 tool attivo) TOOL_USE_HINT (main.js:32-35) │
|
|
||||||
│ → "do not describe your capabilities... call tool immediately" │
|
|
||||||
│ │
|
|
||||||
│ + (se searchMcpEnabled + agentToolDefs) MCP_SEARCH_HINT + │
|
|
||||||
│ "Available MCP tools: …catalogo completo…" │
|
|
||||||
│ │
|
|
||||||
│ + (se searchMcpEnabled + tool suggeriti) riga ranking client-side │
|
|
||||||
│ "Current user request: \"${lastUserQuery}\". Ranking hint: 1. X │
|
|
||||||
│ 2. Y 3. Z — use as secondary signal only" │
|
|
||||||
│ │
|
|
||||||
│ + (se utente autenticato phone) voceroUserContextText() │
|
|
||||||
│ → es: "User profile: name=Juan Pérez phone=+569… account=…" │
|
|
||||||
└───────────────────────────────────────┬────────────────────────────┘
|
|
||||||
│
|
|
||||||
┌──────────────────┴──────────────────┐
|
|
||||||
▼ ▼
|
|
||||||
┌─────────────────────────────┐ ┌──────────────────────────────┐
|
|
||||||
│ 3a. effectiveInstructions() │ │ 3b. POST /api/reasoning │
|
|
||||||
│ (main.js:537-539) │ │ body req.instructions │
|
|
||||||
│ │ │ (main.js:569) │
|
|
||||||
│ • mode=OPENAI → accoda │ │ = effectiveReasoningInst…() │
|
|
||||||
│ OPENAI_RENDER_HINT │ │ tale e quale (NO append) │
|
|
||||||
│ • mode=NODE_RED / PROXY → │ │ │
|
|
||||||
│ ritorna il valore │ │ → server.py lo gira a │
|
|
||||||
│ senza modifiche │ │ Node-RED o OpenAI │
|
|
||||||
└──────────────┬──────────────┘ └──────────────┬───────────────┘
|
|
||||||
│ │
|
|
||||||
▼ ▼
|
|
||||||
┌─────────────────────────────┐ ┌──────────────────────────────┐
|
|
||||||
│ 4a. startInstructions = │ │ 4b. /api/reasoning → │
|
|
||||||
│ effectiveInstructions() │ │ Node-RED log: │
|
|
||||||
│ (main.js:2829) │ │ [S2S NODE_RED LOG_ONLY] │
|
|
||||||
└──────────────┬──────────────┘ │ instructions len=… │
|
|
||||||
│ └──────────────────────────────┘
|
|
||||||
▼
|
|
||||||
┌─────────────────────────────────────────────────────┐
|
|
||||||
│ 5. new S2sWsRealtimeClient({ │
|
|
||||||
│ instructions: startInstructions │
|
|
||||||
│ voice, tools, noiseGate, audioContext │
|
|
||||||
│ }) main.js:2839-2847 │
|
|
||||||
└──────────────┬──────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌─────────────────────────────────────────────────────┐
|
|
||||||
│ 6. this.options.instructions → _sendSessionUpdate() │
|
|
||||||
│ s2s-ws-client.js:995 + 1016 │
|
|
||||||
│ { type: "session.update", │
|
|
||||||
│ session: { type: "realtime", │
|
|
||||||
│ instructions: this.options.…, │
|
|
||||||
│ audio:{output:{voice}}, │
|
|
||||||
│ tools, tool_choice:"auto" } } │
|
|
||||||
└──────────────┬──────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌─────────────────────────────────────────────────────┐
|
|
||||||
│ 7. server.py _client_to_upstream — TUNNEL 1:1 │
|
|
||||||
│ → forward esatto byte-per-byte all'upstream │
|
|
||||||
│ → server.py NON tocca MAI instructions qui │
|
|
||||||
│ → Logga solo summary.instructionLength = len() │
|
|
||||||
│ (server.py:197-203) — NON MODIFICA │
|
|
||||||
└──────────────┬──────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌─────────────────────────────────────────────────────┐
|
|
||||||
│ 8. ai-server:12345 riceve session.update → │
|
|
||||||
│ istruzioni usate come system prompt da VLM/LMM │
|
|
||||||
│ per TUTTA la durata della sessione finché │
|
|
||||||
│ non arriva un altro session.update (patch) │
|
|
||||||
└─────────────────────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
### Patch di `instructions` durante la sessione (viva)
|
|
||||||
Se vuoi aggiornare istruzioni a metà conversazione **senza chiudere la WS**:
|
|
||||||
- Client → chiama `client.updateSession({ instructions })` (main.js:1021-1035)
|
|
||||||
- Oppure, in futuro: **proxy server-side inietta patch** via `upstream.send({type:"session.update", session:{...}})` in qualsiasi momento del tunnel
|
|
||||||
|
|
||||||
### Tabella riassuntiva — chi oggi modifica `instructions`?
|
|
||||||
|
|
||||||
| Livello | Modifica? | Note |
|
|
||||||
|---|---|---|
|
|
||||||
| 🔵 Browser `main.js` | **Sì** — generatore principale | DEFAULT_INSTRUCTIONS + hints + user context |
|
|
||||||
| 🟣 Browser `s2s-ws-client.js` | **Solo forwarding** | Incapsula in `session.update` |
|
|
||||||
| 🟢 Proxy `server.py` tunnel WS | **NO** — copia byte-per-byte | Logga solo lunghezza, non tocca |
|
|
||||||
| 🟠 Proxy `server.py` reasoning REST | **Sì (Modalità B)** | Usa `req.instructions` come system part nel prompt OpenAI (Fase 2) |
|
|
||||||
| 🔴 Upstream `ai-server:12345` | **Sì** — applica come system prompt | Entra nel ciclo STT→VLM→TTS |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Ranking tool MCP lato client + `MCP_SEARCH_TRANSLATION_PHRASES`
|
|
||||||
|
|
||||||
**Obiettivo**: quando arriva la query utente (es. `"Quiero el reporte de hoy"` in spagnolo),
|
|
||||||
ordinare velocemente i tool MCP del catalogo Aquachile per **rilevanza**, così
|
|
||||||
le `instructions` possono includere un hint di ranking ("prova prima questi 3") e
|
|
||||||
il modello sceglie lo strumento giusto con meno tentativi.
|
|
||||||
|
|
||||||
### 2.1 Perché serve la traduzione
|
|
||||||
Il catalogo dei tool `agentToolDefs` arriva dall'endpoint Aquachile MCP:
|
|
||||||
- `tool.name` e `tool.description` sono scritti in **INGLESE** per convenzione di
|
|
||||||
OpenAI Functions / MCP protocol.
|
|
||||||
- Ma l'utente parla **spagnolo del Cile o italiano** con l'agente.
|
|
||||||
|
|
||||||
Senza traduzione il matching `frase ES/IT ↔ tool EN` → **score 0** (zero corrispondenze).
|
|
||||||
I tool non vengono mai suggeriti.
|
|
||||||
|
|
||||||
### 2.2 I due dizionari di traduzione (main.js)
|
|
||||||
|
|
||||||
| Costante | Tipo | Riga | Cosa fa |
|
|
||||||
|---|---|---|---|
|
|
||||||
| `MCP_SEARCH_TRANSLATION_PHRASES` | `[RegExp, string][]` frasi complete | [L51-L70](file:///home/duxiter/s2s/demo/main.js#L51-L70) | Sostituisce **frasi multi-parola** prima del tokenizer. Es. `reporte de hoy → today report`. **Priorità alta**: viene applicato PER PRIMO perché le frasi intere non si possono ricostruire da token singoli. |
|
|
||||||
| `MCP_SEARCH_TRANSLATION_TOKENS` | `Record<it/es, en>` token singolo | [L72](file:///home/duxiter/s2s/demo/main.js#L72) | Sostituisce il **resto** parola per parola dopo la phase-phrase. Es. `ayuda → help`, `cliente → customer`. |
|
|
||||||
|
|
||||||
### 2.3 Catalogo completo traduzioni oggi attivo
|
|
||||||
|
|
||||||
#### Phrase-level (MCP_SEARCH_TRANSLATION_PHRASES)
|
|
||||||
| Lingua | Input | Output EN | Riga |
|
|
||||||
|---|---|---|---|
|
|
||||||
| 🇪🇸 | `informaciones del usuario` / `informacion del usuario` | `user information` | L52-L53 |
|
|
||||||
| 🇮🇹 | `informazioni utente` / `informazione utente` | `user information` | L54-L55 |
|
|
||||||
| 🇪🇸 | `report del dia` / `reporte del dia` | `day report` | L56-L57 |
|
|
||||||
| 🇮🇹 | `report di oggi` | `today report` | L58 |
|
|
||||||
| 🇪🇸 | `reporte de hoy` | `today report` | L59 |
|
|
||||||
| 🇮🇹 | `report giornaliero` | `daily report` | L60 |
|
|
||||||
| 🇪🇸 | `reporte diario` | `daily report` | L61 |
|
|
||||||
| 🇪🇸 | `mi numero es` | `my number is` | L62 |
|
|
||||||
| 🇮🇹 | `il mio numero e` | `my number is` | L63 |
|
|
||||||
| 🇮🇹 | `numero di telefono` | `phone number` | L64 |
|
|
||||||
| 🇪🇸 | `numero telefonico` | `phone number` | L65 |
|
|
||||||
| 🇪🇸 | `numero de telefono` | `phone number` | L66 |
|
|
||||||
| 🇪🇸 | `numero de whatsapp` | `whatsapp number` | L67 |
|
|
||||||
| 🇪🇸 | `informacion api` | `api information` | L68 |
|
|
||||||
| 🇮🇹 | `informazioni api` | `api information` | L69 |
|
|
||||||
|
|
||||||
> 💡 Aggiungere nuove frasi: **è sufficiente aggiungere righe qui**, senza toccare il codice del ranking. La pipeline le applicherà automaticamente.
|
|
||||||
|
|
||||||
### 2.4 Pipeline di ranking chiamata per ogni turno
|
|
||||||
|
|
||||||
```
|
|
||||||
Utente dice: "Quiero el reporte diario y informaciones del usuario"
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
rankAgentTools(query) main.js:1262
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
buildMcpSearchTexts(query) main.js:1330
|
|
||||||
│
|
|
||||||
├─── [0] normalizeMcpSearchText(query) ← versione ORIGINALE normalizzata
|
|
||||||
│ = "quiero el reporte diario y informaciones del usuario"
|
|
||||||
│
|
|
||||||
└─── [1] translateMcpSearchTextToEnglish(query) main.js:1314
|
|
||||||
│
|
|
||||||
├─ step 1: normalize
|
|
||||||
├─ step 2: loop replace per tutte le entry di MCP_SEARCH_TRANSLATION_PHRASES
|
|
||||||
│ → "quiero el daily report y user information"
|
|
||||||
├─ step 3: split token, map per MCP_SEARCH_TRANSLATION_TOKENS singoli
|
|
||||||
│ → "want the daily report and user information"
|
|
||||||
└─ step 4: normalize finale
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
searchTexts = [ originale normalizzato, versione tradotta in EN ]
|
|
||||||
(duplicati eliminati via new Set)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
tokens = split searchTexts + lunghezza ≥ 3 chars + dedup
|
|
||||||
= [ quiero, reporte, diario, informaciones, usuario,
|
|
||||||
want, daily, report, and, user, information ]
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
Check rule hard-coded:
|
|
||||||
dailyReportIntent = match ("day"|"daily"|"today") + "report" → TRUE
|
|
||||||
(se match: boost score dei tool che hanno report in name/description)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
Per ogni tool in agentToolDefs[]:
|
|
||||||
score = somma (frequenza token in tool.name + tool.description)
|
|
||||||
+ 2× bonus dailyReportIntent se tool matcha "report"
|
|
||||||
+ 3× bonus se esatta match name vs query originale
|
|
||||||
Sort(score desc, name asc)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
Torna al chiamante: [ {tool:"get_daily_report", score:42},
|
|
||||||
{tool:"get_user_info", score:31},
|
|
||||||
... ]
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
Usato in 2 punti:
|
|
||||||
A) effectiveReasoningInstructions() main.js:525-529 → hint "Current user request…
|
|
||||||
ranking hint: 1. get_daily_report… 2. get_user_info…" dentro le instructions
|
|
||||||
B) UI pannello (se visibile) — lista tool suggeriti accanto alla trascrizione
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2.5 Quando è chiamato `rankAgentTools` — ogni turno?
|
|
||||||
|
|
||||||
| Trigger | Frequenza | Perché |
|
|
||||||
|---|---|---|
|
|
||||||
| `maybeRunOpenAiReasoning(transcript)` subito dopo transcript STT | 1× per turno utente | Include ranking hint dentro `instructions` che andrà a Node-RED / OpenAI |
|
|
||||||
| Live edit textarea domanda (autocomplete) | Mentre utente scrive | Suggerisce tool nella UI |
|
|
||||||
|
|
||||||
### 2.6 Disattivazione automatica
|
|
||||||
|
|
||||||
Ranking ritorna `[]` vuoto e salta tutta la traduzione se:
|
|
||||||
- `searchMcpEnabled == false` (MCP Aquachile non configurato)
|
|
||||||
- `agentToolDefs.length == 0` (catalogo tool non ricevuto da `/agents/profile`)
|
|
||||||
- Query normalizzata è vuota o troppo corta
|
|
||||||
- Nessun token dopo split ≥ 3 caratteri
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2bis. DOVE VEDERE I LOG MCP — due postazioni
|
|
||||||
|
|
||||||
Il ranking MCP produce log in **2 posti diversi** per design:
|
|
||||||
|
|
||||||
| Postazione | Tipo di log | Filtro DevTools / grep | Quando appare |
|
|
||||||
|---|---|---|---|
|
|
||||||
| 🖥️ **Browser — DevTools Console** | Client-side ranking dettagliato: **tutta la pipeline translate → build → rank per-tool matching** | `MCP RANK` (case-sensitive) | **Sempre** per ogni turno — anche in caso di EARLY EXIT (motivo esplicito: `searchMcpEnabled=false`, `agentToolDefs=0`, ecc.) |
|
|
||||||
| 🐳 **Server — container `s2s_ui` (uvicorn Python)** | Server-side **snapshot del payload `instructions`** quando arriva `POST /api/reasoning` — quanti tool MCP ci sono nel catalogo hint, TOP3 suggeriti dal ranking lato client | `grep '[MCP RANK]'` o filter terminale `MCP RANK` | **1 riga per ogni POST /api/reasoning** (lo hai già nel flusso tra `[INPUT UTENTE]` e `[EXT][NODE_RED] reasoning POST`) |
|
|
||||||
|
|
||||||
### Nel tuo log di oggi (FASE 1 LOG_ONLY, senza MCP tools abilitati)
|
|
||||||
|
|
||||||
Aspettati questo flusso dopo il deploy di adesso:
|
|
||||||
|
|
||||||
```
|
|
||||||
7|s2s_ui | INFO [s2s] [INPUT UTENTE] client=… stt=Necesito el report.
|
|
||||||
7|s2s_ui | INFO [s2s] [MCP RANK] reasoning payload transcript="Necesito el report" instructionsLen=97 hasMcpCatalogHint=False hasRankingHint=False mcpCatalogTools=0 top3Suggested=none
|
|
||||||
7|s2s_ui | DEBUG [s2s] [EXT][NODE_RED] reasoning POST http://localhost:1881/api/reasoning …
|
|
||||||
```
|
|
||||||
|
|
||||||
E se apri **DevTools Console** (Verbose/Debug on, Disable cache ✓ Ctrl+Shift+R) vedi:
|
|
||||||
|
|
||||||
```
|
|
||||||
[MCP RANK] updateMcpToolSuggestions ENTRY partial=false itemId=- query="necesito el report" searchMcpEnabled=false
|
|
||||||
[MCP RANK] ════════════════════════════════════════
|
|
||||||
[MCP RANK] rankAgentTools START query="necesito el report" searchMcpEnabled=false agentToolDefsCount=0
|
|
||||||
[MCP RANK] buildSearchTexts raw="Necesito el report" normalized="necesito el report" translated="necesito el report" count=1 final=["necesito el report"]
|
|
||||||
[MCP RANK] translate input="necesito el report" phrasesApplied=0 tokensApplied=0 output="necesito el report"
|
|
||||||
[MCP RANK] → EARLY EXIT reason="searchMcpEnabled=false — da /api/config: MCP Aquachile feature OFF oppure S2S_TOOL_EXECUTION_MCP_URL non configurato; agentToolDefs=0 — catalogo tool MCP vuoto: profilo agente non ha tools in agentProfile via Aquachile /api/agents/profile" searchMcpEnabled=false normalized=true agentToolDefs=0
|
|
||||||
[MCP RANK] ════════════════════════════════════════ (empty result)
|
|
||||||
[MCP RANK] updateMcpToolSuggestions RESULTS partial=false matchedTools=0 ranked=none
|
|
||||||
```
|
|
||||||
|
|
||||||
### Quando il ranking MCP inizia a produrre score > 0
|
|
||||||
|
|
||||||
Affinché un turno abbia log MCP con tool score>0, devono essere veri **TUTTI e 3**:
|
|
||||||
|
|
||||||
1. `.env` ha `S2S_TOOL_EXECUTION_MCP_URL` configurato (URL Aquachile MCP) **e** `searchMcpEnabled=true` ritorna da `/api/config`
|
|
||||||
2. `/agents/profile` Aquachile torna **almeno 1 tool** in `agentTools[]` (campo `json.agentTools`)
|
|
||||||
3. La query utente ha token ≥ 3 chars che matchano `tool.name` o `tool.description` o `tool.parameters`
|
|
||||||
|
|
||||||
Se 1 solo dei 3 è falso → **early exit spiegato nel log `[MCP RANK] → EARLY EXIT reason="…"`**.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Come estendere (quick cheatsheet)
|
|
||||||
|
|
||||||
| Se vuoi… | Dove modificare |
|
|
||||||
|---|---|
|
|
||||||
| Aggiungere 1 frase tradotta ES/IT → EN per tool matching | `MCP_SEARCH_TRANSLATION_PHRASES` [main.js](file:///home/duxiter/s2s/demo/main.js#L51-L70) — nuova riga |
|
|
||||||
| Aggiungere 1 parola singola tradotta | `MCP_SEARCH_TRANSLATION_TOKENS` [main.js:72](file:///home/duxiter/s2s/demo/main.js#L72) |
|
|
||||||
| Boost score di un tipo di tool (es. report) | `rankAgentTools` [main.js:1268-1270](file:///home/duxiter/s2s/demo/main.js#L1268-L1270) + mappa subito dopo |
|
|
||||||
| Aggiungere nuovo hint a `instructions` all'avvio sessione | `effectiveReasoningInstructions()` [main.js:516-533](file:///home/duxiter/s2s/demo/main.js#L516-L533) — nuova concatenazione |
|
|
||||||
| Iniettare `instructions` server-side (segret@ dati Aquachile) senza toccare client | **Proxy tunnel** `realtime_proxy()` in [server.py](file:///home/duxiter/s2s/demo/server.py#L891-L908) → `await upstream.send({type:"session.update", session:{instructions}})` **prima** dei 2 `create_task` |
|
|
||||||
@@ -1,498 +0,0 @@
|
|||||||
# Inject pipeline — caller context & reference material
|
|
||||||
|
|
||||||
Documento ufficiale di tutti i blocchi di contesto che il server inietta
|
|
||||||
automaticamente nel prompt LLM **all'inizio di ogni sessione**, sia per il
|
|
||||||
canale **WebSocket realtime** (`/v1/realtime`, patch di `session.update`) sia
|
|
||||||
per l'endpoint **HTTP reasoning** (`POST /api/reasoning`, augmentation di
|
|
||||||
`instructions` e `user_context`).
|
|
||||||
|
|
||||||
Tutto il sistema è **generico / deploy-agnostic** per scelta architetturale:
|
|
||||||
nessun campo viene hardcodato. Ogni blocco accetta payload arbitrari e li
|
|
||||||
serializza in JSON direttamente. Qualsiasi errore di rete / JSON non
|
|
||||||
interrompe mai il flusso: il blocco problematico viene solo saltato con
|
|
||||||
una riga `[INJECT-*] ⏭️ skipped` nel log server.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 0. Master flag: `S2S_INJECT_PROFILE`
|
|
||||||
|
|
||||||
Tutta la pipeline è sotto **un unico master flag** lato server.
|
|
||||||
|
|
||||||
| Valore env `S2S_INJECT_PROFILE` | Comportamento |
|
|
||||||
|---|---|
|
|
||||||
| `false` / non impostato | **Pure passthrough.** Nessuna fetch upstream, nessuna modifica a `session.update` o al body `/api/reasoning`. Il proxy passa tutto invariato. |
|
|
||||||
| `true` / `1` / `yes` / `on` | Pipeline inject attiva. I singoli blocchi poi dipendono dagli env specifici e dai query params della URL del client. |
|
|
||||||
|
|
||||||
Perché esiste: su deploy generici (nessuna integrazione con AzurianLab /
|
|
||||||
Aquachile / sistemi interni) il client rimane un puro speech-to-speech
|
|
||||||
passthrough, senza dipendere da endpoint esterni.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Blocco episodio podcast — `episode_id`
|
|
||||||
|
|
||||||
**Priorità massima.** Quando c'è, viene considerato il riferimento PRINCIPALE
|
|
||||||
e più fidato per tutta la sessione.
|
|
||||||
|
|
||||||
### Trigger (quando si attiva)
|
|
||||||
|
|
||||||
Una qualunque delle due:
|
|
||||||
1. URL pagina browser → query string `?episode_id=episode-xxx`
|
|
||||||
2. Body `POST /api/reasoning` → campo `episode_id: "episode-xxx"`
|
|
||||||
|
|
||||||
### Sorgente dati
|
|
||||||
|
|
||||||
GET plain verso l'endpoint Aquachile AzurianLab:
|
|
||||||
|
|
||||||
```
|
|
||||||
https://aquachile.azurianlab.com/api/v1/podcast/episodes/{episode_id}
|
|
||||||
```
|
|
||||||
|
|
||||||
Parametri fetch:
|
|
||||||
- timeout 15 s
|
|
||||||
- `follow_redirects: true`
|
|
||||||
- **Nessuna auth** (endpoint pubblico)
|
|
||||||
|
|
||||||
Formato risposta accettato:
|
|
||||||
- `Content-Type: application/json` → oggetto arbitrario o array
|
|
||||||
- `{"result": <payload>}` → wrapper supportato, il contenuto viene "spacchettato"
|
|
||||||
- `text/plain` → usato come stringa raw
|
|
||||||
|
|
||||||
### Cap
|
|
||||||
|
|
||||||
24.000 caratteri dopo serializzazione JSON (transcript lunghi).
|
|
||||||
Se supera → troncato con `...` append.
|
|
||||||
|
|
||||||
### Wrap applicato
|
|
||||||
|
|
||||||
```
|
|
||||||
< PODCAST_EPISODE_HINT >
|
|
||||||
<<<PODCAST_EPISODE_BEGIN>>>
|
|
||||||
{ <TUTTI I CAMPI DELL'EPISODIO, JSON> }
|
|
||||||
<<<PODCAST_EPISODE_END>>>. This is the episode the caller is currently interacting with — answer with reference to it first.
|
|
||||||
```
|
|
||||||
|
|
||||||
**Hint testuale** (in testa, letto dal modello come primissima frase del blocco):
|
|
||||||
|
|
||||||
> Podcast episode context loaded from the page `episode_id` query param (current episode the user is referencing). Treat the episode fields (title, description, transcript, script, show notes, guests, audio metadata, etc.) as the **PRIMARY trusted reference material** for this session.
|
|
||||||
|
|
||||||
### Campi supportati (esempi — lista NON esaustiva, qualsiasi campo extra viene incluso)
|
|
||||||
|
|
||||||
- metadati base: `id`, `title`, `slug`, `description`, `summary`, `publishedAt`, `durationSeconds`
|
|
||||||
- contenuti: `transcript`, `podcastScript`, `script`, `chapters[]`, `showNotes`
|
|
||||||
- persone: `host`, `guests[]`, `author`, `guestsInfo[]`
|
|
||||||
- audio: `audioUrl`, `audioFile`, `waveform`, `sampleRate`
|
|
||||||
- classificazione: `tags[]`, `topics[]`, `category`, `season`, `episodeNumber`
|
|
||||||
- riferimenti incrociati: `reportId`, `scheduleId`, `company`, `product`
|
|
||||||
|
|
||||||
### Riferimenti codice
|
|
||||||
|
|
||||||
- fetch upstream: [_fetch_podcast_episode](file:///home/duxiter/s2s/s2s-client/server.py#L1010-L1078)
|
|
||||||
- wrap + serialize + cap 24k: [_resolve_episode_context](file:///home/duxiter/s2s/s2s-client/server.py#L475-L504)
|
|
||||||
- hint costante: [PODCAST_EPISODE_HINT](file:///home/duxiter/s2s/s2s-client/server.py#L426-L429)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Blocco profilo chiamante — `phoneNumber`
|
|
||||||
|
|
||||||
Contesto identità del chiamante. Viene usato per il modello quando risponde
|
|
||||||
(riferirsi all'utente per nome, conoscere la sua azienda, ruolo, ecc.).
|
|
||||||
|
|
||||||
### Trigger
|
|
||||||
|
|
||||||
1. URL browser → `?phoneNumber=%2B<numero-internazionale>`
|
|
||||||
2. Body `/api/reasoning` → `phoneNumber: "+…"`
|
|
||||||
|
|
||||||
### Sorgente dati
|
|
||||||
|
|
||||||
```
|
|
||||||
GET VOCERO_USER_PHONE_AUTH_URL?phoneNumber=<phone>
|
|
||||||
```
|
|
||||||
|
|
||||||
Se è configurato anche `S2S_LOGIN_BY_PHONE`, PRIMA della profile fetch:
|
|
||||||
```
|
|
||||||
POST S2S_LOGIN_BY_PHONE body { "phoneNumber": "..." }
|
|
||||||
```
|
|
||||||
Da questa risposta il proxy estrae un bearer token e lo inoltra come
|
|
||||||
`Authorization: Bearer <token>` sulle chiamate successive (profile, reports,
|
|
||||||
summary).
|
|
||||||
|
|
||||||
Formato risposta: JSON arbitrario, wrapper `{"result": ...}` supportato.
|
|
||||||
|
|
||||||
### Cap
|
|
||||||
|
|
||||||
1.600 caratteri dopo JSON serialize.
|
|
||||||
|
|
||||||
### Wrap
|
|
||||||
|
|
||||||
```
|
|
||||||
Caller profile resolved from the page phoneNumber:
|
|
||||||
{ <profilo JSON troncato> }.
|
|
||||||
Use this as trusted caller context when relevant.
|
|
||||||
```
|
|
||||||
|
|
||||||
### Riferimenti codice
|
|
||||||
|
|
||||||
- login by phone: [_login_by_phone](file:///home/duxiter/s2s/s2s-client/server.py#L812-L890)
|
|
||||||
- fetch profile: [_fetch_user_profile_for_phone](file:///home/duxiter/s2s/s2s-client/server.py#L1081-L1134)
|
|
||||||
- wrap serialize: [_resolve_user_context_text](file:///home/duxiter/s2s/s2s-client/server.py#L432-L444)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Blocco lista reports / script schedulati
|
|
||||||
|
|
||||||
Riferimento "storico" dei contenuti preparati prima della sessione. Non sono
|
|
||||||
contesto live, ma materiale che il modello deve sapere esiste (e a cui
|
|
||||||
riferirsi se l'utente chiede di report pregressi, script, podcast schedulati).
|
|
||||||
|
|
||||||
### Trigger
|
|
||||||
|
|
||||||
Solo env: `S2S_REPORT_LIST` deve essere valorizzato a un URL valido.
|
|
||||||
**Funziona anche SENZA `phoneNumber`** (lista reports generica per tutti).
|
|
||||||
|
|
||||||
Se `phoneNumber` è presente lo passa come query param all'endpoint.
|
|
||||||
|
|
||||||
### Sorgente dati
|
|
||||||
|
|
||||||
```
|
|
||||||
GET S2S_REPORT_LIST[?phoneNumber=<phone>]
|
|
||||||
```
|
|
||||||
|
|
||||||
Wrapper `{"result": [...]}` supportato.
|
|
||||||
|
|
||||||
### Cap
|
|
||||||
|
|
||||||
12.000 caratteri sul blocco COMPLETO (ogni podcast script può essere lungo).
|
|
||||||
|
|
||||||
### Wrap + filtri + sort
|
|
||||||
|
|
||||||
```
|
|
||||||
Scheduled reports / podcast scripts available for this caller before the current session.
|
|
||||||
Each entry is a scheduled report; use the podcastScript contents as trusted reference material
|
|
||||||
when the user asks about scheduled topics, reports, or prior content.
|
|
||||||
<<<REPORTS_LIST_BEGIN>>>
|
|
||||||
[
|
|
||||||
{
|
|
||||||
"sourceScheduleLabel": "...",
|
|
||||||
"date": "2026-09-06T…",
|
|
||||||
"podcastScript": "…"
|
|
||||||
},
|
|
||||||
… ordinati per data DESC (nuovi prima) …
|
|
||||||
]
|
|
||||||
<<<REPORTS_LIST_END>>>
|
|
||||||
```
|
|
||||||
|
|
||||||
Regole per ogni entry:
|
|
||||||
1. **Solo dict** vengono tenuti (stringhe/int scartati).
|
|
||||||
2. Per ogni entry estrae 3 campi con fallback a nested shape:
|
|
||||||
- Label: `sourceScheduleLabel` oppure `sourceSchedule.label`
|
|
||||||
- Date: `date` oppure `sourceSchedule.date`
|
|
||||||
- Script: `podcastScript`
|
|
||||||
3. Ordinamento **data DESC** (nuovi prima) con parser robusto:
|
|
||||||
- ISO datetime / date strings
|
|
||||||
- `Date` / `DateTime` Python (se deserializzati)
|
|
||||||
- Unix timestamp int/float
|
|
||||||
- Lexicografico DESC come fallback per stringhe ISO-ish
|
|
||||||
- Valori nulli / non-parse-abili finiscono in fondo.
|
|
||||||
|
|
||||||
### Riferimenti codice
|
|
||||||
|
|
||||||
- fetch: [_fetch_reports_list](file:///home/duxiter/s2s/s2s-client/server.py#L668-L745)
|
|
||||||
- parse data sort key: [_parse_date_key](file:///home/duxiter/s2s/s2s-client/server.py#L507-L515)
|
|
||||||
- wrap + filtro + sort: [_resolve_reports_context](file:///home/duxiter/s2s/s2s-client/server.py#L518-L666)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Blocco summary ultima conversazione
|
|
||||||
|
|
||||||
Sintesi della chiamata/chat IMMEDIATAMENTE precedente a questa. Usato dal
|
|
||||||
modello per "ricordarsi" cosa si è detto l'ultima volta senza conservare
|
|
||||||
storico lunghissimo nel prompt.
|
|
||||||
|
|
||||||
### Trigger
|
|
||||||
|
|
||||||
Entrambe le alternative sono mutuamente esclusive: `S2S_LATEST_SUMMARY_TEXT`
|
|
||||||
ha **priorità assoluta** se valorizzato.
|
|
||||||
|
|
||||||
| Modalità | Trigger |
|
|
||||||
|---|---|
|
|
||||||
| **A — HTTP fetch (dinamico per caller)** | env `S2S_LATEST_SUMMARY` impostato **AND** `phoneNumber` presente in URL/body |
|
|
||||||
| **B — Testo statico hardcoded (override)** | env `S2S_LATEST_SUMMARY_TEXT` impostato (stringa non vuota) — **ignora completamente** `S2S_LATEST_SUMMARY`, `phoneNumber`, login-by-phone, Bearer auth. Utile quando l'operatore vuole iniettare una versione **pulita, essenziale e pre-pulita** del summary, senza dipendere da nessun endpoint upstream. |
|
|
||||||
|
|
||||||
### Sorgente dati
|
|
||||||
|
|
||||||
**Modalità A (HTTP fetch):**
|
|
||||||
```
|
|
||||||
GET S2S_LATEST_SUMMARY?phoneNumber=<phone>
|
|
||||||
```
|
|
||||||
Wrapper `{"result": <valore>}` supportato.
|
|
||||||
Accetta SIA JSON che plain text (diversi backend summary ritornano testo semplice).
|
|
||||||
|
|
||||||
**Modalità B (override statico):**
|
|
||||||
- Nessuna chiamata HTTP. Viene usato direttamente il valore grezzo di `S2S_LATEST_SUMMARY_TEXT` dal file `.env`.
|
|
||||||
- Le sequenze di escape `\n` nel valore env vengono **interpretate** come a-capo vero (stesso comportamento di `S2S_PRIMARY_PROMPT`), quindi è possibile scrivere summary multi-line in una sola riga del `.env`.
|
|
||||||
- Esempio:
|
|
||||||
```env
|
|
||||||
S2S_LATEST_SUMMARY_TEXT=Última conversación 2026-09-22:\n- Usuario preguntó por mortalidad salmón AquaChile semana 38.\n- Se refirió al reporte Porcentaje Mortalidad.
|
|
||||||
```
|
|
||||||
- Il valore viene passato as-is a `_resolve_latest_summary_context` → il wrapper `LATEST_SUMMARY_HINT` e il cap a 1600 caratteri vengono applicati normalmente.
|
|
||||||
|
|
||||||
### Cap
|
|
||||||
|
|
||||||
1.600 caratteri (applicato da `_resolve_latest_summary_context` sia in modalità A che B).
|
|
||||||
|
|
||||||
### Wrap
|
|
||||||
|
|
||||||
```
|
|
||||||
Latest conversation summary for this caller (most recent before the current session):
|
|
||||||
<<<LATEST_SUMMARY_BEGIN>>>
|
|
||||||
<summary, stringa o JSON serializzato>
|
|
||||||
<<<LATEST_SUMMARY_END>>>.
|
|
||||||
Use this as trusted history when the user references prior calls.
|
|
||||||
```
|
|
||||||
|
|
||||||
### Riferimenti codice
|
|
||||||
|
|
||||||
- fetch HTTP (modalità A): [_fetch_latest_summary_for_phone](file:///home/duxiter/s2s/s2s-client/server.py#L1563-L1654)
|
|
||||||
- override statico (modalità B) → riga iniziale `if S2S_LATEST_SUMMARY_TEXT:` dentro la funzione sopra
|
|
||||||
- wrap: [_resolve_latest_summary_context](file:///home/duxiter/s2s/s2s-client/server.py#L987-L1012)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. Blocco `user_context` custom dal browser (SOLO `/api/reasoning`)
|
|
||||||
|
|
||||||
**Canale solo HTTP reasoning.**
|
|
||||||
Non è nel WS: il WS non ha campo user_context nel protocollo realtime
|
|
||||||
OpenAI (tutto passa per `session.instructions`).
|
|
||||||
|
|
||||||
Composizione lato client in [main.js fetchOpenAiReasoningAnswer](file:///home/duxiter/s2s/s2s-client/main.js#L1337-L1364):
|
|
||||||
|
|
||||||
```js
|
|
||||||
const profileCtx = voceroUserContextText();
|
|
||||||
const ragCtx = customRagContextText();
|
|
||||||
const user_context = [profileCtx, ragCtx]
|
|
||||||
.map(s => s.trim()).filter(Boolean).join("\n\n");
|
|
||||||
```
|
|
||||||
|
|
||||||
### Dove va a finire
|
|
||||||
|
|
||||||
Nel body a monte sia NODE_RED sia OPENAI, questo blocco viene **APPESO DOPO**
|
|
||||||
tutti i blocchi iniettati server-side:
|
|
||||||
|
|
||||||
```
|
|
||||||
user_context finale =
|
|
||||||
[1. episode] ← solo se episode_id
|
|
||||||
[2. profile] ← solo se phoneNumber + env profilatura
|
|
||||||
[3. reports] ← solo se env S2S_REPORT_LIST
|
|
||||||
[4. summary] ← solo se phoneNumber + S2S_LATEST_SUMMARY
|
|
||||||
+ browser user_context (profileCtx + ragCtx)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Ordine finale nel prompt LLM
|
|
||||||
|
|
||||||
Questa è la sequenza **letterale** con cui i blocchi vengono concatenati
|
|
||||||
dentro `session.instructions` (WS) e in `instructions` / `user_context`
|
|
||||||
(reasoning HTTP).
|
|
||||||
|
|
||||||
```
|
|
||||||
1. EPISODIO PODCAST ← PRIORITÀ PIÙ ALTA (primary trusted reference)
|
|
||||||
2. PROFILO CHIAMANTE ← identity / caller profile
|
|
||||||
3. REPORTS SCHEDULATI ← scheduled / prior content
|
|
||||||
4. SUMMARY ULTIMA CALL ← history più recente
|
|
||||||
5. Base instructions (prompt predefinito o primary prompt)
|
|
||||||
6. (solo reasoning) user_context custom inviato dal browser
|
|
||||||
```
|
|
||||||
|
|
||||||
Perché questo ordine:
|
|
||||||
- più in alto nel prompt = più peso / attenzione dal modello.
|
|
||||||
- l'episodio attuale è il materiale che l'utente ha esplicitamente richiesto
|
|
||||||
aprendo la pagina con `?episode_id=`, quindi merita massimo peso.
|
|
||||||
- il profilo chiamante va sempre prima di "contenuti generici" (reports e
|
|
||||||
summary) perché serve per coerenza nel riferirsi all'utente.
|
|
||||||
- reports = contenuti schedulati PRIMA della sessione corrente → più vecchi
|
|
||||||
dell'ultima chiamata, quindi messi PRIMA del summary ultima chiamata che
|
|
||||||
è il contenuto più "vivo".
|
|
||||||
|
|
||||||
Implementazione ordine: [_augment_instructions_with_profile](file:///home/duxiter/s2s/s2s-client/server.py#L1149-L1159)
|
|
||||||
```python
|
|
||||||
contexts = [c for c in [episode_ctx, profile_ctx, reports_ctx, summary_ctx] if c]
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Punti di innesco (quando viene calcolato / applicato)
|
|
||||||
|
|
||||||
### Canale WebSocket — `/v1/realtime`
|
|
||||||
|
|
||||||
**MOMENTO 1 (solo una volta):** apertura WS.
|
|
||||||
|
|
||||||
Nel handler [@app.websocket("/v1/realtime")](file:///home/duxiter/s2s/s2s-client/server.py#L1791-L1959)
|
|
||||||
appena la WS viene accettata:
|
|
||||||
1. parsa `ws.url.query` per `phoneNumber` e `episode_id`
|
|
||||||
2. esegue login (se attivo) → profile → episode → reports → summary
|
|
||||||
3. salva TUTTI i blocchi già wrappati e serializzati nel dict `session_ctx`:
|
|
||||||
```python
|
|
||||||
session_ctx = {
|
|
||||||
"injected_episode_context": "...",
|
|
||||||
"injected_profile_context": "...",
|
|
||||||
"injected_reports_context": "...",
|
|
||||||
"injected_summary_context": "...",
|
|
||||||
...
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**MOMENTO 2 (ogni `session.update` client→upstream):** nel task
|
|
||||||
[_client_to_upstream](file:///home/duxiter/s2s/s2s-client/server.py#L2649-L2661)
|
|
||||||
ogni frame in uscita viene passato a:
|
|
||||||
|
|
||||||
```python
|
|
||||||
_patch_client_session_update_with_profile(
|
|
||||||
frame_text,
|
|
||||||
user_context_text = profile_ctx,
|
|
||||||
reports_context_text = reports_ctx,
|
|
||||||
summary_context_text = summary_ctx,
|
|
||||||
episode_context_text = episode_ctx, # ← nuovo
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
Se il frame NON è `session.update` → ritorna **identico** e non viene loggato
|
|
||||||
(per evitare spam).
|
|
||||||
Se è `session.update` → patch del campo `payload["session"]["instructions"]`.
|
|
||||||
|
|
||||||
### Canale HTTP — `POST /api/reasoning`
|
|
||||||
|
|
||||||
**Ogni chiamata** ricostruisce i blocchi da zero nel
|
|
||||||
[handler `/api/reasoning`](file:///home/duxiter/s2s/s2s-client/server.py#L1411-L1577):
|
|
||||||
1. legge `req.phoneNumber` + `req.episode_id`
|
|
||||||
2. fetch episodio (se episode_id) + login + profile + reports + summary
|
|
||||||
3. costruisce `user_context_parts` e `effective_instructions`
|
|
||||||
4. inoltra al backend (NODE_RED o OPENAI)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Quando un blocco viene SKIPPATO (niente inject)
|
|
||||||
|
|
||||||
Tutti i blocchi hanno 5 possibili ragioni di skip. Nessuna rompe mai il
|
|
||||||
flusso: ogni caso produce una riga `[INJECT-*] ⏭️ skipped` nei log e basta.
|
|
||||||
|
|
||||||
| Causa | Log |
|
|
||||||
|---|---|
|
|
||||||
| Master flag `S2S_INJECT_PROFILE=false` | `[INJECT-PROFILE] ⏭️ skipped (S2S_INJECT_PROFILE=false)` |
|
|
||||||
| Parametro mancante (es. `phoneNumber` non in URL) | `[INJECT-*] ⏭️ skipped (<parametro> missing)` |
|
|
||||||
| Endpoint env non configurato | `[INJECT-*] ⏭️ skipped (env for ... empty)` |
|
|
||||||
| Upstream HTTP ≠ 200 | `[INJECT-*] ⛔ upstream HTTP <status_code>` |
|
|
||||||
| Upstream ha ritornato vuoto / null / `{}` / `[]` | `[INJECT-*] ⏭️ upstream returned empty` |
|
|
||||||
| Eccezione di rete (timeout, DNS, SSL) | `[INJECT-*] ⛔ fetch unreachable error=<exc>` |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Marcatori log / marker nel prompt
|
|
||||||
|
|
||||||
Ogni blocco è **auto-documentante** per ispezione dei log con coppie di
|
|
||||||
marker:
|
|
||||||
|
|
||||||
```
|
|
||||||
<<<PODCAST_EPISODE_BEGIN>>> / <<<PODCAST_EPISODE_END>>>
|
|
||||||
<<<REPORTS_LIST_BEGIN>>> / <<<REPORTS_LIST_END>>>
|
|
||||||
<<<LATEST_SUMMARY_BEGIN>>> / <<<LATEST_SUMMARY_END>>>
|
|
||||||
```
|
|
||||||
|
|
||||||
Quando `S2S_DEBUG_PROMPT=true` viene loggato il contenuto COMPLETO di ogni
|
|
||||||
blocco con i tag `[DEBUG-EPISODE]`, `[DEBUG-REPORTS]`, `[DEBUG-SUMMARY]` sia
|
|
||||||
all'atto del load (WS connect) sia dopo l'augment delle instructions (prima
|
|
||||||
di inoltrare a monte). **WARNING:** questo env può loggare PII.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Tabella riepilogo: trigger per ogni blocco
|
|
||||||
|
|
||||||
| # | Blocco | Trigger | env richiesto | parametro | cap |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| 1 | Episodio podcast | `episode_id` in URL o body | (nessuno — URL hardcodato Aquachile) | `episode_id=` | 24 k |
|
|
||||||
| 2 | Profilo chiamante | `phoneNumber` | `VOCERO_USER_PHONE_AUTH_URL` | `phoneNumber=` | 1,6 k |
|
|
||||||
| 3 | Reports schedulati | sempre se env | `S2S_REPORT_LIST` | opzionale `phoneNumber=` | 12 k |
|
|
||||||
| 4 | Summary ultima call | `phoneNumber` **oppure** override statico | `S2S_LATEST_SUMMARY` (HTTP) **o** `S2S_LATEST_SUMMARY_TEXT` (statico, ha priorità) | `phoneNumber=` (solo modalità HTTP; override statico non richiede nessun param) | 1,6 k |
|
|
||||||
| 5 | Browser user_context | solo reasoning | (nessuno) | body `user_context` | illimitato |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Error handling strategico
|
|
||||||
|
|
||||||
Perché tutti gli errori sono silenziosi (log + skip) e non abortiscono:
|
|
||||||
|
|
||||||
> Se il backend dei profili è giù per manutenzione 5 minuti non vogliamo
|
|
||||||
> che il Voice Agent smetta di funzionare. Semplicemente l'assistente non
|
|
||||||
> conosce l'identità del chiamante per 5 minuti, ma parla normalmente.
|
|
||||||
|
|
||||||
L'utente non si accorge di niente. L'operatore legge i log e vede le righe
|
|
||||||
`⛔`; può risolvere offline.
|
|
||||||
|
|
||||||
Se volessi forzare il fail (raro, su deploy dedicato), si trasformerebbero
|
|
||||||
le `return None` finali in `raise HTTPException(...)` — ma per default
|
|
||||||
non è il comportamento voluto.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Appendice A — Start-session modes (`S2S_START_SESSION_MODE`)
|
|
||||||
|
|
||||||
Oltre alla inject pipeline lato server, è presente una feature lato client che
|
|
||||||
governa **come e se la sessione vocale inizia in automatico** e se il modello
|
|
||||||
saluta per primo (in modo identico alla wake-word "Hola Vocero").
|
|
||||||
|
|
||||||
### Valori supportati
|
|
||||||
|
|
||||||
| `S2S_START_SESSION_MODE` | Chi inizia? | Richiede click utente? | Trigger greet iniziale |
|
|
||||||
|---|---|---|---|
|
|
||||||
| *(vuoto) / default* | Utente, tap manuale Orb | ✅ | No |
|
|
||||||
| `TALK` | Auto dopo 2s con overlay unlocker | ✅ (1 click policy-safe su Safari/iOS) | WS client: `sendUserText("hola vocero") + requestResponse()` dopo `session.updated` |
|
|
||||||
| `TALKFIRST` | Auto dopo 2s **SENZA click** (se possibile) | ❌ se mic già "allow always"; fallback unlocker se `getUserMedia` fallisce | **Stesso identico codice della wake-word vera:** `pendingWakeWordConfirmationText = "hola vocero"` (main.js `case "connected"` → `sendUserText + requestResponse`) |
|
|
||||||
|
|
||||||
### Warmup 2s
|
|
||||||
|
|
||||||
Il timer non parte prima di 2000 ms per lasciare terminare:
|
|
||||||
- `fetchVoceroUserProfile()` (identity chiamante)
|
|
||||||
- `fetchVoceroReportsList()` (storico script podcast)
|
|
||||||
- fetch episode (`episode_id`) se presente
|
|
||||||
- `GET /api/config` (con la quale il client apprende `startSessionMode`)
|
|
||||||
|
|
||||||
Senza warmup le injection lato server potrebbero ancora non essere completate e
|
|
||||||
il greet iniziale arriverebbe "cieco" (senza contesto episodio/profilo).
|
|
||||||
|
|
||||||
### Fallback di `TALKFIRST`
|
|
||||||
|
|
||||||
Nel 99% dei casi `getUserMedia()` fuori da una user gesture fallisce
|
|
||||||
(NotAllowedError / NotReadableError su Safari, Chrome con policy strict, iOS).
|
|
||||||
In tal caso:
|
|
||||||
1. si chiude il client WS (`teardown()`)
|
|
||||||
2. stato torna `idle`
|
|
||||||
3. si mostra `#talk-unlocker` con la dicitura:
|
|
||||||
- **"Se requiere permiso de micrófono"**
|
|
||||||
- **"Toca en cualquier lugar para permitir el audio y empezar"**
|
|
||||||
4. `pendingWakeWordConfirmationText` NON viene svuotato → quando l'utente
|
|
||||||
toccherà, `case "connected"` triggerà comunque la greet come wake-word.
|
|
||||||
|
|
||||||
### Riferimenti codice
|
|
||||||
|
|
||||||
| Componente | File | Range righe |
|
|
||||||
|---|---|---|
|
|
||||||
| Lettura env + banner startup + `/api/config` exposure | [server.py](file:///home/duxiter/s2s/s2s-client/server.py) | [L150](file:///home/duxiter/s2s/s2s-client/server.py#L150), [L196-L212](file:///home/duxiter/s2s/s2s-client/server.py#L196-L212), [L1383-L1400](file:///home/duxiter/s2s/s2s-client/server.py#L1383-L1400) |
|
|
||||||
| Overlay HTML unlocker | [index.html](file:///home/duxiter/s2s/s2s-client/index.html#L32-L39) | L32-L39 |
|
|
||||||
| Stili overlay | [style.css](file:///home/duxiter/s2s/s2s-client/style.css#L145-L215) | L145-L215 |
|
|
||||||
| Variabile globale + parse config client | [main.js](file:///home/duxiter/s2s/s2s-client/main.js#L628-L632) | L628-L632 |
|
|
||||||
| `fetchConfig()` parsing + `reportDebug("B")` | [main.js](file:///home/duxiter/s2s/s2s-client/main.js#L2309-L2320) | L2309-L2320 |
|
|
||||||
| `new S2sWsRealtimeClient({startWithResponse})` | [main.js](file:///home/duxiter/s2s/s2s-client/main.js#L2683-L2692) | L2683-L2692 |
|
|
||||||
| Scheduling + `tryTalkFirstDirect()` + unlocker gesture | [main.js](file:///home/duxiter/s2s/s2s-client/main.js#L2976-L3083) | L2976-L3083 |
|
|
||||||
| `onClientStatus("connected")` → wake-word inject flow | [main.js](file:///home/duxiter/s2s/s2s-client/main.js#L2897-L2914) | L2897-L2914 |
|
|
||||||
| WS client → `TALK` mode sendUserText("hola vocero") + `requestResponse()` | [s2s-ws-client.js](file:///home/duxiter/s2s/s2s-client/ws/s2s-ws-client.js#L688-L713) | L688-L713 |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Riferimenti file
|
|
||||||
|
|
||||||
| File | Ruolo |
|
|
||||||
|---|---|
|
|
||||||
| [server.py](file:///home/duxiter/s2s/s2s-client/server.py) | Tutta la pipeline inject: fetch, wrap, patch, handler WS + reasoning |
|
|
||||||
| [main.js](file:///home/duxiter/s2s/s2s-client/main.js) | Passaggio client `?phoneNumber` + `?episode_id` → WS query params e POST `/api/reasoning` body |
|
|
||||||
| [CHANGELOG.md](file:///home/duxiter/s2s/s2s-client/CHANGELOG.md) | History rilasci / modifiche alla inject pipeline |
|
|
||||||
@@ -1,256 +0,0 @@
|
|||||||
---
|
|
||||||
title: HF Realtime Voice
|
|
||||||
emoji: 🎙️
|
|
||||||
colorFrom: indigo
|
|
||||||
colorTo: purple
|
|
||||||
sdk: docker
|
|
||||||
app_port: 7860
|
|
||||||
pinned: false
|
|
||||||
short_description: Voice chat over WebSocket against a HF speech-to-speech
|
|
||||||
hf_oauth: true
|
|
||||||
---
|
|
||||||
|
|
||||||
# Realtime Voice Demo (WebSocket transport)
|
|
||||||
|
|
||||||
Browser voice-chat UI for the
|
|
||||||
backend. The browser streams mic audio over a WebSocket using the OpenAI
|
|
||||||
Realtime **GA** protocol and plays back the assistant's audio as it arrives.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🆕 Novedades / Actualizaciones (Azurian Vocero realtime ver.Beta 2.1)
|
|
||||||
|
|
||||||
> Versión del navegador que debes ver cargada en consola: **`2026-09-04 11:20`**.
|
|
||||||
> Si no aparece, haz **borrado de datos del sitio + recarga dura (Ctrl+Shift+R)** o abre una pestaña en modo incógnito.
|
|
||||||
|
|
||||||
### 🎨 Rediseño visual completo (tema aurora + glassmorphism)
|
|
||||||
- **Tema de fondo aurora**: gradiente radial violeta/rosa/cian sobre lienzo oscuro con malla sutil + animación `auroraShift` lenta.
|
|
||||||
- **Cromado uniforme glass**: todos los elementos flotantes (tarjeta brand, panel user profile, botones icono, botón enviar chat, panel informes sidebar) usan *glass blur + saturate + borde gradiente brand 1px* (técnica mask `padding-box / border-box` doble linear-gradient).
|
|
||||||
- **Tokens de marca 3 colores brand** (violeta `#8b5cf6`, rosa `#ec4899`, cian `#22d3ee`) mezclados con `color-mix(in srgb, brand X%, neutral)` → ningún color hardcodeado deriva; todo el UI sigue estos tres tonos.
|
|
||||||
|
|
||||||
### 🧭 Estructura (orden vertical de arriba → abajo)
|
|
||||||
1. **`header.topbar`** brand "Vocero" → avatar circular iniciales nombre usuario.
|
|
||||||
- 🟦 **Avatar user**: círculo 40×40px con gradiente brand, initials mayúsculas bold 800, hover lift/glow + foco teclado. Al hacer clic → diálogo modal perfil usuario (rejilla 2 columnas email/teléfono/username/rol/empresa/userid/país/fecha creación/último login; links `mailto:`/`tel:`; chips mono para códigos; mobile ≤480px apilado 1 col).
|
|
||||||
- 🟦 **Botón toggle chat (icono speech bubble) → ELIMINADO PERMANENTEMENTE**. La chat ahora siempre está visible inline; el botón toggle ya no existe en el DOM.
|
|
||||||
2. **`main.stage`** (arriba del todo, flexible): orbe/globode voz animado (3 anillos: core, ring, conic spin 3 velocidades + highlight specular + glows breathe `breatheOrb`) + caption `TAP OR SAY "HOLA VOCERO"` micro-bob uppercase gradient.
|
|
||||||
- ✅ Antes había una zona vacía sobre el orbe → eliminada, ahora el stage empieza en la parte superior (`justify-content: flex-start`).
|
|
||||||
3. **`<section id="chat-panel" class="chat-panel chat-panel-inline">` SIEMPRE VISIBLE, 100% ancho, SOTTO stage, SOPRA footer sticky** (rellena la zona vacía inferior).
|
|
||||||
- ✅ Antes: overlay lateral derecho. **Ahora: bloque inline dentro de `#app` como hijo flex-col**.
|
|
||||||
- 🟦 Sin marco / sin riquadro (fondo aurora seamless, 0 borde, 0 shadow, 0 backdrop blur).
|
|
||||||
- Cabecera `CONVERSACIÓN` h3 uppercase 13px gradiente.
|
|
||||||
- `#chat-history` scroll interno. Burbujas de mensaje glass con **código de colores por rol**: usuario (tint 10% voice-user cyan + borde 44% cyan + glow) / assistant (tint 10% voice-assistant violet/pink + borde 42% + glow).
|
|
||||||
- Distancias seguras anti-borde: `.hist-msg.user margin-left:6px auto dx` / `.hist-msg.assistant margin-right:22px auto sx` → **nunca las burbujas tocan el borde del viewport** (fix confirmado).
|
|
||||||
- Formulario input texto `#chat-form` (arriba del botón enviar ✈️) **actualmente oculto (`hidden`)** para modo voz-puro; elimina ese atributo para reactivarlo. El handler submit que envía `client.sendUserText(text)` al websocket ya está conectado.
|
|
||||||
4. **Footer sticky `position: sticky; bottom:0; z:3; backdrop-blur`** "Azurian Vocero realtime ver.Beta 2.1" siempre dentro del viewport, incluso cuando el documento crece y haces scroll (no requiere scroll hacia abajo para verlo). Separador superior línea gradiente brand.
|
|
||||||
|
|
||||||
### 📜 Sidebar de informes a la IZQUIERDA (antes topbar inline)
|
|
||||||
- Desktop **≥ 1024px**: sidebar fija left 320px siempre visible; `#app padding-left 320px` reserva sitio.
|
|
||||||
- Tablet/Mobile **< 1024**: offcanvas `translateX(-105%)` slide-in con backdrop blur; toggle mediante botón icono reportes `#reports-sidebar-toggle` en topbar con badge contador.
|
|
||||||
- `#reports-hint` siempre visible; `.reports-suggestions` y `.topbar-right` con puerta triple de visibilidad (atributo HTML hidden + CSS rule `[data-debug-only]:not([data-debug-enabled]) !important display:none` + JS `applyDebugOnlyVisibilityGate()` ejecutado 6 veces por lifecycle) → solo aparecen cuando el query param `?debug=true` está presente en la URL.
|
|
||||||
|
|
||||||
### ⬇️ Auto-scroll chat SIEMPRE hacia la última respuesta del asistente
|
|
||||||
Arquitectura anti-clipping definitiva (versión 11:20 fix):
|
|
||||||
|
|
||||||
1. **CSS sobrante intencional** (`96 → 160px chat-history padding-bottom`, `40→160px chat-panel clamp`, `56→120px #app padding-bottom + safe-area-inset-bottom`) → el último párrafo siempre está por ENCIMA de la línea del footer glass flotante.
|
|
||||||
2. **Unidades móviles seguras `svh / svw`** (smallest viewport): evita oscilaciones layout cuando aparece/desaparece la barra URL del móvil.
|
|
||||||
3. **JS cancel-and-reschedule** cada vez que `_scrollToBottom()` se llama: si llegan 100 tokens streaming en 1 frame, el último cancela todas las cadenas pendientes y se ejecuta desde cero → **no hay coalescing flag SKIP** (esta era la causa original del scroll insuficiente).
|
|
||||||
4. **5 reintentos escalonados `0 / +30 / +70 / +150 / +250ms`**:
|
|
||||||
- 0 ms (inmediato): 90% de los casos.
|
|
||||||
- 30 ms: tras 1 layout-pass.
|
|
||||||
- 70 ms: **corrige el wrapping de la última palabra en una línea nueva** (bug más común).
|
|
||||||
- 150 ms: después de reflow Chrome scroll-anchor.
|
|
||||||
- 250 ms: después de expandir filas tool / imágenes grandes.
|
|
||||||
5. **Snap con `Number.MAX_SAFE_INTEGER`**: el navegador clampa automáticamente al máximo real (`scrollHeight - clientHeight`); no hay race de leer scrollHeight demasiado pronto.
|
|
||||||
6. `overflow-anchor: none; scroll-behavior: auto !important;` → sin smooth, sin Chrome scroll-anchor-luchando contra nosotros.
|
|
||||||
7. Llamadas scroll explicitas extra en: `onResponseFinished` (fin de respuesta assistant, crítico), `onToolCall`, `onToolResult`.
|
|
||||||
|
|
||||||
### 🛡️ Protecciones anti-error de refs eliminadas
|
|
||||||
- En el refactor de chat eliminamos 4 nodos DOM (`#chat-btn`, `#chat-badge`, `#chat-panel-backdrop`, `#chat-panel-close`).
|
|
||||||
- **Antes**: el constructor de `ChatView` usaba `$("#...")` strict → lanzaba `Error("Missing element: #chat-btn")` al arrancar.
|
|
||||||
- **Ahora**: esas 4 refs opcionales usan `document.querySelector("#...")` → devuelve `null` cuando no existen, sin throw; todos los listeners guarded `if (el)`. Los 3 IDs obligatorios (`#chat-panel`, `#chat-history`, `#bubble-stack`) siguen con strict para fallar fuerte si alguien los borra por error.
|
|
||||||
|
|
||||||
### 🔄 Actualizaciones de versión / cache bust
|
|
||||||
Para evitar caché HTTP agresiva en navegadores móviles (ESM sin bundler + service worker = nunca refrescan URLs iguales), todas las referencias a recursos estáticos usan query param `?v=YYYYMMDDHHMM` sincronizado en **6 puntos**:
|
|
||||||
1. `index.html <link rel="stylesheet" href="style.css?v=...">`
|
|
||||||
2. `index.html <script type="module" src="main.js?v=...">`
|
|
||||||
3-6. Los **4 imports ESM dentro de main.js** (`ws/s2s-ws-client`, `ui/dom`, `ui/chat`, `ui/account`).
|
|
||||||
|
|
||||||
Además hay un **script inline en `<head>`** tras el style.css que al `DOMContentLoaded` fuerza a reescribir el href del stylesheet con la última versión si por algún motivo cargó la antigua (anti-cache móvil).
|
|
||||||
|
|
||||||
### Deploy
|
|
||||||
```bash
|
|
||||||
cd /home/duxiter/s2s
|
|
||||||
docker compose restart s2s_ui
|
|
||||||
```
|
|
||||||
Limpia siempre datos del sitio en el navegador (Application → Storage → Clear site data) y recarga dura `Ctrl+Shift+R` al finalizar.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Quick start (local)
|
|
||||||
|
|
||||||
1. **Start the speech-to-speech backend** in realtime mode (from the repo root;
|
|
||||||
```bash
|
|
||||||
uv run speech-to-speech \
|
|
||||||
--mode realtime \
|
|
||||||
--stt parakeet-tdt \
|
|
||||||
--llm_backend transformers \
|
|
||||||
--tts kokoro \
|
|
||||||
--model_name "Qwen/Qwen3-4B-Instruct-2507" \
|
|
||||||
--llm_device mps \
|
|
||||||
--llm_torch_dtype float16 \
|
|
||||||
--enable_live_transcription
|
|
||||||
```
|
|
||||||
|
|
||||||
The realtime server listens on `ws://localhost:8765/v1/realtime` by default
|
|
||||||
(`--ws_host` / `--ws_port` to change).
|
|
||||||
|
|
||||||
2. **Start this app**, pointing it at the backend with `SPEECH_TO_SPEECH_URL`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
uv pip install -r demo/requirements.txt
|
|
||||||
export SPEECH_TO_SPEECH_URL=ws://localhost:8765/v1/realtime
|
|
||||||
export SERPER_API_KEY=... # optional; web search is disabled without it
|
|
||||||
uv run uvicorn --app-dir demo server:app --reload --port 7860
|
|
||||||
```
|
|
||||||
|
|
||||||
Or with Docker:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker build -t s2s-demo demo/
|
|
||||||
docker run -p 7860:7860 -e SPEECH_TO_SPEECH_URL=ws://host.docker.internal:8765/v1/realtime s2s-demo
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Open <http://localhost:7860/>, click the orb, allow the mic, talk.
|
|
||||||
|
|
||||||
> Browsers require **HTTPS or `localhost`** for `getUserMedia()` (mic + camera).
|
|
||||||
> `127.0.0.1` and `localhost` both work; plain `http://192.168.x.y` does NOT.
|
|
||||||
|
|
||||||
Smoke-test the backend from the shell:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
websocat ws://localhost:8765/v1/realtime
|
|
||||||
# -> you should get a session.created event back immediately
|
|
||||||
```
|
|
||||||
|
|
||||||
## How it works
|
|
||||||
|
|
||||||
1. The browser opens a WebSocket on the configured `/v1/realtime` URL.
|
|
||||||
2. Server pushes `session.created` on connect. Client replies with
|
|
||||||
`session.update` (OpenAI Realtime **GA** schema: `session.audio.input`,
|
|
||||||
`session.audio.output`, `session.output_modalities`).
|
|
||||||
3. Client streams mic audio as PCM16 16 kHz mono base64 chunks
|
|
||||||
(`input_audio_buffer.append`, one frame every ~40 ms).
|
|
||||||
4. Server pushes `response.output_audio.delta` (PCM16 24 kHz mono base64)
|
|
||||||
and transcript deltas.
|
|
||||||
|
|
||||||
The backend exposes one concurrent session per pipeline unit
|
|
||||||
(`--num_pipelines` to serve more).
|
|
||||||
|
|
||||||
## Connecting to a backend
|
|
||||||
|
|
||||||
Three modes, picked by env (`/api/config` tells the client which one is active):
|
|
||||||
|
|
||||||
- **`SPEECH_TO_SPEECH_URL` env** — the mode you want for local use, and the
|
|
||||||
highest priority. The browser connects **directly** to this realtime
|
|
||||||
WebSocket URL; it's shown read-only in Settings. Setting it disables the
|
|
||||||
load-balancer logic entirely (no `/api/session` proxy, no queue, no
|
|
||||||
metering, no sign-in). Unlike the LB address it is not a secret. Accepts a
|
|
||||||
full `ws(s)://host/v1/realtime` URL or a bare host like `localhost:8765`
|
|
||||||
(the app adds `/v1/realtime`).
|
|
||||||
- **Neither env set** — **Settings → Speech-to-speech server URL**: paste a
|
|
||||||
full connect URL or a bare host, and the browser connects to it directly.
|
|
||||||
- **`LOAD_BALANCER_URL` env** — multi-compute deployments only: the browser
|
|
||||||
POSTs the same-origin `/api/session` proxy, the server forwards to the LB,
|
|
||||||
and the browser dials the per-session compute URL the LB hands back. The LB
|
|
||||||
address never reaches the browser; the Settings URL field is hidden.
|
|
||||||
|
|
||||||
| `SPEECH_TO_SPEECH_URL` | `LOAD_BALANCER_URL` | `SPACE_ID` | Connection | URL field | Metering |
|
|
||||||
|:---:|:---:|:---:|---|---|---|
|
|
||||||
| ✅ | any | any | direct → pinned URL | visible, locked | off |
|
|
||||||
| – | – | any | direct → user URL | editable | off |
|
|
||||||
| – | ✅ | ✅ | LB proxy | hidden | **on** |
|
|
||||||
| – | ✅ | – | LB proxy | hidden | off |
|
|
||||||
|
|
||||||
**Settings → Restart** reconnects with the current voice, instructions and URL.
|
|
||||||
|
|
||||||
## Tools
|
|
||||||
|
|
||||||
The assistant can call two tools mid-conversation (toggle them from the **Tools**
|
|
||||||
button, top-right):
|
|
||||||
|
|
||||||
- **Web search** — Google results via Serper.dev, proxied server-side so the key
|
|
||||||
never reaches the browser. Set `SERPER_API_KEY` as an env var / Space secret.
|
|
||||||
Without it, the tool is disabled unless the user pastes their own key in the
|
|
||||||
Tools panel.
|
|
||||||
- **Camera** — while enabled, a live self-view shows bottom-left; when the model
|
|
||||||
calls the tool, the current frame is sent to the vision-language model so it can
|
|
||||||
see what you're showing it.
|
|
||||||
|
|
||||||
## Usage limits (deployed Space only)
|
|
||||||
|
|
||||||
Conversation time is metered per UTC day by sign-in tier (see `limiter.py` /
|
|
||||||
`auth.py`), but **only on the deployed Space** — metering turns on only when BOTH
|
|
||||||
`LOAD_BALANCER_URL` and `SPACE_ID` (injected automatically by the HF Space
|
|
||||||
runtime) are present. Running locally — even with `LOAD_BALANCER_URL` exported —
|
|
||||||
leaves the app unmetered. Tunable via env:
|
|
||||||
|
|
||||||
| Env | Default | What |
|
|
||||||
|-----|---------|------|
|
|
||||||
| `LIMIT_ANON_SEC` | `300` | Daily seconds for anonymous visitors (5 min) |
|
|
||||||
| `LIMIT_FREE_SEC` | `600` | Daily seconds for signed-in non-PRO users (10 min) |
|
|
||||||
| `UNLIMITED_ORGS` | _(adds to defaults)_ | Extra HF org names whose members get **unlimited** usage, like PRO |
|
|
||||||
| `USAGE_HASH_SECRET` | _(random)_ | HMAC secret for hashing identity keys + signing the anon cookie |
|
|
||||||
|
|
||||||
PRO members are always unlimited. Members of `cerebras`, `HuggingFaceM4`,
|
|
||||||
`smolagents`, and `pollen-robotics` are unlimited out of the box (shown as
|
|
||||||
"Team", not "PRO"); set `UNLIMITED_ORGS=my-team` to add more. Matched
|
|
||||||
case-insensitively against the user's organisations from HF OAuth.
|
|
||||||
|
|
||||||
## Settings (stored in `localStorage`)
|
|
||||||
|
|
||||||
| Key | What |
|
|
||||||
|-----|------|
|
|
||||||
| Speech-to-speech server URL | Direct realtime WebSocket URL (hidden/locked when pinned by env) |
|
|
||||||
| Voice | Qwen3-TTS speaker name (Aiden, Ryan, Dylan, Eric, Ono_Anna, Serena, Sohee, Uncle_Fu, Vivian) |
|
|
||||||
| Instructions | System prompt sent in `session.update` once the WS opens |
|
|
||||||
|
|
||||||
LocalStorage keys are namespaced `s2s.ws.*` so this app's settings do
|
|
||||||
NOT collide with the WebRTC variant.
|
|
||||||
|
|
||||||
## Files
|
|
||||||
|
|
||||||
> 🇪🇸 **Notas adicionales (Azurian Vocero)**: `index.html` ahora incluye (1) botón toggle lateral izquierdo reportes `#reports-sidebar-toggle` con badge contador `#reports-sidebar-badge` visible solo <1024px, (2) `<dialog id="user-profile-dialog">` modal perfil usuario que se abre al hacer clic en el avatar circular del encabezado, (3) panel chat `<section id="chat-panel">` permanente dentro de `#app` entre `</main>` y footer sticky (el panel overlay derecho antiguo y el botón toggle chat superior fueron ELIMINADOS completamente).
|
|
||||||
|
|
||||||
| File | Role |
|
|
||||||
|------|------|
|
|
||||||
| `index.html` | Single page, orb + settings modal (identical UI to the WebRTC app) |
|
|
||||||
| `main.js` | State machine, settings, tools, camera, noise-gate UI wiring + chat-form submit handler (`client.sendUserText`) + user profile dialog open/close + reports sidebar open/close + 3-layer debug gate |
|
|
||||||
| `ui/chat.js` | `ChatView`: history panel, ephemeral bubbles, transcript/tool streaming; always-inline 100% width chat panel w/ 5-step retry auto-scroll to last assistant line |
|
|
||||||
| `ui/account.js` | `Account`: HF login chip + popover, daily-limit modal |
|
|
||||||
| `ui/dom.js` | Shared helpers: strict `$(selector)` that throws on missing, `document.querySelector` fallback, `escHtml`, `truncateError`, `DEBUG` from URL |
|
|
||||||
| `auth.py` | HF OAuth + per-request identity (tier, hashed keys) |
|
|
||||||
| `limiter.py` | SQLite per-day talk-time budget (chunked server-clock reservation) |
|
|
||||||
| `ws/s2s-ws-client.js` | WebSocket handshake + OpenAI Realtime GA protocol |
|
|
||||||
| `ws/codec.js` | base64 <-> PCM helpers + transcript extraction (pure) |
|
|
||||||
| `ws/orb-visualizer.js` | `OrbVisualiser`: FFT bands -> orb CSS custom properties (violet/pink/cyan brand gradient) |
|
|
||||||
| `worklets/mic-capture.js` | AudioWorklet: 48 kHz Float32 -> 16 kHz Int16 PCM, posts ~40 ms chunks |
|
|
||||||
| `worklets/audio-playback.js` | AudioWorklet: 24 kHz Float32 ring buffer -> 48 kHz, linear interp, fade in/out |
|
|
||||||
| `style.css` | Orb animations, layout, dark aurora gradient theme, glass chat bubbles, sticky footer, reports sidebar desktop left 320px / mobile offcanvas, user profile 2-col grid modal |
|
|
||||||
|
|
||||||
## Audio pipeline notes
|
|
||||||
|
|
||||||
- **Input**: `getUserMedia({ echoCancellation, noiseSuppression, autoGainControl })`
|
|
||||||
feeds the `mic-capture` worklet at the `AudioContext` rate. The worklet
|
|
||||||
resamples to 16 kHz (boxcar lowpass + decimation on the 48 -> 16 fast
|
|
||||||
path, linear interpolation fallback for odd rates) and packs Int16 LE.
|
|
||||||
- **Output**: `response.output_audio.delta` decodes to Int16 -> Float32
|
|
||||||
and is posted to the `audio-playback` worklet. The worklet maintains a
|
|
||||||
per-context ring buffer, linearly interpolates 24 -> 48, and applies
|
|
||||||
short 32-frame fades on entry/exit to suppress clicks.
|
|
||||||
- **Barge-in**: when the server VAD detects user speech mid-response
|
|
||||||
(`input_audio_buffer.speech_started` while `ai-speaking`), the client
|
|
||||||
posts `{ kind: "clear" }` to the playback worklet to wipe the queue
|
|
||||||
immediately. The server itself cancels the in-flight response.
|
|
||||||
|
|
||||||
## Credits
|
|
||||||
|
|
||||||
@@ -1,229 +0,0 @@
|
|||||||
"""
|
|
||||||
HF OAuth + per-request identity for the duration limiter.
|
|
||||||
|
|
||||||
Login uses Hugging Face's native Spaces OAuth via `huggingface_hub`
|
|
||||||
(`attach_huggingface_oauth` / `parse_huggingface_oauth`). The OAuth env
|
|
||||||
(`OAUTH_CLIENT_ID`, ...) is injected by the platform when the Space README sets
|
|
||||||
`hf_oauth: true`, so this only activates on a deployed Space — locally and in
|
|
||||||
direct mode there's no OAuth and the limiter treats everyone as anonymous.
|
|
||||||
|
|
||||||
Identity:
|
|
||||||
- signed in -> tier 'pro' | 'free', keyed by hashed HF `sub`
|
|
||||||
- anonymous -> tier 'anon', keyed by BOTH hashed client IP and a hashed
|
|
||||||
signed-cookie id (OR-matched in the limiter)
|
|
||||||
"""
|
|
||||||
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
import secrets
|
|
||||||
|
|
||||||
import limiter
|
|
||||||
|
|
||||||
logger = logging.getLogger("s2s.auth")
|
|
||||||
|
|
||||||
# huggingface_hub adds these routes when OAuth is attached. Centralised so a
|
|
||||||
# version change is a one-line fix; the paths are handed to the client via
|
|
||||||
# /api/me rather than hardcoded there.
|
|
||||||
OAUTH_LOGIN_PATH = "/oauth/huggingface/login"
|
|
||||||
OAUTH_LOGOUT_PATH = "/oauth/huggingface/logout"
|
|
||||||
|
|
||||||
ANON_COOKIE = "s2s_anon"
|
|
||||||
_COOKIE_MAX_AGE = 60 * 60 * 24 * 30 # 30 days
|
|
||||||
|
|
||||||
|
|
||||||
# Members of these orgs get unlimited usage (like PRO) out of the box. The
|
|
||||||
# UNLIMITED_ORGS env adds to this set; it doesn't replace it.
|
|
||||||
_DEFAULT_UNLIMITED_ORGS = {"cerebras", "huggingfacem4", "smolagents", "pollen-robotics"}
|
|
||||||
|
|
||||||
|
|
||||||
def _unlimited_orgs() -> "set[str]":
|
|
||||||
"""Org usernames whose members get unlimited usage (like PRO).
|
|
||||||
|
|
||||||
Defaults to {cerebras, HuggingFaceM4, smolagents}; the UNLIMITED_ORGS env
|
|
||||||
(comma/space-separated, e.g. `UNLIMITED_ORGS=my-team`) adds more. Matched
|
|
||||||
case-insensitively against the signed-in user's organisations."""
|
|
||||||
raw = os.environ.get("UNLIMITED_ORGS", "")
|
|
||||||
extra = {o.strip().lower() for o in raw.replace(",", " ").split() if o.strip()}
|
|
||||||
return _DEFAULT_UNLIMITED_ORGS | extra
|
|
||||||
|
|
||||||
try:
|
|
||||||
from huggingface_hub import attach_huggingface_oauth, parse_huggingface_oauth
|
|
||||||
_OAUTH_IMPORTABLE = True
|
|
||||||
except Exception as exc: # pragma: no cover - import guard
|
|
||||||
logger.info("huggingface_hub OAuth unavailable (%s); sign-in disabled.", exc)
|
|
||||||
_OAUTH_IMPORTABLE = False
|
|
||||||
|
|
||||||
# Set by attach(): True once OAuth is actually wired (importable + env present).
|
|
||||||
oauth_enabled = False
|
|
||||||
|
|
||||||
|
|
||||||
def attach(app) -> bool:
|
|
||||||
"""Wire HF OAuth onto the app if it's importable and configured. Returns
|
|
||||||
whether sign-in is available."""
|
|
||||||
global oauth_enabled
|
|
||||||
if not _OAUTH_IMPORTABLE or not os.environ.get("OAUTH_CLIENT_ID"):
|
|
||||||
return False
|
|
||||||
try:
|
|
||||||
attach_huggingface_oauth(app)
|
|
||||||
oauth_enabled = True
|
|
||||||
logger.info("HF OAuth attached (sign-in enabled).")
|
|
||||||
except Exception as exc: # pragma: no cover - defensive
|
|
||||||
logger.warning("Failed to attach HF OAuth: %r", exc)
|
|
||||||
oauth_enabled = False
|
|
||||||
return oauth_enabled
|
|
||||||
|
|
||||||
|
|
||||||
def _field(obj, name, default=None):
|
|
||||||
"""Read a field whether the user-info is an object or a dict."""
|
|
||||||
if obj is None:
|
|
||||||
return default
|
|
||||||
if isinstance(obj, dict):
|
|
||||||
return obj.get(name, default)
|
|
||||||
return getattr(obj, name, default)
|
|
||||||
|
|
||||||
|
|
||||||
# Surface what we detect (orgs, tier) in logs and on /api/me when set. Handy for
|
|
||||||
# verifying org gating on the live Space without guessing.
|
|
||||||
AUTH_DEBUG = bool(os.environ.get("AUTH_DEBUG"))
|
|
||||||
|
|
||||||
# whoami-v2 org lookups are cached for the process lifetime, keyed by token, so
|
|
||||||
# /api/me + /api/session don't each hit the Hub.
|
|
||||||
_orgs_cache: "dict[str, set[str]]" = {}
|
|
||||||
|
|
||||||
|
|
||||||
def current_oauth(request):
|
|
||||||
"""The parsed HF OAuth info (user_info + access_token), or None."""
|
|
||||||
if not oauth_enabled:
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
return parse_huggingface_oauth(request)
|
|
||||||
except Exception:
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def current_user(request):
|
|
||||||
"""The signed-in HF user-info, or None."""
|
|
||||||
return _field(current_oauth(request), "user_info")
|
|
||||||
|
|
||||||
|
|
||||||
def _user_org_names(user) -> "set[str]":
|
|
||||||
"""The user's organisations from the OAuth userinfo, by username/name/id."""
|
|
||||||
names = set()
|
|
||||||
for org in _field(user, "orgs", []) or []:
|
|
||||||
for key in ("preferred_username", "name", "sub"):
|
|
||||||
val = _field(org, key)
|
|
||||||
if val:
|
|
||||||
names.add(str(val).lower())
|
|
||||||
return names
|
|
||||||
|
|
||||||
|
|
||||||
def _orgs_via_token(token: str) -> "set[str]":
|
|
||||||
"""Fallback org lookup via the Hub `whoami-v2` API, using the user's OAuth
|
|
||||||
access token. Covers the case where the userinfo claim omits `orgs`."""
|
|
||||||
if not token:
|
|
||||||
return set()
|
|
||||||
if token in _orgs_cache:
|
|
||||||
return _orgs_cache[token]
|
|
||||||
names: "set[str]" = set()
|
|
||||||
try:
|
|
||||||
import httpx
|
|
||||||
|
|
||||||
resp = httpx.get(
|
|
||||||
"https://huggingface.co/api/whoami-v2",
|
|
||||||
headers={"Authorization": f"Bearer {token}"},
|
|
||||||
timeout=5.0,
|
|
||||||
)
|
|
||||||
resp.raise_for_status()
|
|
||||||
for org in resp.json().get("orgs", []) or []:
|
|
||||||
for key in ("name", "fullname"):
|
|
||||||
val = org.get(key)
|
|
||||||
if val:
|
|
||||||
names.add(str(val).lower())
|
|
||||||
except Exception as exc: # pragma: no cover - network/permission dependent
|
|
||||||
logger.info("whoami-v2 org lookup failed: %r", exc)
|
|
||||||
_orgs_cache[token] = names
|
|
||||||
return names
|
|
||||||
|
|
||||||
|
|
||||||
def _org_names(user, token=None, allow=None) -> "set[str]":
|
|
||||||
"""The user's org usernames from the OAuth userinfo claim. If that doesn't
|
|
||||||
already satisfy `allow`, fall back to the Hub `whoami-v2` API (the claim is
|
|
||||||
often empty or partial), so membership is resolved either way."""
|
|
||||||
names = _user_org_names(user)
|
|
||||||
if token and (allow is None or not (allow & names)):
|
|
||||||
names = names | _orgs_via_token(token)
|
|
||||||
return names
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_tier(user, token=None) -> str:
|
|
||||||
"""Tier for a signed-in user: 'pro' (paying), 'org' (allow-listed org
|
|
||||||
member, unlimited), or 'free'. PRO wins over org if both apply."""
|
|
||||||
if bool(_field(user, "is_pro", False)):
|
|
||||||
return "pro"
|
|
||||||
allow = _unlimited_orgs()
|
|
||||||
names = _org_names(user, token, allow)
|
|
||||||
tier = "org" if (allow & names) else "free"
|
|
||||||
if AUTH_DEBUG:
|
|
||||||
logger.info("tier=%s orgs=%s allow=%s", tier, sorted(names), sorted(allow))
|
|
||||||
return tier
|
|
||||||
|
|
||||||
|
|
||||||
def user_view(request) -> dict:
|
|
||||||
"""Public profile for /api/me."""
|
|
||||||
info = current_oauth(request)
|
|
||||||
user = _field(info, "user_info")
|
|
||||||
if not user:
|
|
||||||
return {"loggedIn": False, "tier": "anon"}
|
|
||||||
token = _field(info, "access_token")
|
|
||||||
out = {
|
|
||||||
"loggedIn": True,
|
|
||||||
"username": _field(user, "preferred_username") or _field(user, "name") or "you",
|
|
||||||
"avatar": _field(user, "picture"),
|
|
||||||
"tier": resolve_tier(user, token),
|
|
||||||
}
|
|
||||||
if AUTH_DEBUG:
|
|
||||||
out["orgs"] = sorted(_org_names(user, token))
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _client_ip(request) -> str:
|
|
||||||
"""Real client IP. On HF the app sits behind a proxy, so the user's address
|
|
||||||
is the first hop in X-Forwarded-For, not request.client.host."""
|
|
||||||
xff = request.headers.get("x-forwarded-for", "")
|
|
||||||
if xff:
|
|
||||||
return xff.split(",")[0].strip()
|
|
||||||
return request.client.host if request.client else "unknown"
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_identity(request):
|
|
||||||
"""Resolve (tier, keys, set_cookie) for this request.
|
|
||||||
|
|
||||||
`keys` are the limiter usage_daily keys to debit (one for signed-in, two for
|
|
||||||
anonymous). `set_cookie` is a signed value to Set-Cookie when we minted a new
|
|
||||||
anonymous id, else None.
|
|
||||||
"""
|
|
||||||
info = current_oauth(request)
|
|
||||||
user = _field(info, "user_info")
|
|
||||||
if user:
|
|
||||||
sub = _field(user, "sub") or _field(user, "preferred_username")
|
|
||||||
token = _field(info, "access_token")
|
|
||||||
return resolve_tier(user, token), [limiter.hash_key(f"sub:{sub}")], None
|
|
||||||
|
|
||||||
# Anonymous: key by IP and a signed cookie id, minting the cookie if absent.
|
|
||||||
ip = _client_ip(request)
|
|
||||||
cookie_id = limiter.verify_cookie(request.cookies.get(ANON_COOKIE, ""))
|
|
||||||
set_cookie = None
|
|
||||||
if not cookie_id:
|
|
||||||
cookie_id = secrets.token_urlsafe(18)
|
|
||||||
set_cookie = limiter.sign_cookie(cookie_id)
|
|
||||||
keys = [limiter.hash_key(f"ip:{ip}"), limiter.hash_key(f"cookie:{cookie_id}")]
|
|
||||||
return "anon", keys, set_cookie
|
|
||||||
|
|
||||||
|
|
||||||
def set_anon_cookie(response, signed: str) -> None:
|
|
||||||
# The Space runs inside an iframe on huggingface.co, so the cookie lives in a
|
|
||||||
# cross-site context — it must be SameSite=None; Secure or the browser drops it.
|
|
||||||
response.set_cookie(
|
|
||||||
ANON_COOKIE, signed,
|
|
||||||
max_age=_COOKIE_MAX_AGE, httponly=True, samesite="none", secure=True,
|
|
||||||
)
|
|
||||||
@@ -1,728 +0,0 @@
|
|||||||
<!doctype html>
|
|
||||||
<html lang="en">
|
|
||||||
<head>
|
|
||||||
<meta charset="UTF-8" />
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
|
|
||||||
<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate, max-age=0" />
|
|
||||||
<meta http-equiv="Pragma" content="no-cache" />
|
|
||||||
<meta http-equiv="Expires" content="0" />
|
|
||||||
<title>Minimal Conversation · S2S backend (WebSocket)</title>
|
|
||||||
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
|
||||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
|
|
||||||
<link
|
|
||||||
href="https://fonts.googleapis.com/css2?family=Geist+Mono:wght@400;500;600&family=Inter:wght@400;500;600;700;800&display=swap"
|
|
||||||
rel="stylesheet"
|
|
||||||
/>
|
|
||||||
<!-- ═══════════════════════════════════════════════════════════════════
|
|
||||||
ANTI-FOUC VEIL (S2S_THEME flash fix) — MUST stay INLINE and
|
|
||||||
BEFORE the <link rel="stylesheet" href="style.css"> tag below.
|
|
||||||
|
|
||||||
Problem reported by user 2026-09-10:
|
|
||||||
"quando faccio refresh per un momento si vede il theme precedente
|
|
||||||
quello purple"
|
|
||||||
|
|
||||||
The earlier guard `html:not([data-theme]) [data-theme-only]` only
|
|
||||||
hid the two parallel widget branches, but the UNTHEMED DEFAULTS in
|
|
||||||
style.css (:root purple palette, body background #07080d, topbar
|
|
||||||
glass, #app deep-space colours, purple orb border gradients) were
|
|
||||||
still rendering for the ~1-2 frames between body paint commit and
|
|
||||||
fetchConfig() returning api/config.theme → setAttribute("data-theme").
|
|
||||||
The user saw a dark-purple flash before the white Vocero theme
|
|
||||||
applied — exactly the symptom described.
|
|
||||||
|
|
||||||
FIX (3 layers, fail-safe):
|
|
||||||
1. While html has NO data-theme attribute (pre-fetchConfig),
|
|
||||||
render <body> COMPLETELY INVISIBLE + background:white +
|
|
||||||
border:none (the two most visible paint leaks fixed first).
|
|
||||||
Display:none would block CSS transitions from being recorded;
|
|
||||||
visibility:hidden keeps layout but forbids painting entirely,
|
|
||||||
which is exactly what we want.
|
|
||||||
2. Safety timeout: if fetchConfig() fails to resolve (e.g.
|
|
||||||
static hosting / no server / offline) the veil auto-lifts
|
|
||||||
after 2500 ms so the user is NOT stuck on a blank page.
|
|
||||||
3. Immediately after root.setAttribute("data-theme", …) in
|
|
||||||
fetchConfig() we also add class="theme-ready" which the veil
|
|
||||||
additionally checks so the user sees a 1-frame lift, not a
|
|
||||||
delayed paint. The visibility guard is a double-OR so it
|
|
||||||
expires on EITHER signal.
|
|
||||||
═══════════════════════════════════════════════════════════════════ -->
|
|
||||||
<style id="s2s-anti-fouc-veil">
|
|
||||||
html:not([data-theme]) body,
|
|
||||||
html:not(.theme-ready) body {
|
|
||||||
visibility: hidden !important;
|
|
||||||
background: #ffffff !important;
|
|
||||||
color: #0f172a !important;
|
|
||||||
opacity: 0 !important;
|
|
||||||
transition: none !important;
|
|
||||||
animation: none !important;
|
|
||||||
}
|
|
||||||
html:not([data-theme]),
|
|
||||||
html:not(.theme-ready) {
|
|
||||||
background: #ffffff !important;
|
|
||||||
}
|
|
||||||
/* Also kill the purple orb + any glass chrome during the veil window,
|
|
||||||
because even when <body> is visibility:hidden some browsers do a
|
|
||||||
first compositing pass that leaks a few pixels of background. */
|
|
||||||
html:not([data-theme]) #main-circle,
|
|
||||||
html:not([data-theme]) .orb-wrap,
|
|
||||||
html:not([data-theme]) .stage,
|
|
||||||
html:not([data-theme]) .topbar,
|
|
||||||
html:not([data-theme]) #app,
|
|
||||||
html:not(.theme-ready) #main-circle,
|
|
||||||
html:not(.theme-ready) .orb-wrap,
|
|
||||||
html:not(.theme-ready) .stage,
|
|
||||||
html:not(.theme-ready) .topbar,
|
|
||||||
html:not(.theme-ready) #app {
|
|
||||||
background: transparent !important;
|
|
||||||
background-image: none !important;
|
|
||||||
backdrop-filter: none !important;
|
|
||||||
-webkit-backdrop-filter: none !important;
|
|
||||||
box-shadow: none !important;
|
|
||||||
border: none !important;
|
|
||||||
color: transparent !important;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
<script>
|
|
||||||
// Fail-safe veil lift: if fetchConfig() fails to reply (static hosting,
|
|
||||||
// offline, api/config 500) the user never sees <body> stuck invisible.
|
|
||||||
// setTimeout (not rAF) because we want a real wall-clock timeout.
|
|
||||||
window.__s2sVeilLiftTimeout = window.setTimeout(function () {
|
|
||||||
var r = document.documentElement;
|
|
||||||
if (!r.hasAttribute("data-theme")) r.setAttribute("data-theme", "dark");
|
|
||||||
r.classList.add("theme-ready");
|
|
||||||
}, 2500);
|
|
||||||
// Also try to clear the fail-safe as soon as DOMContentLoaded fires so
|
|
||||||
// we don't accidentally fire the lift AFTER a normal-config path.
|
|
||||||
document.addEventListener("DOMContentLoaded", function () {
|
|
||||||
var r = document.documentElement;
|
|
||||||
if (r.hasAttribute("data-theme") && !r.classList.contains("theme-ready")) {
|
|
||||||
r.classList.add("theme-ready");
|
|
||||||
if (window.__s2sVeilLiftTimeout) window.clearTimeout(window.__s2sVeilLiftTimeout);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
</script>
|
|
||||||
<link rel="stylesheet" href="style.css?v=202609291430" />
|
|
||||||
<script>
|
|
||||||
document.addEventListener("DOMContentLoaded", () => {
|
|
||||||
const l = document.querySelector('link[rel="stylesheet"]');
|
|
||||||
if (l && !l.href.includes("v=202609291430")) {
|
|
||||||
l.href = l.href.replace(/v=\d+/, "v=202609291430");
|
|
||||||
}
|
|
||||||
});
|
|
||||||
</script>
|
|
||||||
</head>
|
|
||||||
<!--
|
|
||||||
`booting` disables transitions/animations until the first paint commits.
|
|
||||||
Without this the orb visibly fades/scales in on the very first frame
|
|
||||||
because `state-idle` and the generic `.ind` defaults differ. The
|
|
||||||
bootstrap script at the end of main.js strips the class after one rAF.
|
|
||||||
-->
|
|
||||||
<body class="booting">
|
|
||||||
<div id="talk-unlocker" class="talk-unlocker" hidden data-debug-only-never>
|
|
||||||
<div class="talk-unlocker-hint">
|
|
||||||
<div class="talk-unlocker-spinner" aria-hidden="true"></div>
|
|
||||||
<p class="talk-unlocker-title">Preparando Vocero…</p>
|
|
||||||
<p class="talk-unlocker-sub">Toca en cualquier lugar para empezar a hablar</p>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div id="app">
|
|
||||||
<header class="topbar">
|
|
||||||
<!-- Hamburger (theme=white only): icon button 3 horizontal lines, sits left of brand cluster, top-left corner, square icon, dark text, no visible borders.
|
|
||||||
Mirrors the 3-line menu icon in the screenshot header. -->
|
|
||||||
<button id="menu-btn" class="icon-btn menu-btn" type="button" title="Menú" aria-label="Abrir menú" hidden data-theme-only="white">
|
|
||||||
<svg width="26" height="26" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round" stroke-linejoin="round">
|
|
||||||
<line x1="3" y1="6" x2="21" y2="6"/>
|
|
||||||
<line x1="3" y1="12" x2="21" y2="12"/>
|
|
||||||
<line x1="3" y1="18" x2="21" y2="18"/>
|
|
||||||
</svg>
|
|
||||||
</button>
|
|
||||||
<div class="brand">
|
|
||||||
<div class="ident">
|
|
||||||
<div class="ident-head">
|
|
||||||
<a class="ident-title">Vocero</a>
|
|
||||||
<!-- Subtitle shown below Vocero title (theme=white only) -->
|
|
||||||
<p class="ident-subtitle" hidden data-theme-only="white">Agente conversacional</p>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<!-- User avatar circle (theme=white only): grid item next to the brand, not
|
|
||||||
absolute-positioned → eliminates overlap with title/subtitle entirely.
|
|
||||||
Same element id so main.js syncVoceroHeaderUser() still fills the text. -->
|
|
||||||
<div id="ident-user-name" class="ident-user-name" hidden data-theme-only="white" aria-hidden="true">AV</div>
|
|
||||||
<!-- Left sidebar (reports) toggle: always visible, pinned left side of the
|
|
||||||
topbar control cluster. Clicking slides in the offcanvas sidebar on
|
|
||||||
mobile; it's a no-op hover hint on desktop where sidebar is always
|
|
||||||
shown. -->
|
|
||||||
<button id="reports-sidebar-toggle" class="icon-btn" title="Reports programados" aria-label="Abrir panel de reportes" aria-expanded="false" aria-controls="reports-sidebar" hidden data-reports-sidebar-toggle data-debug-only="true" data-theme-only="dark">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M3 3h18v18H3z" fill="none"/><line x1="9" y1="3" x2="9" y2="21"/></svg>
|
|
||||||
<span id="reports-sidebar-badge" class="chat-badge" aria-hidden="true"></span>
|
|
||||||
</button>
|
|
||||||
<div class="topbar-right" hidden data-debug-only="true">
|
|
||||||
<!-- HF login chip / sign-in pill. Populated by ui/account.js; only
|
|
||||||
shown when the deploy runs behind a load balancer with OAuth. -->
|
|
||||||
<div id="account" class="account" hidden></div>
|
|
||||||
<button id="about-btn-m" class="icon-btn about-btn-mobile" title="About this space" aria-label="About this space">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"/><line x1="12" y1="16" x2="12" y2="12"/><line x1="12" y1="8" x2="12.01" y2="8"/></svg>
|
|
||||||
</button>
|
|
||||||
<button id="tools-btn" class="icon-btn" title="Tools" aria-label="Tools">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/></svg>
|
|
||||||
</button>
|
|
||||||
<button id="rag-btn" class="icon-btn" title="Custom RAG context" aria-label="Custom RAG context">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M4 19.5A2.5 2.5 0 0 1 6.5 17H20"/><path d="M6.5 2H20v20H6.5A2.5 2.5 0 0 1 4 19.5v-15A2.5 2.5 0 0 1 6.5 2z"/></svg>
|
|
||||||
</button>
|
|
||||||
<button id="settings-btn" class="icon-btn" title="Settings" aria-label="Settings">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="3"/><path d="M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 1 1-2.83 2.83l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 1 1-4 0v-.09A1.65 1.65 0 0 0 9 19.4a1.65 1.65 0 0 0-1.82.33l-.06.06a2 2 0 1 1-2.83-2.83l.06-.06a1.65 1.65 0 0 0 .33-1.82 1.65 1.65 0 0 0-1.51-1H3a2 2 0 1 1 0-4h.09A1.65 1.65 0 0 0 4.6 9a1.65 1.65 0 0 0-.33-1.82l-.06-.06a2 2 0 1 1 2.83-2.83l.06.06a1.65 1.65 0 0 0 1.82.33H9a1.65 1.65 0 0 0 1-1.51V3a2 2 0 1 1 4 0v.09a1.65 1.65 0 0 0 1 1.51 1.65 1.65 0 0 0 1.82-.33l.06-.06a2 2 0 1 1 2.83 2.83l-.06.06a1.65 1.65 0 0 0-.33 1.82V9c0 .66.39 1.25 1 1.51H21a2 2 0 1 1 0 4h-.09a1.65 1.65 0 0 0-1.51 1z"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<a class="feedback-link" href="https://tally.so/r/pb8LWq" target="_blank" rel="noopener noreferrer">Enviar feedback</a>
|
|
||||||
|
|
||||||
<!-- Theme=white only: Status card between topbar and stage
|
|
||||||
"Listo para conversar / Informes, análisis, datos y mucho más."
|
|
||||||
with light-blue chart icon and chevron-right arrow. -->
|
|
||||||
<div class="status-card" hidden data-theme-only="white" aria-label="Estado de la sesión">
|
|
||||||
<div class="status-card-icon" aria-hidden="true">
|
|
||||||
<svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.4" stroke-linecap="round" stroke-linejoin="round">
|
|
||||||
<line x1="6" y1="20" x2="6" y2="12"/>
|
|
||||||
<line x1="11" y1="20" x2="11" y2="6"/>
|
|
||||||
<line x1="16" y1="20" x2="16" y2="15"/>
|
|
||||||
<line x1="21" y1="20" x2="21" y2="10"/>
|
|
||||||
</svg>
|
|
||||||
</div>
|
|
||||||
<div class="status-card-text">
|
|
||||||
<h2 class="status-card-title">Listo para conversar</h2>
|
|
||||||
<p class="status-card-sub">Informes, análisis, datos y mucho más.</p>
|
|
||||||
</div>
|
|
||||||
<button class="status-card-arrow icon-btn" type="button" aria-label="Ver más detalles" title="Detalles">
|
|
||||||
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.4" stroke-linecap="round" stroke-linejoin="round">
|
|
||||||
<polyline points="9 18 15 12 9 6"></polyline>
|
|
||||||
</svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<main class="stage">
|
|
||||||
<!-- DARK THEME orb (purple gradient + purple icons): shown only on native dark theme, hidden in white -->
|
|
||||||
<div class="orb-wrap" data-theme-only="dark">
|
|
||||||
<div id="mic-gate" class="mic-gate">
|
|
||||||
<svg id="mic-gate-arc" class="mic-gate-arc" viewBox="0 0 100 100" aria-hidden="true">
|
|
||||||
<path id="mga-track" class="mga-track" fill="none" />
|
|
||||||
<path id="mga-fill" class="mga-fill" fill="none" />
|
|
||||||
<path id="mga-hit" class="mga-hit" fill="none" />
|
|
||||||
<circle id="mga-handle" class="mga-handle" r="3" />
|
|
||||||
</svg>
|
|
||||||
<button id="mic-btn" class="side-btn" type="button" aria-label="Mute" title="Mute" aria-hidden="true">
|
|
||||||
<svg class="mic-on" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="2" width="6" height="12" rx="3"/><path d="M5 10a7 7 0 0 0 14 0"/><line x1="12" y1="19" x2="12" y2="22"/></svg>
|
|
||||||
<svg class="mic-off" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><line x1="2" y1="2" x2="22" y2="22"/><path d="M9 5a3 3 0 0 1 6 0v4"/><path d="M9 10v1a3 3 0 0 0 5.1 2.1"/><path d="M19 10a7 7 0 0 1-1.24 3.97"/><path d="M5 10a7 7 0 0 0 11 5.67"/><line x1="12" y1="19" x2="12" y2="22"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<button
|
|
||||||
id="main-circle"
|
|
||||||
class="circle state-idle"
|
|
||||||
type="button"
|
|
||||||
aria-label="Start voice conversation"
|
|
||||||
>
|
|
||||||
<span class="circle-glow" aria-hidden="true"></span>
|
|
||||||
<span class="circle-ring" aria-hidden="true"></span>
|
|
||||||
<span class="circle-ring-outer" aria-hidden="true"></span>
|
|
||||||
<span class="circle-core">
|
|
||||||
<span class="circle-indicator" aria-hidden="true">
|
|
||||||
<svg class="ind ind-mic" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<rect x="9" y="2" width="6" height="12" rx="3" fill="currentColor" stroke="none"/>
|
|
||||||
<path d="M5 10a7 7 0 0 0 14 0"/>
|
|
||||||
<line x1="12" y1="19" x2="12" y2="22"/>
|
|
||||||
<line x1="8" y1="22" x2="16" y2="22"/>
|
|
||||||
</svg>
|
|
||||||
<svg class="ind ind-error" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="9"/><line x1="12" y1="8" x2="12" y2="13"/><line x1="12" y1="16" x2="12" y2="16"/></svg>
|
|
||||||
<span class="ind ind-spinner"></span>
|
|
||||||
<span class="ind ind-thinking">
|
|
||||||
<span class="dot"></span>
|
|
||||||
<span class="dot"></span>
|
|
||||||
<span class="dot"></span>
|
|
||||||
</span>
|
|
||||||
<span class="ind ind-bars">
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
</span>
|
|
||||||
<svg class="ind ind-voice" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<path d="M3 10v4a1 1 0 0 0 1 1h3l5 4V5L7 9H4a1 1 0 0 0-1 1z" fill="currentColor" stroke="none"/>
|
|
||||||
<path class="wave wave-1" d="M16 8a5 5 0 0 1 0 8"/>
|
|
||||||
<path class="wave wave-2" d="M19 5a9 9 0 0 1 0 14"/>
|
|
||||||
</svg>
|
|
||||||
</span>
|
|
||||||
</span>
|
|
||||||
</button>
|
|
||||||
|
|
||||||
<button id="stop-btn" class="side-btn" type="button" aria-label="End" title="End" aria-hidden="true">
|
|
||||||
<svg viewBox="0 0 24 24" fill="currentColor"><rect x="6" y="6" width="12" height="12" rx="2"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- WHITE THEME mic stage (replaces the purple orb)
|
|
||||||
- Blue mic circular button (116 px, deep blue, white mic)
|
|
||||||
- Two waveform banks left + right (audio-reactive CSS bars)
|
|
||||||
- "Escuchando..." status text
|
|
||||||
- "Detener" pill button below.
|
|
||||||
Mirrors the screenshot center UI exactly. -->
|
|
||||||
<div class="white-mic-stage" hidden data-theme-only="white" aria-label="Control de voz">
|
|
||||||
<!-- Waveform left side: 7 bars mirror right side, heights driven by JS vars -->
|
|
||||||
<div class="white-waveform white-waveform-left" aria-hidden="true">
|
|
||||||
<span class="w-bar wb-1" style="--bar-h: 0.4"></span>
|
|
||||||
<span class="w-bar wb-2" style="--bar-h: 0.65"></span>
|
|
||||||
<span class="w-bar wb-3" style="--bar-h: 0.9"></span>
|
|
||||||
<span class="w-bar wb-4" style="--bar-h: 1"></span>
|
|
||||||
<span class="w-bar wb-5" style="--bar-h: 0.7"></span>
|
|
||||||
<span class="w-bar wb-6" style="--bar-h: 0.55"></span>
|
|
||||||
<span class="w-bar wb-7" style="--bar-h: 0.3"></span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Big circular mic button (theme=white). Click = start/stop voice session.
|
|
||||||
Forwards all its clicks to the real #main-circle so the entire WS
|
|
||||||
pipeline still works. -->
|
|
||||||
<button id="white-mic-circle"
|
|
||||||
class="white-mic-circle state-idle"
|
|
||||||
type="button"
|
|
||||||
aria-label="Iniciar conversación de voz">
|
|
||||||
<span class="wmc-halo" aria-hidden="true"></span>
|
|
||||||
<span class="wmc-ring" aria-hidden="true"></span>
|
|
||||||
<span class="wmc-core" aria-hidden="true"></span>
|
|
||||||
<svg class="wmc-icon wmc-mic" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<rect x="9" y="2" width="6" height="12" rx="3" fill="currentColor" stroke="none"/>
|
|
||||||
<path d="M5 10a7 7 0 0 0 14 0"/>
|
|
||||||
<line x1="12" y1="19" x2="12" y2="22"/>
|
|
||||||
<line x1="8" y1="22" x2="16" y2="22"/>
|
|
||||||
</svg>
|
|
||||||
<svg class="wmc-icon wmc-stop" viewBox="0 0 24 24" fill="currentColor" aria-hidden="true"><rect x="6" y="6" width="12" height="12" rx="2"/></svg>
|
|
||||||
<span class="wmc-icon wmc-spinner" aria-hidden="true"></span>
|
|
||||||
</button>
|
|
||||||
|
|
||||||
<!-- Waveform right side: 7 bars mirroring left -->
|
|
||||||
<div class="white-waveform white-waveform-right" aria-hidden="true">
|
|
||||||
<span class="w-bar wb-1" style="--bar-h: 0.3"></span>
|
|
||||||
<span class="w-bar wb-2" style="--bar-h: 0.55"></span>
|
|
||||||
<span class="w-bar wb-3" style="--bar-h: 0.7"></span>
|
|
||||||
<span class="w-bar wb-4" style="--bar-h: 1"></span>
|
|
||||||
<span class="w-bar wb-5" style="--bar-h: 0.9"></span>
|
|
||||||
<span class="w-bar wb-6" style="--bar-h: 0.65"></span>
|
|
||||||
<span class="w-bar wb-7" style="--bar-h: 0.4"></span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Status caption below mic circle: "Escuchando...", "Pensando..." etc. -->
|
|
||||||
<p id="white-mic-caption" class="white-mic-caption" role="status">Escuchando…</p>
|
|
||||||
|
|
||||||
<!-- Secondary pill "Detener" (Stop) below caption, with dark stop-square icon. -->
|
|
||||||
<button id="white-stop-btn"
|
|
||||||
class="white-stop-btn"
|
|
||||||
type="button"
|
|
||||||
aria-label="Detener"
|
|
||||||
title="Detener sesión de voz">
|
|
||||||
<svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true"><rect x="6" y="6" width="12" height="12" rx="2"/></svg>
|
|
||||||
<span>Detener</span>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<p id="circle-caption" class="circle-caption" role="status">Tap to start</p>
|
|
||||||
<p id="circle-subcaption" class="circle-subcaption" hidden></p>
|
|
||||||
|
|
||||||
<div id="queue-actions" class="queue-actions" hidden>
|
|
||||||
<button id="join-queue-btn" class="join-queue-btn" type="button" hidden>
|
|
||||||
Join now
|
|
||||||
</button>
|
|
||||||
<button id="leave-queue-btn" class="leave-queue-btn" type="button" hidden>
|
|
||||||
Leave queue
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</main>
|
|
||||||
|
|
||||||
<!-- Theme=white only: Quick action pills below mic area
|
|
||||||
"Resumen" · "Consultar datos" · "Crear respuesta" each with icon -->
|
|
||||||
<div class="quick-actions" hidden data-theme-only="white" aria-label="Acciones rápidas">
|
|
||||||
<button class="quick-action-btn" type="button" data-action="resumen">
|
|
||||||
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/>
|
|
||||||
<polyline points="14 2 14 8 20 8"/>
|
|
||||||
<line x1="8" y1="13" x2="16" y2="13"/>
|
|
||||||
<line x1="8" y1="17" x2="14" y2="17"/>
|
|
||||||
</svg>
|
|
||||||
<span>Resumen</span>
|
|
||||||
</button>
|
|
||||||
<button class="quick-action-btn" type="button" data-action="consultar-datos">
|
|
||||||
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<line x1="6" y1="20" x2="6" y2="12"/>
|
|
||||||
<line x1="11" y1="20" x2="11" y2="6"/>
|
|
||||||
<line x1="16" y1="20" x2="16" y2="15"/>
|
|
||||||
<line x1="21" y1="20" x2="21" y2="10"/>
|
|
||||||
</svg>
|
|
||||||
<span>Consultar datos</span>
|
|
||||||
</button>
|
|
||||||
<button class="quick-action-btn" type="button" data-action="crear-respuesta">
|
|
||||||
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<line x1="18" y1="2" x2="22" y2="6"/>
|
|
||||||
<path d="M7.5 20.5 19 9l-4-4L3.5 16.5 2 22z"/>
|
|
||||||
</svg>
|
|
||||||
<span>Crear respuesta</span>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Chat di testo SEMPRE VISIBILE (fissa in basso, tra stage e footer).
|
|
||||||
Nota: ora è un blocco diretto del flex-col #app, non più un overlay
|
|
||||||
destro → non c'è più bisogno del bottone toggle in topbar. -->
|
|
||||||
<div id="chat-panel" class="chat-panel chat-panel-inline" aria-label="Chat de texto">
|
|
||||||
<div class="chat-panel-inner">
|
|
||||||
<header class="chat-panel-header">
|
|
||||||
<h3>Conversación</h3>
|
|
||||||
<div class="chat-header-actions" aria-label="Acciones de la conversación">
|
|
||||||
<!-- "En línea" status pill (green dot + text) — theme=white only.
|
|
||||||
Mirrors the screenshot chat header top-right corner. -->
|
|
||||||
<span class="chat-status-pill" hidden data-theme-only="white" role="status" aria-live="polite">
|
|
||||||
<span class="chat-status-dot" aria-hidden="true"></span>
|
|
||||||
<span>En línea</span>
|
|
||||||
</span>
|
|
||||||
<button id="chat-export-btn" class="chat-export-btn icon-btn"
|
|
||||||
type="button"
|
|
||||||
aria-label="Descargar conversación"
|
|
||||||
title="Descargar conversación (JSON + TXT)">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.9" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/>
|
|
||||||
<polyline points="7 10 12 15 17 10"/>
|
|
||||||
<line x1="12" y1="15" x2="12" y2="3"/>
|
|
||||||
</svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</header>
|
|
||||||
<div id="chat-history" class="chat-history" role="log" aria-live="polite" aria-atomic="false"></div>
|
|
||||||
<!-- Text input row — HIDDEN in dark theme per user request 2026-09-04.
|
|
||||||
In white theme VISIBLE with a new design (clip button + rounded input + blue circular send),
|
|
||||||
which mirrors the screenshot footer input bar.
|
|
||||||
Restore the dark-theme input by removing the `hidden` attr + style below. -->
|
|
||||||
<form id="chat-form" class="chat-form" autocomplete="off" spellcheck="false" hidden style="display:none!important">
|
|
||||||
<button id="chat-attach-btn" class="chat-attach-btn icon-btn" type="button" aria-label="Adjuntar archivo" title="Adjuntar">
|
|
||||||
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round">
|
|
||||||
<path d="M21.44 11.05l-9.19 9.19a6 6 0 0 1-8.49-8.49l9.19-9.19a4 4 0 0 1 5.66 5.66l-9.2 9.19a2 2 0 0 1-2.83-2.83l8.49-8.48"/>
|
|
||||||
</svg>
|
|
||||||
</button>
|
|
||||||
<input id="chat-input"
|
|
||||||
class="chat-input"
|
|
||||||
type="text"
|
|
||||||
name="message"
|
|
||||||
placeholder="Escribe un mensaje…"
|
|
||||||
aria-label="Enviar mensaje de texto a Vocero"
|
|
||||||
maxlength="500"
|
|
||||||
/>
|
|
||||||
<button id="chat-send" type="submit" class="chat-send-btn-round" aria-label="Enviar mensaje">
|
|
||||||
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
|
||||||
<line x1="22" y1="2" x2="11" y2="13"/>
|
|
||||||
<polygon points="22 2 15 22 11 13 2 9 22 2"/>
|
|
||||||
</svg>
|
|
||||||
</button>
|
|
||||||
</form>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Ephemeral chat bubbles anchored to the top-right -->
|
|
||||||
<div id="bubble-stack" class="bubble-stack" aria-live="polite" aria-atomic="false"></div>
|
|
||||||
|
|
||||||
<!-- Webcam self-view, shown bottom-left while the camera tool is enabled.
|
|
||||||
HIDDEN BY DEFAULT TODAY: riattivare togliendo hidden + scommentando listener in main.js -->
|
|
||||||
<div id="cam-pip" class="cam-pip" aria-hidden="true" hidden>
|
|
||||||
<video id="cam-video" class="cam-video" autoplay playsinline muted></video>
|
|
||||||
<span class="cam-flash" aria-hidden="true"></span>
|
|
||||||
<span class="cam-label">camera</span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Left sidebar: reports programados.
|
|
||||||
Desktop: fixed left, 320px, always visible (below topbar + above sticky footer).
|
|
||||||
Mobile : off-canvas left overlay, toggled by #reports-sidebar-toggle button in topbar. -->
|
|
||||||
<aside id="reports-sidebar" class="reports-sidebar" aria-label="Reportes programados" hidden data-theme-only="dark">
|
|
||||||
<div id="reports-sidebar-backdrop" class="reports-sidebar-backdrop" data-close-reports-sidebar></div>
|
|
||||||
<div class="reports-sidebar-inner">
|
|
||||||
<header class="reports-sidebar-header">
|
|
||||||
<h3>Reportes</h3>
|
|
||||||
<button id="reports-sidebar-close" class="icon-btn" aria-label="Cerrar panel de reportes" data-close-reports-sidebar>
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
<!-- MOVED from inside .ident in the topbar — same #id so main.js syncVoceroHeaderUser()
|
|
||||||
continues to work with zero JS changes to reports rendering logic. -->
|
|
||||||
<div id="reports-hint" class="reports-hint" hidden data-theme-only="dark"></div>
|
|
||||||
<!-- Debug-only: Spanish suggestions panel (gated behind ?debug=true).
|
|
||||||
Shown as the bottom block inside the left sidebar. -->
|
|
||||||
<div id="reports-suggestions" class="reports-suggestions" hidden data-debug-only="true"></div>
|
|
||||||
</div>
|
|
||||||
</aside>
|
|
||||||
|
|
||||||
<dialog id="about-modal" class="modal about-modal">
|
|
||||||
<div class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>About</h2>
|
|
||||||
<button id="about-close" class="icon-btn" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<!-- General intro to the speech-to-speech project -->
|
|
||||||
<div class="about-intro">
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- The backend, shown as the path a turn actually travels -->
|
|
||||||
<div class="about-pipeline">
|
|
||||||
<p class="pipeline-title">The pipeline</p>
|
|
||||||
<ol class="pipeline">
|
|
||||||
<li class="pipe-endpoint">You speak</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">VAD</span>
|
|
||||||
<span class="pipe-job">detects speech</span>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">STT</span>
|
|
||||||
<span class="pipe-job">transcribes it</span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/nvidia/parakeet-tdt-1.1b" target="_blank" rel="noopener">nvidia/parakeet-tdt-1.1b<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">VLM</span>
|
|
||||||
<span class="pipe-job">analyses and composes the reply <span class="pipe-note">· via <a class="cerebras-credit" href="https://cerebras.ai" target="_blank" rel="noopener"><svg class="cerebras-mark" viewBox="0 0 50 50" aria-hidden="true"><path fill-rule="evenodd" clip-rule="evenodd" d="M29.4186 5.62713C24.2806 5.62713 19.353 7.6682 15.7199 11.3013C12.0868 14.9345 10.0457 19.862 10.0457 25C10.0457 30.1381 12.0868 35.0656 15.7199 38.6988C19.353 42.3319 24.2806 44.373 29.4186 44.373V47.2917C17.1061 47.2917 7.12695 37.3105 7.12695 24.998C7.12695 12.6855 17.104 2.7063 29.4165 2.7063V5.62505L29.4186 5.62713ZM39.3186 13.2875C37.7794 11.9809 35.9971 10.9915 34.0741 10.3761C32.1512 9.76067 30.1256 9.53147 28.1137 9.70166C26.1019 9.87185 24.1435 10.4381 22.3513 11.3677C20.559 12.2974 18.9683 13.5722 17.6704 15.1189C16.3726 16.6655 15.3933 18.4534 14.7888 20.3798C14.1844 22.3062 13.9667 24.3332 14.1483 26.344C14.33 28.3548 14.9073 30.3099 15.8472 32.0968C16.787 33.8838 18.0709 35.4673 19.6249 36.7563L17.7478 38.9938C15.9126 37.4545 14.3987 35.5687 13.2924 33.4442C12.1862 31.3197 11.5092 28.9981 11.3003 26.6119C11.0914 24.2258 11.3545 21.8219 12.0747 19.5374C12.7949 17.253 13.9581 15.1328 15.4978 13.298C17.0373 11.4628 18.9233 9.94877 21.0479 8.84247C23.1726 7.73618 25.4944 7.05923 27.8807 6.8503C30.267 6.64136 32.6711 6.90453 34.9557 7.62477C37.2403 8.34501 39.3607 9.50822 41.1957 11.048L39.3186 13.2875ZM34.6207 15.0459C31.9818 13.6829 28.9113 13.4172 26.0776 14.3068C23.2438 15.1964 20.876 17.1691 19.4895 19.7958C18.1029 22.4224 17.8099 25.4903 18.6741 28.332C19.5384 31.1736 21.4899 33.5589 24.104 34.9688L22.7374 37.5521C19.4721 35.7617 17.0418 32.7592 15.9707 29.1927C14.8996 25.6261 15.2737 21.7815 17.0123 18.4883C18.7509 15.1952 21.7146 12.7176 25.2637 11.5903C28.8129 10.463 32.663 10.7763 35.9832 12.4625L34.6207 15.0459ZM29.4165 17.7896C27.5042 17.7896 25.6702 18.5493 24.318 19.9015C22.9658 21.2537 22.2061 23.0877 22.2061 25C22.2061 26.9124 22.9658 28.7464 24.318 30.0986C25.6702 31.4508 27.5042 32.2105 29.4165 32.2105V35.1313C26.7296 35.1313 24.1526 34.0639 22.2527 32.1639C20.3527 30.2639 19.2853 27.687 19.2853 25C19.2853 22.3131 20.3527 19.7362 22.2527 17.8362C24.1526 15.9362 26.7296 14.8688 29.4165 14.8688V17.7896Z" fill="#F15A29"/><path d="M32.0981 22.5749C31.7875 22.2404 31.4295 21.9534 31.0356 21.7228C30.6925 21.519 30.3014 21.4097 29.9023 21.4062C29.371 21.4062 28.896 21.5041 28.4773 21.6978C28.0725 21.8802 27.7089 22.1425 27.4081 22.469C27.1074 22.7955 26.8758 23.1795 26.7273 23.5978C26.5731 24.0207 26.4981 24.4645 26.4981 24.9124C26.4981 25.3666 26.5731 25.8082 26.7273 26.227C26.877 26.6449 27.1091 27.0286 27.4096 27.3553C27.7102 27.682 28.0733 27.9451 28.4773 28.1291C28.8939 28.3228 29.3731 28.4207 29.9023 28.4207C30.3523 28.4207 30.771 28.3249 31.1564 28.1395C31.5481 27.952 31.8856 27.6707 32.146 27.3228L34.0794 29.4187C33.7877 29.7103 33.4544 29.9624 33.0752 30.1749C32.374 30.566 31.6108 30.8338 30.8189 30.9666C30.4648 31.0207 30.1585 31.0499 29.9023 31.0499C29.0617 31.0552 28.2271 30.9069 27.4398 30.6124C26.6954 30.3371 26.0144 29.914 25.4377 29.3687C24.8643 28.8208 24.4078 28.1624 24.096 27.4332C23.7572 26.6366 23.5883 25.778 23.6002 24.9124C23.6002 23.9874 23.7669 23.1478 24.096 22.3916C24.4085 21.6624 24.8627 21.0041 25.4356 20.4562C26.0148 19.9124 26.696 19.4895 27.4398 19.2145C28.2271 18.92 29.0617 18.7717 29.9023 18.777C30.6419 18.777 31.3856 18.9187 32.1356 19.202C32.8877 19.4895 33.5627 19.952 34.1023 20.5541L32.0981 22.5749Z" fill="#fff"/></svg>Cerebras</a></span></span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/google/gemma-4-31B-it" target="_blank" rel="noopener">google/gemma-4-31B-it<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">TTS</span>
|
|
||||||
<span class="pipe-job">speaks it back</span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice" target="_blank" rel="noopener">Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-endpoint">The orb replies</li>
|
|
||||||
</ol>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<dialog id="settings-modal" class="modal">
|
|
||||||
<form method="dialog" class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Settings</h2>
|
|
||||||
<button class="icon-btn" value="close" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<div class="tab-panels">
|
|
||||||
<div class="tab-panel active" role="tabpanel">
|
|
||||||
<label class="field" id="conn-field">
|
|
||||||
<span id="conn-label">Speech-to-speech server URL</span>
|
|
||||||
<input id="lb-url" type="text" autocomplete="off" spellcheck="false" placeholder="http://localhost:port" />
|
|
||||||
<small id="conn-hint">
|
|
||||||
URL of your speech-to-speech server, e.g.
|
|
||||||
<code>http://localhost:8080</code> (the app adds <code>/v1/realtime</code>).
|
|
||||||
</small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<div class="field-row">
|
|
||||||
<label class="field">
|
|
||||||
<span>Voice</span>
|
|
||||||
<select id="voice">
|
|
||||||
<option value="Aiden" selected>Aiden</option>
|
|
||||||
<option value="Ryan">Ryan</option>
|
|
||||||
<option value="Dylan">Dylan</option>
|
|
||||||
<option value="Eric">Eric</option>
|
|
||||||
<option value="Ono_Anna">Ono_Anna</option>
|
|
||||||
<option value="Serena">Serena</option>
|
|
||||||
<option value="Sohee">Sohee</option>
|
|
||||||
<option value="Uncle_Fu">Uncle_Fu</option>
|
|
||||||
<option value="Vivian">Vivian</option>
|
|
||||||
</select>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="field">
|
|
||||||
<span class="field-head">
|
|
||||||
Noise gate
|
|
||||||
<span id="gate-value" class="field-value">Off</span>
|
|
||||||
</span>
|
|
||||||
<div class="gate">
|
|
||||||
<div class="gate-track" aria-hidden="true">
|
|
||||||
<div id="gate-meter-fill" class="gate-meter-fill"></div>
|
|
||||||
<input
|
|
||||||
id="noise-gate"
|
|
||||||
type="range"
|
|
||||||
min="-66"
|
|
||||||
max="-3"
|
|
||||||
step="1"
|
|
||||||
value="-50"
|
|
||||||
aria-label="Noise gate threshold"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
<div class="gate-ends">
|
|
||||||
<span>Off</span>
|
|
||||||
<span>−3 dB</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<small>
|
|
||||||
Mutes the mic below the handle so room noise isn't sent. Slide fully
|
|
||||||
left to turn it off; the bar shows your live input while in a call.
|
|
||||||
</small>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<label class="field">
|
|
||||||
<span>Instructions</span>
|
|
||||||
<textarea id="instructions" rows="5" placeholder="You are Vocero, a friendly voice assistant for the company Aquachile..."></textarea>
|
|
||||||
<small id="primary-prompt-meta" hidden></small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<div class="field">
|
|
||||||
<button id="restart-conversation" type="button" class="btn primary wide" disabled>
|
|
||||||
Restart conversation with these settings
|
|
||||||
</button>
|
|
||||||
<small id="restart-hint">Connect first, then come back here to apply live changes.</small>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<footer class="modal-footer">
|
|
||||||
<button id="settings-save" type="submit" class="btn primary" value="save">Save</button>
|
|
||||||
</footer>
|
|
||||||
</form>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<dialog id="tools-modal" class="modal">
|
|
||||||
<div class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Tools</h2>
|
|
||||||
<button id="tools-close" class="icon-btn" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<p class="tools-intro">Let the assistant act during the conversation. Changes apply live.</p>
|
|
||||||
|
|
||||||
<div class="tool-list">
|
|
||||||
<div class="tool-row" id="tool-web-row">
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Web search</span>
|
|
||||||
<span class="tool-desc">Look things up on Google, via Serper.</span>
|
|
||||||
</div>
|
|
||||||
<label class="switch">
|
|
||||||
<input id="tool-web" type="checkbox" />
|
|
||||||
<span class="switch-track" aria-hidden="true"></span>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
<label class="field tools-key">
|
|
||||||
<span>Search API key</span>
|
|
||||||
<input id="search-key" type="password" autocomplete="off" spellcheck="false" />
|
|
||||||
<small id="tool-web-hint"></small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<!-- Camera tool row — HIDDEN BY DEFAULT TODAY.
|
|
||||||
Per riattivarlo in futuro: togli hidden da div + small,
|
|
||||||
scommenta listener toolCamSwitch + funzioni enableCamera in main.js. -->
|
|
||||||
<div class="tool-row tool-row-sep" id="tool-cam-row" hidden>
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Camera</span>
|
|
||||||
<span class="tool-desc">Let the assistant see through your webcam.</span>
|
|
||||||
</div>
|
|
||||||
<label class="switch">
|
|
||||||
<input id="tool-cam" type="checkbox" />
|
|
||||||
<span class="switch-track" aria-hidden="true"></span>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
<small id="tool-cam-hint" class="tool-hint" hidden>A live preview shows bottom-left while the camera is on.</small>
|
|
||||||
|
|
||||||
<div class="tool-row tool-row-sep">
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Loaded MCP tools</span>
|
|
||||||
<span class="tool-desc">Tools published by the remote agent profile.</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<small id="agent-tools-hint" class="tool-hint"></small>
|
|
||||||
<div id="agent-tools-list" class="agent-tools-list"></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<dialog id="rag-modal" class="modal">
|
|
||||||
<div class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Custom RAG context</h2>
|
|
||||||
<button id="rag-close" class="icon-btn" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<p class="tools-intro">
|
|
||||||
Incolla qui testo, documenti, o estratti RAG che vuoi siano disponibili
|
|
||||||
per la sessione vocale corrente. Il testo viene aggiunto sia alle
|
|
||||||
istruzioni del LLM sia al campo <code>user_context</code> del body
|
|
||||||
<code>/api/reasoning</code>.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<label class="field">
|
|
||||||
<span>RAG context (testo libero, puro)</span>
|
|
||||||
<textarea
|
|
||||||
id="rag-textarea"
|
|
||||||
rows="12"
|
|
||||||
spellcheck="false"
|
|
||||||
placeholder="Incolla o scrivi qui il contesto RAG che il modello deve usare durante la risposta. Es: dati cliente, policy, estratti documentali, ecc."
|
|
||||||
></textarea>
|
|
||||||
<small id="rag-hint" class="tool-hint">
|
|
||||||
Nessun RAG context caricato. Premi "Invia nella sessione" per applicare le modifiche.
|
|
||||||
</small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<footer class="modal-footer">
|
|
||||||
<button id="rag-clear" class="btn secondary" type="button" value="clear">Svuota</button>
|
|
||||||
<div style="flex: 1"></div>
|
|
||||||
<button id="rag-submit" class="btn primary" type="button" value="submit">Invia nella sessione</button>
|
|
||||||
</footer>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<!-- Shown when the per-day conversation budget is spent (mid-call or at
|
|
||||||
start). Title / message / CTA / note are filled per tier by
|
|
||||||
ui/account.js. The hero is the HF mark — a friendly smiling face. -->
|
|
||||||
<dialog id="limit-modal" class="modal limit-modal">
|
|
||||||
<div class="modal-content limit-card">
|
|
||||||
<button id="limit-close" class="icon-btn limit-close" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
<div class="limit-badge" aria-hidden="true">
|
|
||||||
<svg class="hf-logo" viewBox="0 0 95 88" fill="none" aria-hidden="true"><path fill="#FFD21E" d="M47.21 76.5a34.75 34.75 0 1 0 0-69.5 34.75 34.75 0 0 0 0 69.5Z"/><path fill="#FF9D0B" d="M81.96 41.75a34.75 34.75 0 1 0-69.5 0 34.75 34.75 0 0 0 69.5 0Zm-73.5 0a38.75 38.75 0 1 1 77.5 0 38.75 38.75 0 0 1-77.5 0Z"/><path fill="#3A3B45" d="M58.5 32.3c1.28.44 1.78 3.06 3.07 2.38a5 5 0 1 0-6.76-2.07c.61 1.15 2.55-.72 3.7-.32ZM34.95 32.3c-1.28.44-1.79 3.06-3.07 2.38a5 5 0 1 1 6.76-2.07c-.61 1.15-2.56-.72-3.7-.32Z"/><path fill="#FF323D" d="M46.96 56.29c9.83 0 13-8.76 13-13.26 0-2.34-1.57-1.6-4.09-.36-2.33 1.15-5.46 2.74-8.9 2.74-7.19 0-13-6.88-13-2.38s3.16 13.26 13 13.26Z"/><path fill="#3A3B45" fill-rule="evenodd" d="M39.43 54a8.7 8.7 0 0 1 5.3-4.49c.4-.12.81.57 1.24 1.28.4.68.82 1.37 1.24 1.37.45 0 .9-.68 1.33-1.35.45-.7.89-1.38 1.32-1.25a8.61 8.61 0 0 1 5 4.17c3.73-2.94 5.1-7.74 5.1-10.7 0-2.34-1.57-1.6-4.09-.36l-.14.07c-2.31 1.15-5.39 2.67-8.77 2.67s-6.45-1.52-8.77-2.67c-2.6-1.29-4.23-2.1-4.23.29 0 3.05 1.46 8.06 5.47 10.97Z" clip-rule="evenodd"/><path fill="#FF9D0B" d="M70.71 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM24.21 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM17.52 48c-1.62 0-3.06.66-4.07 1.87a5.97 5.97 0 0 0-1.33 3.76 7.1 7.1 0 0 0-1.94-.3c-1.55 0-2.95.59-3.94 1.66a5.8 5.8 0 0 0-.8 7 5.3 5.3 0 0 0-1.79 2.82c-.24.9-.48 2.8.8 4.74a5.22 5.22 0 0 0-.37 5.02c1.02 2.32 3.57 4.14 8.52 6.1 3.07 1.22 5.89 2 5.91 2.01a44.33 44.33 0 0 0 10.93 1.6c5.86 0 10.05-1.8 12.46-5.34 3.88-5.69 3.33-10.9-1.7-15.92-2.77-2.78-4.62-6.87-5-7.77-.78-2.66-2.84-5.62-6.25-5.62a5.7 5.7 0 0 0-4.6 2.46c-1-1.26-1.98-2.25-2.86-2.82A7.4 7.4 0 0 0 17.52 48Zm0 4c.51 0 1.14.22 1.82.65 2.14 1.36 6.25 8.43 7.76 11.18.5.92 1.37 1.31 2.14 1.31 1.55 0 2.75-1.53.15-3.48-3.92-2.93-2.55-7.72-.68-8.01.08-.02.17-.02.24-.02 1.7 0 2.45 2.93 2.45 2.93s2.2 5.52 5.98 9.3c3.77 3.77 3.97 6.8 1.22 10.83-1.88 2.75-5.47 3.58-9.16 3.58-3.81 0-7.73-.9-9.92-1.46-.11-.03-13.45-3.8-11.76-7 .28-.54.75-.76 1.34-.76 2.38 0 6.7 3.54 8.57 3.54.41 0 .7-.17.83-.6.79-2.85-12.06-4.05-10.98-8.17.2-.73.71-1.02 1.44-1.02 3.14 0 10.2 5.53 11.68 5.53.11 0 .2-.03.24-.1.74-1.2.33-2.04-4.9-5.2-5.21-3.16-8.88-5.06-6.8-7.33.24-.26.58-.38 1-.38 3.17 0 10.66 6.82 10.66 6.82s2.02 2.1 3.25 2.1c.28 0 .52-.1.68-.38.86-1.46-8.06-8.22-8.56-11.01-.34-1.9.24-2.85 1.31-2.85Z"/><path fill="#FFD21E" d="M38.6 76.69c2.75-4.04 2.55-7.07-1.22-10.84-3.78-3.77-5.98-9.3-5.98-9.3s-.82-3.2-2.69-2.9c-1.87.3-3.24 5.08.68 8.01 3.91 2.93-.78 4.92-2.29 2.17-1.5-2.75-5.62-9.82-7.76-11.18-2.13-1.35-3.63-.6-3.13 2.2.5 2.79 9.43 9.55 8.56 11-.87 1.47-3.93-1.71-3.93-1.71s-9.57-8.71-11.66-6.44c-2.08 2.27 1.59 4.17 6.8 7.33 5.23 3.16 5.64 4 4.9 5.2-.75 1.2-12.28-8.53-13.36-4.4-1.08 4.11 11.77 5.3 10.98 8.15-.8 2.85-9.06-5.38-10.74-2.18-1.7 3.21 11.65 6.98 11.76 7.01 4.3 1.12 15.25 3.49 19.08-2.12Z"/><path fill="#FF9D0B" d="M77.4 48c1.62 0 3.07.66 4.07 1.87a5.97 5.97 0 0 1 1.33 3.76 7.1 7.1 0 0 1 1.95-.3c1.55 0 2.95.59 3.94 1.66a5.8 5.8 0 0 1 .8 7 5.3 5.3 0 0 1 1.78 2.82c.24.9.48 2.8-.8 4.74a5.22 5.22 0 0 1 .37 5.02c-1.02 2.32-3.57 4.14-8.51 6.1-3.08 1.22-5.9 2-5.92 2.01a44.33 44.33 0 0 1-10.93 1.6c-5.86 0-10.05-1.8-12.46-5.34-3.88-5.69-3.33-10.9 1.7-15.92 2.78-2.78 4.63-6.87 5.01-7.77.78-2.66 2.83-5.62 6.24-5.62a5.7 5.7 0 0 1 4.6 2.46c1-1.26 1.98-2.25 2.87-2.82A7.4 7.4 0 0 1 77.4 48Zm0 4c-.51 0-1.13.22-1.82.65-2.13 1.36-6.25 8.43-7.76 11.18a2.43 2.43 0 0 1-2.14 1.31c-1.54 0-2.75-1.53-.14-3.48 3.91-2.93 2.54-7.72.67-8.01a1.54 1.54 0 0 0-.24-.02c-1.7 0-2.45 2.93-2.45 2.93s-2.2 5.52-5.97 9.3c-3.78 3.77-3.98 6.8-1.22 10.83 1.87 2.75 5.47 3.58 9.15 3.58 3.82 0 7.73-.9 9.93-1.46.1-.03 13.45-3.8 11.76-7-.29-.54-.75-.76-1.34-.76-2.38 0-6.71 3.54-8.57 3.54-.42 0-.71-.17-.83-.6-.8-2.85 12.05-4.05 10.97-8.17-.19-.73-.7-1.02-1.44-1.02-3.14 0-10.2 5.53-11.68 5.53-.1 0-.19-.03-.23-.1-.74-1.2-.34-2.04 4.88-5.2 5.23-3.16 8.9-5.06 6.8-7.33-.23-.26-.57-.38-.98-.38-3.18 0-10.67 6.82-10.67 6.82s-2.02 2.1-3.24 2.1a.74.74 0 0 1-.68-.38c-.87-1.46 8.05-8.22 8.55-11.01.34-1.9-.24-2.85-1.31-2.85Z"/><path fill="#FFD21E" d="M56.33 76.69c-2.75-4.04-2.56-7.07 1.22-10.84 3.77-3.77 5.97-9.3 5.97-9.3s.82-3.2 2.7-2.9c1.86.3 3.23 5.08-.68 8.01-3.92 2.93.78 4.92 2.28 2.17 1.51-2.75 5.63-9.82 7.76-11.18 2.13-1.35 3.64-.6 3.13 2.2-.5 2.79-9.42 9.55-8.55 11 .86 1.47 3.92-1.71 3.92-1.71s9.58-8.71 11.66-6.44c2.08 2.27-1.58 4.17-6.8 7.33-5.23 3.16-5.63 4-4.9 5.2.75 1.2 12.28-8.53 13.36-4.4 1.08 4.11-11.76 5.3-10.97 8.15.8 2.85 9.05-5.38 10.74-2.18 1.69 3.21-11.65 6.98-11.76 7.01-4.31 1.12-15.26 3.49-19.08-2.12Z"/></svg>
|
|
||||||
</div>
|
|
||||||
<h2 id="limit-title" class="limit-title">That's a wrap for now</h2>
|
|
||||||
<p id="limit-msg" class="limit-msg"></p>
|
|
||||||
<a id="limit-cta" class="btn primary wide limit-cta" href="#"></a>
|
|
||||||
<p id="limit-note" class="limit-note"></p>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<!-- User profile dialog — opened by clicking the circular avatar badge in
|
|
||||||
the brand header. Styling: large glass hero with initials + rows of
|
|
||||||
user metadata populated via main.js populateUserProfileDialog(). -->
|
|
||||||
<dialog id="user-profile-dialog" class="modal user-profile-modal" aria-labelledby="user-profile-title">
|
|
||||||
<div class="modal-content user-profile-card">
|
|
||||||
<button id="user-profile-close" class="icon-btn user-profile-close" aria-label="Close user profile">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
|
|
||||||
<!-- Hero: big glass avatar circle + title + subtitle (email fallback). -->
|
|
||||||
<div class="user-profile-hero">
|
|
||||||
<div id="user-profile-avatar" class="user-profile-avatar" aria-hidden="true">AV</div>
|
|
||||||
<h2 id="user-profile-title" class="user-profile-title"></h2>
|
|
||||||
<p id="user-profile-subtitle" class="user-profile-subtitle"></p>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Metadata: a 1-col definition-list-like grid. Empty rows are hidden
|
|
||||||
automatically via JS when the field has no value, so we never show
|
|
||||||
blank labels. -->
|
|
||||||
<div id="user-profile-metadata" class="user-profile-metadata" aria-label="Informazioni utente"></div>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<footer class="app-footer" aria-label="App version footer">
|
|
||||||
<span class="app-footer-text">Azurian Vocero realtime ver.Beta 2.1</span>
|
|
||||||
</footer>
|
|
||||||
|
|
||||||
<script type="module" src="main.js?v=202609102207"></script>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
@@ -1,356 +0,0 @@
|
|||||||
<!doctype html>
|
|
||||||
<html lang="en">
|
|
||||||
<head>
|
|
||||||
<meta charset="UTF-8" />
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
|
|
||||||
<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate, max-age=0" />
|
|
||||||
<meta http-equiv="Pragma" content="no-cache" />
|
|
||||||
<meta http-equiv="Expires" content="0" />
|
|
||||||
<title>Minimal Conversation · S2S backend (WebSocket)</title>
|
|
||||||
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
|
||||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
|
|
||||||
<link
|
|
||||||
href="https://fonts.googleapis.com/css2?family=Geist+Mono:wght@400;500;600&family=Inter:wght@400;500;600;700;800&display=swap"
|
|
||||||
rel="stylesheet"
|
|
||||||
/>
|
|
||||||
<link rel="stylesheet" href="style.css?v=20260825003" />
|
|
||||||
</head>
|
|
||||||
<!--
|
|
||||||
`booting` disables transitions/animations until the first paint commits.
|
|
||||||
Without this the orb visibly fades/scales in on the very first frame
|
|
||||||
because `state-idle` and the generic `.ind` defaults differ. The
|
|
||||||
bootstrap script at the end of main.js strips the class after one rAF.
|
|
||||||
-->
|
|
||||||
<body class="booting">
|
|
||||||
<div id="app">
|
|
||||||
<header class="topbar">
|
|
||||||
<div class="brand">
|
|
||||||
<div class="ident">
|
|
||||||
<div class="ident-head">
|
|
||||||
<a class="ident-title">Azurian Vocero realtime test</a>
|
|
||||||
</div>
|
|
||||||
<div id="ident-user-name" class="ident-user-name" hidden></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div class="topbar-right">
|
|
||||||
<!-- HF login chip / sign-in pill. Populated by ui/account.js; only
|
|
||||||
shown when the deploy runs behind a load balancer with OAuth. -->
|
|
||||||
<div id="account" class="account" hidden></div>
|
|
||||||
<button id="about-btn-m" class="icon-btn about-btn-mobile" title="About this space" aria-label="About this space">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"/><line x1="12" y1="16" x2="12" y2="12"/><line x1="12" y1="8" x2="12.01" y2="8"/></svg>
|
|
||||||
</button>
|
|
||||||
<button id="tools-btn" class="icon-btn" title="Tools" aria-label="Tools">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/></svg>
|
|
||||||
</button>
|
|
||||||
<button id="chat-btn" class="icon-btn" title="Conversation history" aria-label="Conversation history">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/></svg>
|
|
||||||
<span id="chat-badge" class="chat-badge" aria-hidden="true"></span>
|
|
||||||
</button>
|
|
||||||
<button id="settings-btn" class="icon-btn" title="Settings" aria-label="Settings">
|
|
||||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="3"/><path d="M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 1 1-2.83 2.83l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 1 1-4 0v-.09A1.65 1.65 0 0 0 9 19.4a1.65 1.65 0 0 0-1.82.33l-.06.06a2 2 0 1 1-2.83-2.83l.06-.06a1.65 1.65 0 0 0 .33-1.82 1.65 1.65 0 0 0-1.51-1H3a2 2 0 1 1 0-4h.09A1.65 1.65 0 0 0 4.6 9a1.65 1.65 0 0 0-.33-1.82l-.06-.06a2 2 0 1 1 2.83-2.83l.06.06a1.65 1.65 0 0 0 1.82.33H9a1.65 1.65 0 0 0 1-1.51V3a2 2 0 1 1 4 0v.09a1.65 1.65 0 0 0 1 1.51 1.65 1.65 0 0 0 1.82-.33l.06-.06a2 2 0 1 1 2.83 2.83l-.06.06a1.65 1.65 0 0 0-.33 1.82V9c0 .66.39 1.25 1 1.51H21a2 2 0 1 1 0 4h-.09a1.65 1.65 0 0 0-1.51 1z"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<main class="stage">
|
|
||||||
<div class="orb-wrap">
|
|
||||||
<div id="mic-gate" class="mic-gate">
|
|
||||||
<svg id="mic-gate-arc" class="mic-gate-arc" viewBox="0 0 100 100" aria-hidden="true">
|
|
||||||
<path id="mga-track" class="mga-track" fill="none" />
|
|
||||||
<path id="mga-fill" class="mga-fill" fill="none" />
|
|
||||||
<path id="mga-hit" class="mga-hit" fill="none" />
|
|
||||||
<circle id="mga-handle" class="mga-handle" r="3" />
|
|
||||||
</svg>
|
|
||||||
<button id="mic-btn" class="side-btn" type="button" aria-label="Mute" title="Mute" aria-hidden="true">
|
|
||||||
<svg class="mic-on" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="2" width="6" height="12" rx="3"/><path d="M5 10a7 7 0 0 0 14 0"/><line x1="12" y1="19" x2="12" y2="22"/></svg>
|
|
||||||
<svg class="mic-off" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><line x1="2" y1="2" x2="22" y2="22"/><path d="M9 5a3 3 0 0 1 6 0v4"/><path d="M9 10v1a3 3 0 0 0 5.1 2.1"/><path d="M19 10a7 7 0 0 1-1.24 3.97"/><path d="M5 10a7 7 0 0 0 11 5.67"/><line x1="12" y1="19" x2="12" y2="22"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<button
|
|
||||||
id="main-circle"
|
|
||||||
class="circle state-idle"
|
|
||||||
type="button"
|
|
||||||
aria-label="Start voice conversation"
|
|
||||||
>
|
|
||||||
<span class="circle-glow" aria-hidden="true"></span>
|
|
||||||
<span class="circle-ring" aria-hidden="true"></span>
|
|
||||||
<span class="circle-ring-outer" aria-hidden="true"></span>
|
|
||||||
<span class="circle-core">
|
|
||||||
<span class="circle-indicator" aria-hidden="true">
|
|
||||||
<svg class="ind ind-mic" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<rect x="9" y="2" width="6" height="12" rx="3" fill="currentColor" stroke="none"/>
|
|
||||||
<path d="M5 10a7 7 0 0 0 14 0"/>
|
|
||||||
<line x1="12" y1="19" x2="12" y2="22"/>
|
|
||||||
<line x1="8" y1="22" x2="16" y2="22"/>
|
|
||||||
</svg>
|
|
||||||
<svg class="ind ind-error" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="9"/><line x1="12" y1="8" x2="12" y2="13"/><line x1="12" y1="16" x2="12" y2="16"/></svg>
|
|
||||||
<span class="ind ind-spinner"></span>
|
|
||||||
<span class="ind ind-thinking">
|
|
||||||
<span class="dot"></span>
|
|
||||||
<span class="dot"></span>
|
|
||||||
<span class="dot"></span>
|
|
||||||
</span>
|
|
||||||
<span class="ind ind-bars">
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
<span class="bar"></span>
|
|
||||||
</span>
|
|
||||||
<svg class="ind ind-voice" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
|
|
||||||
<path d="M3 10v4a1 1 0 0 0 1 1h3l5 4V5L7 9H4a1 1 0 0 0-1 1z" fill="currentColor" stroke="none"/>
|
|
||||||
<path class="wave wave-1" d="M16 8a5 5 0 0 1 0 8"/>
|
|
||||||
<path class="wave wave-2" d="M19 5a9 9 0 0 1 0 14"/>
|
|
||||||
</svg>
|
|
||||||
</span>
|
|
||||||
</span>
|
|
||||||
</button>
|
|
||||||
|
|
||||||
<button id="stop-btn" class="side-btn" type="button" aria-label="End" title="End" aria-hidden="true">
|
|
||||||
<svg viewBox="0 0 24 24" fill="currentColor"><rect x="6" y="6" width="12" height="12" rx="2"/></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<p id="circle-caption" class="circle-caption" role="status">Tap to start</p>
|
|
||||||
<p id="circle-subcaption" class="circle-subcaption" hidden></p>
|
|
||||||
|
|
||||||
<div id="queue-actions" class="queue-actions" hidden>
|
|
||||||
<button id="join-queue-btn" class="join-queue-btn" type="button" hidden>
|
|
||||||
Join now
|
|
||||||
</button>
|
|
||||||
<button id="leave-queue-btn" class="leave-queue-btn" type="button" hidden>
|
|
||||||
Leave queue
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</main>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Ephemeral chat bubbles anchored to the top-right -->
|
|
||||||
<div id="bubble-stack" class="bubble-stack" aria-live="polite" aria-atomic="false"></div>
|
|
||||||
|
|
||||||
<!-- Webcam self-view, shown bottom-left while the camera tool is enabled -->
|
|
||||||
<div id="cam-pip" class="cam-pip" aria-hidden="true">
|
|
||||||
<video id="cam-video" class="cam-video" autoplay playsinline muted></video>
|
|
||||||
<span class="cam-flash" aria-hidden="true"></span>
|
|
||||||
<span class="cam-label">camera</span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Conversation history panel (overlay) -->
|
|
||||||
<div id="chat-panel" class="chat-panel">
|
|
||||||
<div id="chat-panel-backdrop" class="chat-panel-backdrop"></div>
|
|
||||||
<div class="chat-panel-inner">
|
|
||||||
<header class="chat-panel-header">
|
|
||||||
<h3>Conversation</h3>
|
|
||||||
<button id="chat-panel-close" class="icon-btn" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
<div id="chat-history" class="chat-history"></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<dialog id="about-modal" class="modal about-modal">
|
|
||||||
<div class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>About</h2>
|
|
||||||
<button id="about-close" class="icon-btn" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<!-- General intro to the speech-to-speech project -->
|
|
||||||
<div class="about-intro">
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- The backend, shown as the path a turn actually travels -->
|
|
||||||
<div class="about-pipeline">
|
|
||||||
<p class="pipeline-title">The pipeline</p>
|
|
||||||
<ol class="pipeline">
|
|
||||||
<li class="pipe-endpoint">You speak</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">VAD</span>
|
|
||||||
<span class="pipe-job">detects speech</span>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">STT</span>
|
|
||||||
<span class="pipe-job">transcribes it</span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/nvidia/parakeet-tdt-1.1b" target="_blank" rel="noopener">nvidia/parakeet-tdt-1.1b<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">VLM</span>
|
|
||||||
<span class="pipe-job">analyses and composes the reply <span class="pipe-note">· via <a class="cerebras-credit" href="https://cerebras.ai" target="_blank" rel="noopener"><svg class="cerebras-mark" viewBox="0 0 50 50" aria-hidden="true"><path fill-rule="evenodd" clip-rule="evenodd" d="M29.4186 5.62713C24.2806 5.62713 19.353 7.6682 15.7199 11.3013C12.0868 14.9345 10.0457 19.862 10.0457 25C10.0457 30.1381 12.0868 35.0656 15.7199 38.6988C19.353 42.3319 24.2806 44.373 29.4186 44.373V47.2917C17.1061 47.2917 7.12695 37.3105 7.12695 24.998C7.12695 12.6855 17.104 2.7063 29.4165 2.7063V5.62505L29.4186 5.62713ZM39.3186 13.2875C37.7794 11.9809 35.9971 10.9915 34.0741 10.3761C32.1512 9.76067 30.1256 9.53147 28.1137 9.70166C26.1019 9.87185 24.1435 10.4381 22.3513 11.3677C20.559 12.2974 18.9683 13.5722 17.6704 15.1189C16.3726 16.6655 15.3933 18.4534 14.7888 20.3798C14.1844 22.3062 13.9667 24.3332 14.1483 26.344C14.33 28.3548 14.9073 30.3099 15.8472 32.0968C16.787 33.8838 18.0709 35.4673 19.6249 36.7563L17.7478 38.9938C15.9126 37.4545 14.3987 35.5687 13.2924 33.4442C12.1862 31.3197 11.5092 28.9981 11.3003 26.6119C11.0914 24.2258 11.3545 21.8219 12.0747 19.5374C12.7949 17.253 13.9581 15.1328 15.4978 13.298C17.0373 11.4628 18.9233 9.94877 21.0479 8.84247C23.1726 7.73618 25.4944 7.05923 27.8807 6.8503C30.267 6.64136 32.6711 6.90453 34.9557 7.62477C37.2403 8.34501 39.3607 9.50822 41.1957 11.048L39.3186 13.2875ZM34.6207 15.0459C31.9818 13.6829 28.9113 13.4172 26.0776 14.3068C23.2438 15.1964 20.876 17.1691 19.4895 19.7958C18.1029 22.4224 17.8099 25.4903 18.6741 28.332C19.5384 31.1736 21.4899 33.5589 24.104 34.9688L22.7374 37.5521C19.4721 35.7617 17.0418 32.7592 15.9707 29.1927C14.8996 25.6261 15.2737 21.7815 17.0123 18.4883C18.7509 15.1952 21.7146 12.7176 25.2637 11.5903C28.8129 10.463 32.663 10.7763 35.9832 12.4625L34.6207 15.0459ZM29.4165 17.7896C27.5042 17.7896 25.6702 18.5493 24.318 19.9015C22.9658 21.2537 22.2061 23.0877 22.2061 25C22.2061 26.9124 22.9658 28.7464 24.318 30.0986C25.6702 31.4508 27.5042 32.2105 29.4165 32.2105V35.1313C26.7296 35.1313 24.1526 34.0639 22.2527 32.1639C20.3527 30.2639 19.2853 27.687 19.2853 25C19.2853 22.3131 20.3527 19.7362 22.2527 17.8362C24.1526 15.9362 26.7296 14.8688 29.4165 14.8688V17.7896Z" fill="#F15A29"/><path d="M32.0981 22.5749C31.7875 22.2404 31.4295 21.9534 31.0356 21.7228C30.6925 21.519 30.3014 21.4097 29.9023 21.4062C29.371 21.4062 28.896 21.5041 28.4773 21.6978C28.0725 21.8802 27.7089 22.1425 27.4081 22.469C27.1074 22.7955 26.8758 23.1795 26.7273 23.5978C26.5731 24.0207 26.4981 24.4645 26.4981 24.9124C26.4981 25.3666 26.5731 25.8082 26.7273 26.227C26.877 26.6449 27.1091 27.0286 27.4096 27.3553C27.7102 27.682 28.0733 27.9451 28.4773 28.1291C28.8939 28.3228 29.3731 28.4207 29.9023 28.4207C30.3523 28.4207 30.771 28.3249 31.1564 28.1395C31.5481 27.952 31.8856 27.6707 32.146 27.3228L34.0794 29.4187C33.7877 29.7103 33.4544 29.9624 33.0752 30.1749C32.374 30.566 31.6108 30.8338 30.8189 30.9666C30.4648 31.0207 30.1585 31.0499 29.9023 31.0499C29.0617 31.0552 28.2271 30.9069 27.4398 30.6124C26.6954 30.3371 26.0144 29.914 25.4377 29.3687C24.8643 28.8208 24.4078 28.1624 24.096 27.4332C23.7572 26.6366 23.5883 25.778 23.6002 24.9124C23.6002 23.9874 23.7669 23.1478 24.096 22.3916C24.4085 21.6624 24.8627 21.0041 25.4356 20.4562C26.0148 19.9124 26.696 19.4895 27.4398 19.2145C28.2271 18.92 29.0617 18.7717 29.9023 18.777C30.6419 18.777 31.3856 18.9187 32.1356 19.202C32.8877 19.4895 33.5627 19.952 34.1023 20.5541L32.0981 22.5749Z" fill="#fff"/></svg>Cerebras</a></span></span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/google/gemma-4-31B-it" target="_blank" rel="noopener">google/gemma-4-31B-it<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-stage">
|
|
||||||
<span class="pipe-tag">TTS</span>
|
|
||||||
<span class="pipe-job">speaks it back</span>
|
|
||||||
<a class="pipe-model" href="https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice" target="_blank" rel="noopener">Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
|
|
||||||
</li>
|
|
||||||
<li class="pipe-endpoint">The orb replies</li>
|
|
||||||
</ol>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<dialog id="settings-modal" class="modal">
|
|
||||||
<form method="dialog" class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Settings</h2>
|
|
||||||
<button class="icon-btn" value="close" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<div class="tab-panels">
|
|
||||||
<section class="tab-panel active" role="tabpanel">
|
|
||||||
<label class="field" id="conn-field">
|
|
||||||
<span id="conn-label">Speech-to-speech server URL</span>
|
|
||||||
<input id="lb-url" type="text" autocomplete="off" spellcheck="false" placeholder="http://localhost:port" />
|
|
||||||
<small id="conn-hint">
|
|
||||||
URL of your speech-to-speech server, e.g.
|
|
||||||
<code>http://localhost:8080</code> (the app adds <code>/v1/realtime</code>).
|
|
||||||
</small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<div class="field-row">
|
|
||||||
<label class="field">
|
|
||||||
<span>Voice</span>
|
|
||||||
<select id="voice">
|
|
||||||
<option value="Aiden" selected>Aiden</option>
|
|
||||||
<option value="Ryan">Ryan</option>
|
|
||||||
<option value="Dylan">Dylan</option>
|
|
||||||
<option value="Eric">Eric</option>
|
|
||||||
<option value="Ono_Anna">Ono_Anna</option>
|
|
||||||
<option value="Serena">Serena</option>
|
|
||||||
<option value="Sohee">Sohee</option>
|
|
||||||
<option value="Uncle_Fu">Uncle_Fu</option>
|
|
||||||
<option value="Vivian">Vivian</option>
|
|
||||||
</select>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="field">
|
|
||||||
<span class="field-head">
|
|
||||||
Noise gate
|
|
||||||
<span id="gate-value" class="field-value">Off</span>
|
|
||||||
</span>
|
|
||||||
<div class="gate">
|
|
||||||
<div class="gate-track" aria-hidden="true">
|
|
||||||
<div id="gate-meter-fill" class="gate-meter-fill"></div>
|
|
||||||
<input
|
|
||||||
id="noise-gate"
|
|
||||||
type="range"
|
|
||||||
min="-66"
|
|
||||||
max="-3"
|
|
||||||
step="1"
|
|
||||||
value="-50"
|
|
||||||
aria-label="Noise gate threshold"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
<div class="gate-ends">
|
|
||||||
<span>Off</span>
|
|
||||||
<span>−3 dB</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<small>
|
|
||||||
Mutes the mic below the handle so room noise isn't sent. Slide fully
|
|
||||||
left to turn it off; the bar shows your live input while in a call.
|
|
||||||
</small>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<label class="field">
|
|
||||||
<span>Instructions</span>
|
|
||||||
<textarea id="instructions" rows="5" placeholder="You are Vocero, a friendly voice assistant for the company Aquachile..."></textarea>
|
|
||||||
<small id="primary-prompt-meta" hidden></small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<div class="field">
|
|
||||||
<button id="restart-conversation" type="button" class="btn primary wide" disabled>
|
|
||||||
Restart conversation with these settings
|
|
||||||
</button>
|
|
||||||
<small id="restart-hint">Connect first, then come back here to apply live changes.</small>
|
|
||||||
</div>
|
|
||||||
</section>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<footer class="modal-footer">
|
|
||||||
<button id="settings-save" type="submit" class="btn primary" value="save">Save</button>
|
|
||||||
</footer>
|
|
||||||
</form>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<dialog id="tools-modal" class="modal">
|
|
||||||
<div class="modal-content">
|
|
||||||
<header class="modal-header">
|
|
||||||
<h2>Tools</h2>
|
|
||||||
<button id="tools-close" class="icon-btn" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<p class="tools-intro">Let the assistant act during the conversation. Changes apply live.</p>
|
|
||||||
|
|
||||||
<div class="tool-list">
|
|
||||||
<div class="tool-row" id="tool-web-row">
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Web search</span>
|
|
||||||
<span class="tool-desc">Look things up on Google, via Serper.</span>
|
|
||||||
</div>
|
|
||||||
<label class="switch">
|
|
||||||
<input id="tool-web" type="checkbox" />
|
|
||||||
<span class="switch-track" aria-hidden="true"></span>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
<label class="field tools-key">
|
|
||||||
<span>Search API key</span>
|
|
||||||
<input id="search-key" type="password" autocomplete="off" spellcheck="false" />
|
|
||||||
<small id="tool-web-hint"></small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<div class="tool-row tool-row-sep" id="tool-cam-row">
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Camera</span>
|
|
||||||
<span class="tool-desc">Let the assistant see through your webcam.</span>
|
|
||||||
</div>
|
|
||||||
<label class="switch">
|
|
||||||
<input id="tool-cam" type="checkbox" />
|
|
||||||
<span class="switch-track" aria-hidden="true"></span>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
<small id="tool-cam-hint" class="tool-hint">A live preview shows bottom-left while the camera is on.</small>
|
|
||||||
|
|
||||||
<div class="tool-row tool-row-sep">
|
|
||||||
<div class="tool-info">
|
|
||||||
<span class="tool-name">Loaded MCP tools</span>
|
|
||||||
<span class="tool-desc">Tools published by the remote agent profile.</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<small id="agent-tools-hint" class="tool-hint"></small>
|
|
||||||
<div id="agent-tools-list" class="agent-tools-list"></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<!-- Shown when the per-day conversation budget is spent (mid-call or at
|
|
||||||
start). Title / message / CTA / note are filled per tier by
|
|
||||||
ui/account.js. The hero is the HF mark — a friendly smiling face. -->
|
|
||||||
<dialog id="limit-modal" class="modal limit-modal">
|
|
||||||
<div class="modal-content limit-card">
|
|
||||||
<button id="limit-close" class="icon-btn limit-close" aria-label="Close">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
|
|
||||||
</button>
|
|
||||||
<div class="limit-badge" aria-hidden="true">
|
|
||||||
<svg class="hf-logo" viewBox="0 0 95 88" fill="none" aria-hidden="true"><path fill="#FFD21E" d="M47.21 76.5a34.75 34.75 0 1 0 0-69.5 34.75 34.75 0 0 0 0 69.5Z"/><path fill="#FF9D0B" d="M81.96 41.75a34.75 34.75 0 1 0-69.5 0 34.75 34.75 0 0 0 69.5 0Zm-73.5 0a38.75 38.75 0 1 1 77.5 0 38.75 38.75 0 0 1-77.5 0Z"/><path fill="#3A3B45" d="M58.5 32.3c1.28.44 1.78 3.06 3.07 2.38a5 5 0 1 0-6.76-2.07c.61 1.15 2.55-.72 3.7-.32ZM34.95 32.3c-1.28.44-1.79 3.06-3.07 2.38a5 5 0 1 1 6.76-2.07c-.61 1.15-2.56-.72-3.7-.32Z"/><path fill="#FF323D" d="M46.96 56.29c9.83 0 13-8.76 13-13.26 0-2.34-1.57-1.6-4.09-.36-2.33 1.15-5.46 2.74-8.9 2.74-7.19 0-13-6.88-13-2.38s3.16 13.26 13 13.26Z"/><path fill="#3A3B45" fill-rule="evenodd" d="M39.43 54a8.7 8.7 0 0 1 5.3-4.49c.4-.12.81.57 1.24 1.28.4.68.82 1.37 1.24 1.37.45 0 .9-.68 1.33-1.35.45-.7.89-1.38 1.32-1.25a8.61 8.61 0 0 1 5 4.17c3.73-2.94 5.1-7.74 5.1-10.7 0-2.34-1.57-1.6-4.09-.36l-.14.07c-2.31 1.15-5.39 2.67-8.77 2.67s-6.45-1.52-8.77-2.67c-2.6-1.29-4.23-2.1-4.23.29 0 3.05 1.46 8.06 5.47 10.97Z" clip-rule="evenodd"/><path fill="#FF9D0B" d="M70.71 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM24.21 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM17.52 48c-1.62 0-3.06.66-4.07 1.87a5.97 5.97 0 0 0-1.33 3.76 7.1 7.1 0 0 0-1.94-.3c-1.55 0-2.95.59-3.94 1.66a5.8 5.8 0 0 0-.8 7 5.3 5.3 0 0 0-1.79 2.82c-.24.9-.48 2.8.8 4.74a5.22 5.22 0 0 0-.37 5.02c1.02 2.32 3.57 4.14 8.52 6.1 3.07 1.22 5.89 2 5.91 2.01a44.33 44.33 0 0 0 10.93 1.6c5.86 0 10.05-1.8 12.46-5.34 3.88-5.69 3.33-10.9-1.7-15.92-2.77-2.78-4.62-6.87-5-7.77-.78-2.66-2.84-5.62-6.25-5.62a5.7 5.7 0 0 0-4.6 2.46c-1-1.26-1.98-2.25-2.86-2.82A7.4 7.4 0 0 0 17.52 48Zm0 4c.51 0 1.14.22 1.82.65 2.14 1.36 6.25 8.43 7.76 11.18.5.92 1.37 1.31 2.14 1.31 1.55 0 2.75-1.53.15-3.48-3.92-2.93-2.55-7.72-.68-8.01.08-.02.17-.02.24-.02 1.7 0 2.45 2.93 2.45 2.93s2.2 5.52 5.98 9.3c3.77 3.77 3.97 6.8 1.22 10.83-1.88 2.75-5.47 3.58-9.16 3.58-3.81 0-7.73-.9-9.92-1.46-.11-.03-13.45-3.8-11.76-7 .28-.54.75-.76 1.34-.76 2.38 0 6.7 3.54 8.57 3.54.41 0 .7-.17.83-.6.79-2.85-12.06-4.05-10.98-8.17.2-.73.71-1.02 1.44-1.02 3.14 0 10.2 5.53 11.68 5.53.11 0 .2-.03.24-.1.74-1.2.33-2.04-4.9-5.2-5.21-3.16-8.88-5.06-6.8-7.33.24-.26.58-.38 1-.38 3.17 0 10.66 6.82 10.66 6.82s2.02 2.1 3.25 2.1c.28 0 .52-.1.68-.38.86-1.46-8.06-8.22-8.56-11.01-.34-1.9.24-2.85 1.31-2.85Z"/><path fill="#FFD21E" d="M38.6 76.69c2.75-4.04 2.55-7.07-1.22-10.84-3.78-3.77-5.98-9.3-5.98-9.3s-.82-3.2-2.69-2.9c-1.87.3-3.24 5.08.68 8.01 3.91 2.93-.78 4.92-2.29 2.17-1.5-2.75-5.62-9.82-7.76-11.18-2.13-1.35-3.63-.6-3.13 2.2.5 2.79 9.43 9.55 8.56 11-.87 1.47-3.93-1.71-3.93-1.71s-9.57-8.71-11.66-6.44c-2.08 2.27 1.59 4.17 6.8 7.33 5.23 3.16 5.64 4 4.9 5.2-.75 1.2-12.28-8.53-13.36-4.4-1.08 4.11 11.77 5.3 10.98 8.15-.8 2.85-9.06-5.38-10.74-2.18-1.7 3.21 11.65 6.98 11.76 7.01 4.3 1.12 15.25 3.49 19.08-2.12Z"/><path fill="#FF9D0B" d="M77.4 48c1.62 0 3.07.66 4.07 1.87a5.97 5.97 0 0 1 1.33 3.76 7.1 7.1 0 0 1 1.95-.3c1.55 0 2.95.59 3.94 1.66a5.8 5.8 0 0 1 .8 7 5.3 5.3 0 0 1 1.78 2.82c.24.9.48 2.8-.8 4.74a5.22 5.22 0 0 1 .37 5.02c-1.02 2.32-3.57 4.14-8.51 6.1-3.08 1.22-5.9 2-5.92 2.01a44.33 44.33 0 0 1-10.93 1.6c-5.86 0-10.05-1.8-12.46-5.34-3.88-5.69-3.33-10.9 1.7-15.92 2.78-2.78 4.63-6.87 5.01-7.77.78-2.66 2.83-5.62 6.24-5.62a5.7 5.7 0 0 1 4.6 2.46c1-1.26 1.98-2.25 2.87-2.82A7.4 7.4 0 0 1 77.4 48Zm0 4c-.51 0-1.13.22-1.82.65-2.13 1.36-6.25 8.43-7.76 11.18a2.43 2.43 0 0 1-2.14 1.31c-1.54 0-2.75-1.53-.14-3.48 3.91-2.93 2.54-7.72.67-8.01a1.54 1.54 0 0 0-.24-.02c-1.7 0-2.45 2.93-2.45 2.93s-2.2 5.52-5.97 9.3c-3.78 3.77-3.98 6.8-1.22 10.83 1.87 2.75 5.47 3.58 9.15 3.58 3.82 0 7.73-.9 9.93-1.46.1-.03 13.45-3.8 11.76-7-.29-.54-.75-.76-1.34-.76-2.38 0-6.71 3.54-8.57 3.54-.42 0-.71-.17-.83-.6-.8-2.85 12.05-4.05 10.97-8.17-.19-.73-.7-1.02-1.44-1.02-3.14 0-10.2 5.53-11.68 5.53-.1 0-.19-.03-.23-.1-.74-1.2-.34-2.04 4.88-5.2 5.23-3.16 8.9-5.06 6.8-7.33-.23-.26-.57-.38-.98-.38-3.18 0-10.67 6.82-10.67 6.82s-2.02 2.1-3.24 2.1a.74.74 0 0 1-.68-.38c-.87-1.46 8.05-8.22 8.55-11.01.34-1.9-.24-2.85-1.31-2.85Z"/><path fill="#FFD21E" d="M56.33 76.69c-2.75-4.04-2.56-7.07 1.22-10.84 3.77-3.77 5.97-9.3 5.97-9.3s.82-3.2 2.7-2.9c1.86.3 3.23 5.08-.68 8.01-3.92 2.93.78 4.92 2.28 2.17 1.51-2.75 5.63-9.82 7.76-11.18 2.13-1.35 3.64-.6 3.13 2.2-.5 2.79-9.42 9.55-8.55 11 .86 1.47 3.92-1.71 3.92-1.71s9.58-8.71 11.66-6.44c2.08 2.27-1.58 4.17-6.8 7.33-5.23 3.16-5.63 4-4.9 5.2.75 1.2 12.28-8.53 13.36-4.4 1.08 4.11-11.76 5.3-10.97 8.15.8 2.85 9.05-5.38 10.74-2.18 1.69 3.21-11.65 6.98-11.76 7.01-4.31 1.12-15.26 3.49-19.08-2.12Z"/></svg>
|
|
||||||
</div>
|
|
||||||
<h2 id="limit-title" class="limit-title">That's a wrap for now</h2>
|
|
||||||
<p id="limit-msg" class="limit-msg"></p>
|
|
||||||
<a id="limit-cta" class="btn primary wide limit-cta" href="#"></a>
|
|
||||||
<p id="limit-note" class="limit-note"></p>
|
|
||||||
</div>
|
|
||||||
</dialog>
|
|
||||||
|
|
||||||
<script type="module" src="main.js?v=20260825003"></script>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
@@ -1,276 +0,0 @@
|
|||||||
"""
|
|
||||||
Per-day talk-time budget for the speech-to-speech demo.
|
|
||||||
|
|
||||||
Our server isn't in the audio path (the browser dials the compute WebSocket
|
|
||||||
directly), so it can't cut a live stream. What it *can* do is meter time with a
|
|
||||||
server-clock, chunked reservation:
|
|
||||||
|
|
||||||
- At grant we reserve the first chunk (CHUNK_SEC) and debit it from the day's
|
|
||||||
budget. A parallel grant therefore sees the budget already spent.
|
|
||||||
- The client heartbeats; each heartbeat extends the reservation one chunk at a
|
|
||||||
time until the daily budget runs out, then we report `expired` so the client
|
|
||||||
tears down.
|
|
||||||
- On a clean end (sendBeacon) we reconcile to the real elapsed time and refund
|
|
||||||
the unused chunk. A crash (no end, no heartbeats) is reaped by a sweep and
|
|
||||||
forfeits at most one chunk.
|
|
||||||
|
|
||||||
All time is the server's clock. Budgets are per UTC day; a new day is simply a
|
|
||||||
new row (no explicit reset). Logged-in users are keyed by a hashed HF `sub`;
|
|
||||||
anonymous users by BOTH a hashed IP and a hashed signed-cookie id, OR-matched
|
|
||||||
(spent = max of the two) so clearing one identifier doesn't reset the budget.
|
|
||||||
|
|
||||||
Storage is SQLite at $USAGE_DB_PATH, else /data (persistent Spaces storage),
|
|
||||||
else a /tmp fallback. On /tmp the budget is only per-uptime — flagged in logs.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import hashlib
|
|
||||||
import hmac
|
|
||||||
import logging
|
|
||||||
import math
|
|
||||||
import os
|
|
||||||
import sqlite3
|
|
||||||
import tempfile
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
logger = logging.getLogger("s2s.limiter")
|
|
||||||
|
|
||||||
# ── Tunables (env-overridable) ───────────────────────────────────────────────
|
|
||||||
ANON_SEC = int(os.environ.get("LIMIT_ANON_SEC", "300")) # 5 min/day, not signed in
|
|
||||||
FREE_SEC = int(os.environ.get("LIMIT_FREE_SEC", "600")) # 10 min/day, signed in, no PRO
|
|
||||||
CHUNK_SEC = int(os.environ.get("RESERVE_CHUNK_SEC", "10")) # reservation granularity
|
|
||||||
HEARTBEAT_SEC = int(os.environ.get("HEARTBEAT_SEC", "5")) # advertised client cadence
|
|
||||||
REAP_AFTER_SEC = int(os.environ.get("SESSION_REAP_SEC", "15")) # silence before sweep
|
|
||||||
|
|
||||||
# Stable across restarts or the hashed keys (and signed cookies) rotate and the
|
|
||||||
# budget effectively resets. Set it as a Space secret. Falls back to a per-boot
|
|
||||||
# random value (keys then only hold within one uptime).
|
|
||||||
_HASH_SECRET = (os.environ.get("USAGE_HASH_SECRET", "").strip() or os.urandom(32).hex()).encode()
|
|
||||||
|
|
||||||
_lock = threading.Lock()
|
|
||||||
_db_path: "Path | None" = None
|
|
||||||
|
|
||||||
|
|
||||||
def budget_for(tier: str) -> "int | None":
|
|
||||||
"""Daily second-budget for a tier, or None for unlimited.
|
|
||||||
|
|
||||||
Unlimited tiers: 'pro' (paying PRO members) and 'org' (members of an
|
|
||||||
allow-listed organisation, see UNLIMITED_ORGS in auth.py)."""
|
|
||||||
if tier in ("pro", "org"):
|
|
||||||
return None
|
|
||||||
if tier == "free":
|
|
||||||
return FREE_SEC
|
|
||||||
return ANON_SEC
|
|
||||||
|
|
||||||
|
|
||||||
def hash_key(raw: str) -> str:
|
|
||||||
"""HMAC a raw identifier (sub / ip / cookie id) into an opaque storage key."""
|
|
||||||
digest = hmac.new(_HASH_SECRET, raw.encode("utf-8"), hashlib.sha256).hexdigest()
|
|
||||||
return f"k_{digest}"
|
|
||||||
|
|
||||||
|
|
||||||
def sign_cookie(value: str) -> str:
|
|
||||||
"""`<id>.<sig>` so a forged anon-cookie id is rejected on read."""
|
|
||||||
sig = hmac.new(_HASH_SECRET, value.encode("utf-8"), hashlib.sha256).hexdigest()[:32]
|
|
||||||
return f"{value}.{sig}"
|
|
||||||
|
|
||||||
|
|
||||||
def verify_cookie(signed: str) -> "str | None":
|
|
||||||
"""Return the id if the signature checks out, else None."""
|
|
||||||
if not signed or "." not in signed:
|
|
||||||
return None
|
|
||||||
value, _, sig = signed.rpartition(".")
|
|
||||||
want = hmac.new(_HASH_SECRET, value.encode("utf-8"), hashlib.sha256).hexdigest()[:32]
|
|
||||||
return value if hmac.compare_digest(sig, want) else None
|
|
||||||
|
|
||||||
|
|
||||||
def _today() -> str:
|
|
||||||
return datetime.now(timezone.utc).date().isoformat()
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_db_path() -> Path:
|
|
||||||
explicit = os.environ.get("USAGE_DB_PATH", "").strip()
|
|
||||||
if explicit:
|
|
||||||
return Path(explicit)
|
|
||||||
data = Path("/data")
|
|
||||||
if data.is_dir() and os.access(data, os.W_OK):
|
|
||||||
return data / "s2s-usage.sqlite3"
|
|
||||||
logger.warning("No persistent /data — usage budget falls back to /tmp (per-uptime only).")
|
|
||||||
return Path(tempfile.gettempdir()) / "s2s-usage.sqlite3"
|
|
||||||
|
|
||||||
|
|
||||||
def _connect() -> sqlite3.Connection:
|
|
||||||
con = sqlite3.connect(_db_path, timeout=5.0)
|
|
||||||
con.execute("PRAGMA journal_mode=WAL")
|
|
||||||
con.execute("PRAGMA busy_timeout=5000")
|
|
||||||
return con
|
|
||||||
|
|
||||||
|
|
||||||
def init() -> None:
|
|
||||||
"""Create the schema. Call once at startup."""
|
|
||||||
global _db_path
|
|
||||||
_db_path = _resolve_db_path()
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
con.execute(
|
|
||||||
"""CREATE TABLE IF NOT EXISTS usage_daily (
|
|
||||||
user_key TEXT NOT NULL,
|
|
||||||
day TEXT NOT NULL,
|
|
||||||
spent_sec INTEGER NOT NULL DEFAULT 0,
|
|
||||||
updated_at INTEGER NOT NULL,
|
|
||||||
PRIMARY KEY (user_key, day)
|
|
||||||
)"""
|
|
||||||
)
|
|
||||||
con.execute(
|
|
||||||
"""CREATE TABLE IF NOT EXISTS sessions (
|
|
||||||
session_id TEXT PRIMARY KEY,
|
|
||||||
keys TEXT NOT NULL, -- comma-joined usage_daily keys to debit
|
|
||||||
day TEXT NOT NULL,
|
|
||||||
tier TEXT NOT NULL,
|
|
||||||
grant_ts REAL NOT NULL,
|
|
||||||
last_seen_ts REAL NOT NULL,
|
|
||||||
reserved_sec INTEGER NOT NULL,
|
|
||||||
ended INTEGER NOT NULL DEFAULT 0
|
|
||||||
)"""
|
|
||||||
)
|
|
||||||
logger.info("Usage limiter ready at %s (anon=%ss free=%ss chunk=%ss)", _db_path, ANON_SEC, FREE_SEC, CHUNK_SEC)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Internal helpers (call under _lock) ───────────────────────────────────────
|
|
||||||
|
|
||||||
def _spent(con, key: str, day: str) -> int:
|
|
||||||
row = con.execute(
|
|
||||||
"SELECT spent_sec FROM usage_daily WHERE user_key=? AND day=?", (key, day)
|
|
||||||
).fetchone()
|
|
||||||
return int(row[0]) if row else 0
|
|
||||||
|
|
||||||
|
|
||||||
def _spent_max(con, keys, day: str) -> int:
|
|
||||||
"""OR-match: the most-spent identifier governs."""
|
|
||||||
return max((_spent(con, k, day) for k in keys), default=0)
|
|
||||||
|
|
||||||
|
|
||||||
def _add(con, keys, day: str, delta: int) -> None:
|
|
||||||
now = int(time.time())
|
|
||||||
for k in keys:
|
|
||||||
cur = _spent(con, k, day)
|
|
||||||
nxt = max(0, cur + delta)
|
|
||||||
con.execute(
|
|
||||||
"""INSERT INTO usage_daily (user_key, day, spent_sec, updated_at)
|
|
||||||
VALUES (?, ?, ?, ?)
|
|
||||||
ON CONFLICT(user_key, day) DO UPDATE SET
|
|
||||||
spent_sec = excluded.spent_sec, updated_at = excluded.updated_at""",
|
|
||||||
(k, day, nxt, now),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Public API ────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def remaining(keys, tier: str) -> "int | None":
|
|
||||||
"""Seconds left today for these keys (None = unlimited). No mutation."""
|
|
||||||
budget = budget_for(tier)
|
|
||||||
if budget is None:
|
|
||||||
return None
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
return max(0, budget - _spent_max(con, keys, _today()))
|
|
||||||
|
|
||||||
|
|
||||||
def begin(session_id: str, keys, tier: str) -> int:
|
|
||||||
"""Reserve the first chunk for a new session and record it. Returns the
|
|
||||||
chunk reserved (0 if the budget is already exhausted — the first heartbeat
|
|
||||||
will then expire it). PRO (unlimited) is never tracked; don't call it here."""
|
|
||||||
day = _today()
|
|
||||||
budget = budget_for(tier)
|
|
||||||
now = time.time()
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
avail = budget - _spent_max(con, keys, day) if budget is not None else CHUNK_SEC
|
|
||||||
chunk = max(0, min(CHUNK_SEC, avail))
|
|
||||||
if chunk:
|
|
||||||
_add(con, keys, day, chunk)
|
|
||||||
con.execute(
|
|
||||||
"""INSERT OR REPLACE INTO sessions
|
|
||||||
(session_id, keys, day, tier, grant_ts, last_seen_ts, reserved_sec, ended)
|
|
||||||
VALUES (?, ?, ?, ?, ?, ?, ?, 0)""",
|
|
||||||
(session_id, ",".join(keys), day, tier, now, now, chunk),
|
|
||||||
)
|
|
||||||
return chunk
|
|
||||||
|
|
||||||
|
|
||||||
def heartbeat(session_id: str) -> bool:
|
|
||||||
"""Keep a session alive: extend the reservation toward `elapsed + 1 chunk`,
|
|
||||||
debiting the budget chunk by chunk. Returns True while alive, False once the
|
|
||||||
budget is spent (caller should tear down) or the session is unknown/ended."""
|
|
||||||
now = time.time()
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
row = con.execute(
|
|
||||||
"SELECT keys, day, tier, grant_ts, reserved_sec, ended FROM sessions WHERE session_id=?",
|
|
||||||
(session_id,),
|
|
||||||
).fetchone()
|
|
||||||
if not row or row[5]:
|
|
||||||
return False
|
|
||||||
keys = row[0].split(",")
|
|
||||||
day, tier, grant_ts, reserved = row[1], row[2], row[3], int(row[4])
|
|
||||||
budget = budget_for(tier)
|
|
||||||
elapsed = now - grant_ts
|
|
||||||
|
|
||||||
# Grow the reservation one chunk at a time until it covers elapsed + a
|
|
||||||
# one-chunk lookahead, or the budget runs dry.
|
|
||||||
while reserved < elapsed + CHUNK_SEC:
|
|
||||||
if budget is not None and _spent_max(con, keys, day) >= budget:
|
|
||||||
break
|
|
||||||
_add(con, keys, day, CHUNK_SEC)
|
|
||||||
reserved += CHUNK_SEC
|
|
||||||
|
|
||||||
alive = reserved > elapsed # could we cover the time already elapsed?
|
|
||||||
con.execute(
|
|
||||||
"UPDATE sessions SET last_seen_ts=?, reserved_sec=?, ended=? WHERE session_id=?",
|
|
||||||
(now, reserved, 0 if alive else 1, session_id),
|
|
||||||
)
|
|
||||||
if not alive:
|
|
||||||
_reconcile(con, keys, day, grant_ts, reserved, end_ts=now)
|
|
||||||
return alive
|
|
||||||
|
|
||||||
|
|
||||||
def end(session_id: str) -> None:
|
|
||||||
"""Clean teardown: reconcile to actual elapsed time and refund the unused
|
|
||||||
reservation. Idempotent."""
|
|
||||||
now = time.time()
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
row = con.execute(
|
|
||||||
"SELECT keys, day, grant_ts, reserved_sec, ended FROM sessions WHERE session_id=?",
|
|
||||||
(session_id,),
|
|
||||||
).fetchone()
|
|
||||||
if not row or row[4]:
|
|
||||||
return
|
|
||||||
keys, day, grant_ts, reserved = row[0].split(","), row[1], row[2], int(row[3])
|
|
||||||
_reconcile(con, keys, day, grant_ts, reserved, end_ts=now)
|
|
||||||
con.execute("UPDATE sessions SET ended=1, last_seen_ts=? WHERE session_id=?", (now, session_id))
|
|
||||||
|
|
||||||
|
|
||||||
def sweep() -> None:
|
|
||||||
"""Reap sessions that went silent (crash / closed without a beacon): bill
|
|
||||||
their elapsed time, refund the rest, mark ended. Forfeits ≤ one chunk."""
|
|
||||||
now = time.time()
|
|
||||||
cutoff = now - REAP_AFTER_SEC
|
|
||||||
with _lock, _connect() as con:
|
|
||||||
stale = con.execute(
|
|
||||||
"SELECT session_id, keys, day, grant_ts, last_seen_ts, reserved_sec FROM sessions "
|
|
||||||
"WHERE ended=0 AND last_seen_ts < ?",
|
|
||||||
(cutoff,),
|
|
||||||
).fetchall()
|
|
||||||
for session_id, keys_s, day, grant_ts, last_seen, reserved in stale:
|
|
||||||
_reconcile(con, keys_s.split(","), day, grant_ts, int(reserved), end_ts=last_seen)
|
|
||||||
con.execute("UPDATE sessions SET ended=1 WHERE session_id=?", (session_id,))
|
|
||||||
if stale:
|
|
||||||
logger.debug("swept %d stale session(s)", len(stale))
|
|
||||||
|
|
||||||
|
|
||||||
def _reconcile(con, keys, day: str, grant_ts: float, reserved: int, end_ts: float) -> None:
|
|
||||||
"""Refund reserved-but-unused time. Bill elapsed rounded up to a chunk,
|
|
||||||
capped at what was reserved."""
|
|
||||||
elapsed = max(0.0, end_ts - grant_ts)
|
|
||||||
billed = min(reserved, int(math.ceil(elapsed / CHUNK_SEC) * CHUNK_SEC))
|
|
||||||
refund = reserved - billed
|
|
||||||
if refund > 0:
|
|
||||||
_add(con, keys, day, -refund)
|
|
||||||
-3741
File diff suppressed because it is too large
Load Diff
@@ -1,139 +0,0 @@
|
|||||||
<!doctype html>
|
|
||||||
<html lang="en">
|
|
||||||
<head>
|
|
||||||
<meta charset="UTF-8" />
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
|
|
||||||
<title>Project Docs</title>
|
|
||||||
<style>
|
|
||||||
:root {
|
|
||||||
color-scheme: dark;
|
|
||||||
--bg: #081018;
|
|
||||||
--panel: #0f1b26;
|
|
||||||
--line: rgba(149, 180, 201, 0.18);
|
|
||||||
--text: #edf6fb;
|
|
||||||
--muted: #9db2c2;
|
|
||||||
--accent: #59d0ff;
|
|
||||||
}
|
|
||||||
|
|
||||||
* {
|
|
||||||
box-sizing: border-box;
|
|
||||||
}
|
|
||||||
|
|
||||||
html,
|
|
||||||
body {
|
|
||||||
margin: 0;
|
|
||||||
width: 100%;
|
|
||||||
height: 100%;
|
|
||||||
background: var(--bg);
|
|
||||||
color: var(--text);
|
|
||||||
font-family: Inter, Arial, Helvetica, sans-serif;
|
|
||||||
}
|
|
||||||
|
|
||||||
body {
|
|
||||||
display: grid;
|
|
||||||
grid-template-rows: auto 1fr;
|
|
||||||
}
|
|
||||||
|
|
||||||
.topbar {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: space-between;
|
|
||||||
gap: 16px;
|
|
||||||
padding: 14px 18px;
|
|
||||||
border-bottom: 1px solid var(--line);
|
|
||||||
background: rgba(8, 16, 24, 0.92);
|
|
||||||
backdrop-filter: blur(10px);
|
|
||||||
}
|
|
||||||
|
|
||||||
.title {
|
|
||||||
margin: 0;
|
|
||||||
font-size: 16px;
|
|
||||||
font-weight: 700;
|
|
||||||
letter-spacing: 0.02em;
|
|
||||||
}
|
|
||||||
|
|
||||||
.subtitle {
|
|
||||||
margin: 4px 0 0;
|
|
||||||
color: var(--muted);
|
|
||||||
font-size: 13px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.actions {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 10px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn {
|
|
||||||
display: inline-flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: center;
|
|
||||||
min-height: 38px;
|
|
||||||
padding: 0 14px;
|
|
||||||
border: 1px solid var(--line);
|
|
||||||
border-radius: 10px;
|
|
||||||
color: var(--text);
|
|
||||||
background: var(--panel);
|
|
||||||
text-decoration: none;
|
|
||||||
font-size: 13px;
|
|
||||||
font-weight: 600;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn.primary {
|
|
||||||
border-color: rgba(89, 208, 255, 0.35);
|
|
||||||
color: #021018;
|
|
||||||
background: var(--accent);
|
|
||||||
}
|
|
||||||
|
|
||||||
.frame-wrap {
|
|
||||||
padding: 14px;
|
|
||||||
min-height: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.frame {
|
|
||||||
width: 100%;
|
|
||||||
height: 100%;
|
|
||||||
border: 1px solid var(--line);
|
|
||||||
border-radius: 16px;
|
|
||||||
background: #ffffff;
|
|
||||||
}
|
|
||||||
|
|
||||||
@media (max-width: 760px) {
|
|
||||||
.topbar {
|
|
||||||
flex-direction: column;
|
|
||||||
align-items: flex-start;
|
|
||||||
}
|
|
||||||
|
|
||||||
.actions {
|
|
||||||
width: 100%;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn {
|
|
||||||
flex: 1 1 auto;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<header class="topbar">
|
|
||||||
<div>
|
|
||||||
<h1 class="title">Vocero Project</h1>
|
|
||||||
<p class="subtitle">Embedded technical document viewer for `demo/public/docs.html`.</p>
|
|
||||||
</div>
|
|
||||||
<div class="actions">
|
|
||||||
<a class="btn" href="/">Back to app</a>
|
|
||||||
<a class="btn primary" href="/public/docs.html" target="_blank" rel="noopener">Open raw document</a>
|
|
||||||
</div>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<main class="frame-wrap">
|
|
||||||
<iframe
|
|
||||||
class="frame"
|
|
||||||
src="/public/docs.html"
|
|
||||||
title="Project documentation"
|
|
||||||
loading="eager"
|
|
||||||
referrerpolicy="no-referrer"
|
|
||||||
></iframe>
|
|
||||||
</main>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
File diff suppressed because one or more lines are too long
@@ -1,6 +0,0 @@
|
|||||||
fastapi==0.115.6
|
|
||||||
uvicorn[standard]==0.34.0
|
|
||||||
httpx==0.28.1
|
|
||||||
python-dotenv==1.2.2
|
|
||||||
# [oauth] extra pulls authlib + itsdangerous for attach_huggingface_oauth.
|
|
||||||
huggingface_hub[oauth]>=0.30,<1.0
|
|
||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,189 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* Account — the HF login chip and the daily-limit modal.
|
|
||||||
*
|
|
||||||
* Reads `/api/me` to learn the current tier (anonymous / signed-in / PRO) and
|
|
||||||
* remaining daily talk-time, renders a sign-in pill or a signed-in chip with a
|
|
||||||
* small popover (tier, remaining, sign out, upgrade), and shows the limit modal
|
|
||||||
* when a conversation is refused or cut. The time metering itself lives in
|
|
||||||
* main.js (heartbeat loop) + the server; this module is just the surface.
|
|
||||||
*
|
|
||||||
* Inert unless the deploy is in LB mode (`/api/me` → `{enabled:true}`).
|
|
||||||
*/
|
|
||||||
|
|
||||||
import { $, escHtml } from "./dom.js";
|
|
||||||
|
|
||||||
const PRO_URL = "https://huggingface.co/subscribe/pro";
|
|
||||||
|
|
||||||
// Official multi-color Hugging Face logo, used in the badge + sign-in CTA.
|
|
||||||
const HF_MARK = `<svg class="hf-logo" viewBox="0 0 95 88" fill="none" aria-hidden="true"><path fill="#FFD21E" d="M47.21 76.5a34.75 34.75 0 1 0 0-69.5 34.75 34.75 0 0 0 0 69.5Z"/><path fill="#FF9D0B" d="M81.96 41.75a34.75 34.75 0 1 0-69.5 0 34.75 34.75 0 0 0 69.5 0Zm-73.5 0a38.75 38.75 0 1 1 77.5 0 38.75 38.75 0 0 1-77.5 0Z"/><path fill="#3A3B45" d="M58.5 32.3c1.28.44 1.78 3.06 3.07 2.38a5 5 0 1 0-6.76-2.07c.61 1.15 2.55-.72 3.7-.32ZM34.95 32.3c-1.28.44-1.79 3.06-3.07 2.38a5 5 0 1 1 6.76-2.07c-.61 1.15-2.56-.72-3.7-.32Z"/><path fill="#FF323D" d="M46.96 56.29c9.83 0 13-8.76 13-13.26 0-2.34-1.57-1.6-4.09-.36-2.33 1.15-5.46 2.74-8.9 2.74-7.19 0-13-6.88-13-2.38s3.16 13.26 13 13.26Z"/><path fill="#3A3B45" fill-rule="evenodd" d="M39.43 54a8.7 8.7 0 0 1 5.3-4.49c.4-.12.81.57 1.24 1.28.4.68.82 1.37 1.24 1.37.45 0 .9-.68 1.33-1.35.45-.7.89-1.38 1.32-1.25a8.61 8.61 0 0 1 5 4.17c3.73-2.94 5.1-7.74 5.1-10.7 0-2.34-1.57-1.6-4.09-.36l-.14.07c-2.31 1.15-5.39 2.67-8.77 2.67s-6.45-1.52-8.77-2.67c-2.6-1.29-4.23-2.1-4.23.29 0 3.05 1.46 8.06 5.47 10.97Z" clip-rule="evenodd"/><path fill="#FF9D0B" d="M70.71 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM24.21 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM17.52 48c-1.62 0-3.06.66-4.07 1.87a5.97 5.97 0 0 0-1.33 3.76 7.1 7.1 0 0 0-1.94-.3c-1.55 0-2.95.59-3.94 1.66a5.8 5.8 0 0 0-.8 7 5.3 5.3 0 0 0-1.79 2.82c-.24.9-.48 2.8.8 4.74a5.22 5.22 0 0 0-.37 5.02c1.02 2.32 3.57 4.14 8.52 6.1 3.07 1.22 5.89 2 5.91 2.01a44.33 44.33 0 0 0 10.93 1.6c5.86 0 10.05-1.8 12.46-5.34 3.88-5.69 3.33-10.9-1.7-15.92-2.77-2.78-4.62-6.87-5-7.77-.78-2.66-2.84-5.62-6.25-5.62a5.7 5.7 0 0 0-4.6 2.46c-1-1.26-1.98-2.25-2.86-2.82A7.4 7.4 0 0 0 17.52 48Zm0 4c.51 0 1.14.22 1.82.65 2.14 1.36 6.25 8.43 7.76 11.18.5.92 1.37 1.31 2.14 1.31 1.55 0 2.75-1.53.15-3.48-3.92-2.93-2.55-7.72-.68-8.01.08-.02.17-.02.24-.02 1.7 0 2.45 2.93 2.45 2.93s2.2 5.52 5.98 9.3c3.77 3.77 3.97 6.8 1.22 10.83-1.88 2.75-5.47 3.58-9.16 3.58-3.81 0-7.73-.9-9.92-1.46-.11-.03-13.45-3.8-11.76-7 .28-.54.75-.76 1.34-.76 2.38 0 6.7 3.54 8.57 3.54.41 0 .7-.17.83-.6.79-2.85-12.06-4.05-10.98-8.17.2-.73.71-1.02 1.44-1.02 3.14 0 10.2 5.53 11.68 5.53.11 0 .2-.03.24-.1.74-1.2.33-2.04-4.9-5.2-5.21-3.16-8.88-5.06-6.8-7.33.24-.26.58-.38 1-.38 3.17 0 10.66 6.82 10.66 6.82s2.02 2.1 3.25 2.1c.28 0 .52-.1.68-.38.86-1.46-8.06-8.22-8.56-11.01-.34-1.9.24-2.85 1.31-2.85Z"/><path fill="#FFD21E" d="M38.6 76.69c2.75-4.04 2.55-7.07-1.22-10.84-3.78-3.77-5.98-9.3-5.98-9.3s-.82-3.2-2.69-2.9c-1.87.3-3.24 5.08.68 8.01 3.91 2.93-.78 4.92-2.29 2.17-1.5-2.75-5.62-9.82-7.76-11.18-2.13-1.35-3.63-.6-3.13 2.2.5 2.79 9.43 9.55 8.56 11-.87 1.47-3.93-1.71-3.93-1.71s-9.57-8.71-11.66-6.44c-2.08 2.27 1.59 4.17 6.8 7.33 5.23 3.16 5.64 4 4.9 5.2-.75 1.2-12.28-8.53-13.36-4.4-1.08 4.11 11.77 5.3 10.98 8.15-.8 2.85-9.06-5.38-10.74-2.18-1.7 3.21 11.65 6.98 11.76 7.01 4.3 1.12 15.25 3.49 19.08-2.12Z"/><path fill="#FF9D0B" d="M77.4 48c1.62 0 3.07.66 4.07 1.87a5.97 5.97 0 0 1 1.33 3.76 7.1 7.1 0 0 1 1.95-.3c1.55 0 2.95.59 3.94 1.66a5.8 5.8 0 0 1 .8 7 5.3 5.3 0 0 1 1.78 2.82c.24.9.48 2.8-.8 4.74a5.22 5.22 0 0 1 .37 5.02c-1.02 2.32-3.57 4.14-8.51 6.1-3.08 1.22-5.9 2-5.92 2.01a44.33 44.33 0 0 1-10.93 1.6c-5.86 0-10.05-1.8-12.46-5.34-3.88-5.69-3.33-10.9 1.7-15.92 2.78-2.78 4.63-6.87 5.01-7.77.78-2.66 2.83-5.62 6.24-5.62a5.7 5.7 0 0 1 4.6 2.46c1-1.26 1.98-2.25 2.87-2.82A7.4 7.4 0 0 1 77.4 48Zm0 4c-.51 0-1.13.22-1.82.65-2.13 1.36-6.25 8.43-7.76 11.18a2.43 2.43 0 0 1-2.14 1.31c-1.54 0-2.75-1.53-.14-3.48 3.91-2.93 2.54-7.72.67-8.01a1.54 1.54 0 0 0-.24-.02c-1.7 0-2.45 2.93-2.45 2.93s-2.2 5.52-5.97 9.3c-3.78 3.77-3.98 6.8-1.22 10.83 1.87 2.75 5.47 3.58 9.15 3.58 3.82 0 7.73-.9 9.93-1.46.1-.03 13.45-3.8 11.76-7-.29-.54-.75-.76-1.34-.76-2.38 0-6.71 3.54-8.57 3.54-.42 0-.71-.17-.83-.6-.8-2.85 12.05-4.05 10.97-8.17-.19-.73-.7-1.02-1.44-1.02-3.14 0-10.2 5.53-11.68 5.53-.1 0-.19-.03-.23-.1-.74-1.2-.34-2.04 4.88-5.2 5.23-3.16 8.9-5.06 6.8-7.33-.23-.26-.57-.38-.98-.38-3.18 0-10.67 6.82-10.67 6.82s-2.02 2.1-3.24 2.1a.74.74 0 0 1-.68-.38c-.87-1.46 8.05-8.22 8.55-11.01.34-1.9-.24-2.85-1.31-2.85Z"/><path fill="#FFD21E" d="M56.33 76.69c-2.75-4.04-2.56-7.07 1.22-10.84 3.77-3.77 5.97-9.3 5.97-9.3s.82-3.2 2.7-2.9c1.86.3 3.23 5.08-.68 8.01-3.92 2.93.78 4.92 2.28 2.17 1.51-2.75 5.63-9.82 7.76-11.18 2.13-1.35 3.64-.6 3.13 2.2-.5 2.79-9.42 9.55-8.55 11 .86 1.47 3.92-1.71 3.92-1.71s9.58-8.71 11.66-6.44c2.08 2.27-1.58 4.17-6.8 7.33-5.23 3.16-5.63 4-4.9 5.2.75 1.2 12.28-8.53 13.36-4.4 1.08 4.11-11.76 5.3-10.97 8.15.8 2.85 9.05-5.38 10.74-2.18 1.69 3.21-11.65 6.98-11.76 7.01-4.31 1.12-15.26 3.49-19.08-2.12Z"/></svg>`;
|
|
||||||
|
|
||||||
/** @param {number} sec @returns {string} "m:ss" */
|
|
||||||
function fmt(sec) {
|
|
||||||
const s = Math.max(0, Math.round(sec));
|
|
||||||
return `${Math.floor(s / 60)}:${String(s % 60).padStart(2, "0")}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
export class Account {
|
|
||||||
constructor() {
|
|
||||||
/** @type {HTMLElement} */
|
|
||||||
this._root = $("#account");
|
|
||||||
/** @type {HTMLDialogElement} */
|
|
||||||
this._modal = $("#limit-modal");
|
|
||||||
this._modalTitle = $("#limit-title");
|
|
||||||
this._modalMsg = $("#limit-msg");
|
|
||||||
this._modalNote = $("#limit-note");
|
|
||||||
/** @type {HTMLAnchorElement} */
|
|
||||||
this._modalCta = /** @type {any} */ ($("#limit-cta"));
|
|
||||||
|
|
||||||
/** @type {{enabled:boolean, auth?:boolean, loggedIn?:boolean, username?:string, avatar?:string, tier?:string, remainingSec?:number|null, limitSec?:number|null, loginUrl?:string|null, logoutUrl?:string|null}} */
|
|
||||||
this._me = { enabled: false };
|
|
||||||
this._popoverOpen = false;
|
|
||||||
|
|
||||||
$("#limit-close").addEventListener("click", () => this._modal.close());
|
|
||||||
this._modal.addEventListener("click", (e) => {
|
|
||||||
if (e.target === this._modal) this._modal.close();
|
|
||||||
});
|
|
||||||
// Close the popover on an outside click.
|
|
||||||
document.addEventListener("click", (e) => {
|
|
||||||
if (this._popoverOpen && !this._root.contains(/** @type {Node} */ (e.target))) {
|
|
||||||
this._closePopover();
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
get tier() {
|
|
||||||
return this._me.tier || "anon";
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Fetch `/api/me` and (re)render the chip. Safe to call repeatedly (load,
|
|
||||||
* after the OAuth redirect, after a conversation ends). */
|
|
||||||
async refresh() {
|
|
||||||
try {
|
|
||||||
const res = await fetch("api/me");
|
|
||||||
this._me = res.ok ? await res.json() : { enabled: false };
|
|
||||||
} catch {
|
|
||||||
this._me = { enabled: false };
|
|
||||||
}
|
|
||||||
this._render();
|
|
||||||
}
|
|
||||||
|
|
||||||
_render() {
|
|
||||||
const me = this._me;
|
|
||||||
if (!me.enabled) {
|
|
||||||
this._root.hidden = true;
|
|
||||||
this._root.innerHTML = "";
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
this._root.hidden = false;
|
|
||||||
|
|
||||||
if (!me.loggedIn) {
|
|
||||||
// Signed-out: a sign-in pill (only when OAuth is actually available).
|
|
||||||
if (me.auth && me.loginUrl) {
|
|
||||||
this._root.innerHTML = `<a class="signin-pill" href="${escHtml(me.loginUrl)}" title="Sign in for more time"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M15 3h4a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2h-4"/><polyline points="10 17 15 12 10 7"/><line x1="15" y1="12" x2="3" y2="12"/></svg><span>Sign in</span></a>`;
|
|
||||||
} else {
|
|
||||||
this._root.innerHTML = "";
|
|
||||||
this._root.hidden = true;
|
|
||||||
}
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Signed-in: avatar + handle chip that toggles a popover.
|
|
||||||
const isPro = me.tier === "pro";
|
|
||||||
// Org members get unlimited usage too, but aren't PRO — don't brand them so.
|
|
||||||
const isUnlimited = isPro || me.tier === "org";
|
|
||||||
const avatar = me.avatar
|
|
||||||
? `<img class="account-avatar" src="${escHtml(me.avatar)}" alt="" />`
|
|
||||||
: `<span class="account-avatar account-avatar-fallback">${escHtml((me.username || "?")[0].toUpperCase())}</span>`;
|
|
||||||
const remaining =
|
|
||||||
isUnlimited || me.remainingSec == null
|
|
||||||
? "Unlimited"
|
|
||||||
: `${fmt(me.remainingSec)} left today`;
|
|
||||||
const tierLabel = isPro ? "PRO" : isUnlimited ? "Team" : "Free";
|
|
||||||
|
|
||||||
this._root.innerHTML = `
|
|
||||||
<button id="account-chip" class="account-chip" aria-haspopup="true" aria-expanded="false">
|
|
||||||
${avatar}
|
|
||||||
<span class="account-handle">${escHtml(me.username || "you")}</span>
|
|
||||||
${isPro
|
|
||||||
? '<span class="account-pro">PRO</span>'
|
|
||||||
: me.tier === "org"
|
|
||||||
? '<span class="account-pro account-team">TEAM</span>'
|
|
||||||
: ""}
|
|
||||||
</button>
|
|
||||||
<div id="account-pop" class="account-pop" hidden>
|
|
||||||
<div class="account-pop-row account-pop-name">${escHtml(me.username || "you")}</div>
|
|
||||||
<div class="account-pop-row account-pop-meta">
|
|
||||||
<span class="account-tier">${tierLabel}</span>
|
|
||||||
<span class="account-remaining">${escHtml(remaining)}</span>
|
|
||||||
</div>
|
|
||||||
${isUnlimited ? "" : `<a class="account-pop-link" href="${PRO_URL}" target="_blank" rel="noopener">Upgrade to PRO</a>`}
|
|
||||||
<a class="account-pop-link account-signout" href="${escHtml(me.logoutUrl || "#")}">Sign out</a>
|
|
||||||
</div>`;
|
|
||||||
|
|
||||||
const chip = $("#account-chip");
|
|
||||||
chip.addEventListener("click", (e) => {
|
|
||||||
e.stopPropagation();
|
|
||||||
this._popoverOpen ? this._closePopover() : this._openPopover();
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
_openPopover() {
|
|
||||||
const pop = document.getElementById("account-pop");
|
|
||||||
const chip = document.getElementById("account-chip");
|
|
||||||
if (!pop || !chip) return;
|
|
||||||
pop.hidden = false;
|
|
||||||
chip.setAttribute("aria-expanded", "true");
|
|
||||||
this._popoverOpen = true;
|
|
||||||
}
|
|
||||||
|
|
||||||
_closePopover() {
|
|
||||||
const pop = document.getElementById("account-pop");
|
|
||||||
const chip = document.getElementById("account-chip");
|
|
||||||
if (pop) pop.hidden = true;
|
|
||||||
if (chip) chip.setAttribute("aria-expanded", "false");
|
|
||||||
this._popoverOpen = false;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Show the limit modal for a tier — used both when a conversation is refused
|
|
||||||
* at start (402) and when a live one is cut (heartbeat `expired`).
|
|
||||||
* @param {string} [tier]
|
|
||||||
*/
|
|
||||||
showLimit(tier = this.tier) {
|
|
||||||
const canSignIn = this._me.auth && this._me.loginUrl;
|
|
||||||
this._modalTitle.textContent = "Thanks for chatting!";
|
|
||||||
if (tier === "anon") {
|
|
||||||
this._modalMsg.textContent =
|
|
||||||
"Guest conversations run for 5 minutes. Sign in with Hugging Face to get 10 minutes a day for free, and PRO members chat with no limit at all.";
|
|
||||||
this._modalNote.textContent = "Your free minutes refresh tomorrow.";
|
|
||||||
if (canSignIn) {
|
|
||||||
this._modalCta.innerHTML = `${HF_MARK}<span>Sign in with Hugging Face</span>`;
|
|
||||||
this._modalCta.href = /** @type {string} */ (this._me.loginUrl);
|
|
||||||
this._modalCta.hidden = false;
|
|
||||||
} else {
|
|
||||||
this._modalCta.hidden = true;
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
// Signed-in, non-PRO.
|
|
||||||
this._modalMsg.textContent =
|
|
||||||
"You've enjoyed your 10 minutes for today. Go PRO for unlimited conversations and to support open source AI.";
|
|
||||||
this._modalNote.textContent = "Or come back tomorrow. Your minutes reset daily.";
|
|
||||||
this._modalCta.innerHTML = "<span>Upgrade to PRO</span>";
|
|
||||||
this._modalCta.href = PRO_URL;
|
|
||||||
this._modalCta.hidden = false;
|
|
||||||
}
|
|
||||||
if (!this._modal.open) this._modal.showModal();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Show a warm "we're at capacity" message when even the waiting line is full.
|
|
||||||
* Reuses the limit modal shell; no call-to-action, just reassurance. */
|
|
||||||
showBusy() {
|
|
||||||
this._modalTitle.textContent = "Hugged to the limit 🤗";
|
|
||||||
this._modalMsg.textContent =
|
|
||||||
"Every slot and the whole line are full right now. Too much love! Grab a coffee and pop back in a minute.";
|
|
||||||
this._modalNote.textContent = "A spot usually opens up within a few minutes.";
|
|
||||||
this._modalCta.hidden = true;
|
|
||||||
if (!this._modal.open) this._modal.showModal();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,823 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* ChatView — owns the whole conversation surface: the slide-in history panel,
|
|
||||||
* the ephemeral on-orb bubbles, and all the transcript/tool/streaming
|
|
||||||
* bookkeeping. main.js wires the realtime client's events straight to the
|
|
||||||
* `on*` methods here and otherwise doesn't touch chat state.
|
|
||||||
*
|
|
||||||
* Two parallel surfaces share one shape (see `_buildMessageEl`):
|
|
||||||
* - ephemeral bubbles (`.bubble` / `.bubble-*`) fade on a timer
|
|
||||||
* - persistent history (`.hist-msg` / `.hist-*`) the durable panel log
|
|
||||||
*
|
|
||||||
* Keying:
|
|
||||||
* - user transcripts by the server's `item_id` — a speculative continuation
|
|
||||||
* REUSES it, so both segments land in one row/bubble; deltas are CUMULATIVE
|
|
||||||
* (each carries the full sentence so far), so we replace text wholesale.
|
|
||||||
* - assistant transcripts by `response_id`, so a cancelled speculative reply
|
|
||||||
* can be marked interrupted without erasing what was already shown.
|
|
||||||
*/
|
|
||||||
|
|
||||||
import { $, escHtml, DEBUG } from "./dom.js";
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @param {string} hypothesisId
|
|
||||||
* @param {string} location
|
|
||||||
* @param {string} msg
|
|
||||||
* @param {Record<string, unknown>} [data]
|
|
||||||
*/
|
|
||||||
function reportChatDebug(hypothesisId, location, msg, data = {}) {
|
|
||||||
void hypothesisId;
|
|
||||||
void location;
|
|
||||||
void msg;
|
|
||||||
void data;
|
|
||||||
}
|
|
||||||
|
|
||||||
const WRENCH_PATH = `<path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/>`;
|
|
||||||
const CHAT_BUBBLE_SVG = `<svg width="28" height="28" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/></svg>`;
|
|
||||||
const EMPTY_STATE_HTML = `<div id="chat-empty" class="chat-empty">${CHAT_BUBBLE_SVG}<span class="chat-empty-title">No messages yet</span><span class="chat-empty-hint">Tap the orb and start talking</span></div>`;
|
|
||||||
/**
|
|
||||||
* @param {string} prefix
|
|
||||||
* @param {string} [sep]
|
|
||||||
* @returns {string} safe filename prefix (a-z 0-9 _ - no spaces / slashes / unicode),
|
|
||||||
* sanitised so any OS accepts it.
|
|
||||||
*/
|
|
||||||
function _safeName(prefix, sep = "-") {
|
|
||||||
const base = String(prefix || "").toLowerCase().trim()
|
|
||||||
.normalize("NFKD").replace(/[^\x20-\x7E]/g, "")
|
|
||||||
.replace(/[^a-z0-9]+/gi, sep || "-")
|
|
||||||
.replace(new RegExp(`^${sep}+|${sep}+$`, "g"), "");
|
|
||||||
return base || "vocero";
|
|
||||||
}
|
|
||||||
|
|
||||||
export class ChatView {
|
|
||||||
constructor() {
|
|
||||||
// In the V2 layout the chat panel lives inline in #app, shown permanently
|
|
||||||
// below the orb — there is NO toggle button, close button, backdrop, or
|
|
||||||
// unread badge any more. The strict $() helper THROWS when the selector is
|
|
||||||
// missing, so for the optional overlay-era refs we MUST use plain
|
|
||||||
// document.querySelector (returns null when absent). Only the
|
|
||||||
// always-present inline refs go through $().
|
|
||||||
/** @type {HTMLButtonElement | null} */
|
|
||||||
this._chatBtn = /** @type {HTMLButtonElement | null} */ (document.querySelector("#chat-btn"));
|
|
||||||
/** @type {HTMLSpanElement | null} */
|
|
||||||
this._chatBadge = /** @type {HTMLSpanElement | null} */ (document.querySelector("#chat-badge"));
|
|
||||||
/** @type {HTMLElement} */
|
|
||||||
this._chatPanel = $("#chat-panel");
|
|
||||||
/** @type {HTMLElement | null} */
|
|
||||||
this._chatPanelBackdrop = /** @type {HTMLElement | null} */ (document.querySelector("#chat-panel-backdrop"));
|
|
||||||
/** @type {HTMLButtonElement | null} */
|
|
||||||
this._chatPanelClose = /** @type {HTMLButtonElement | null} */ (document.querySelector("#chat-panel-close"));
|
|
||||||
/** @type {HTMLElement} */
|
|
||||||
this._chatHistory = $("#chat-history");
|
|
||||||
/** @type {HTMLElement} */
|
|
||||||
this._bubbleStack = $("#bubble-stack");
|
|
||||||
/** @type {HTMLButtonElement | null} Export conversation download button in header. */
|
|
||||||
this._exportBtn = /** @type {HTMLButtonElement | null} */ (document.getElementById("chat-export-btn"));
|
|
||||||
if (this._exportBtn) {
|
|
||||||
this._exportBtn.addEventListener("click", () => this.exportConversation());
|
|
||||||
}
|
|
||||||
|
|
||||||
// Inline chat is always "open" (permanently visible on screen).
|
|
||||||
this._panelOpen = true;
|
|
||||||
// ── Always-bottom scroll bookkeeping ─────────────────────────────────
|
|
||||||
// Active timer for 4-step retry chain (0/30/100/250ms snap-to-bottom).
|
|
||||||
/** @type {number | null} */
|
|
||||||
this._scrollTimerId = null;
|
|
||||||
// Current step index inside a retry chain (0..3).
|
|
||||||
this._scrollStep = 0;
|
|
||||||
|
|
||||||
// ── User transcript state (keyed by item_id) ───────────────────────────
|
|
||||||
/** @type {Map<string, HTMLElement>} */
|
|
||||||
this._userHistByItem = new Map();
|
|
||||||
/** @type {HTMLElement | null} */
|
|
||||||
this._activeUserBubble = null;
|
|
||||||
this._activeUserItemId = "";
|
|
||||||
// Monotonic counter for synthesizing unique keys when the server omits an
|
|
||||||
// item_id / response_id, so id-less messages never collapse onto each other.
|
|
||||||
this._anonSeq = 0;
|
|
||||||
|
|
||||||
// ── Assistant transcript state (keyed by response_id) ──────────────────
|
|
||||||
/** @type {Map<string, { bubble: HTMLElement, hist: HTMLElement }>} */
|
|
||||||
this._asstByResp = new Map();
|
|
||||||
|
|
||||||
// ── Ephemeral bubble auto-dismiss ──────────────────────────────────────
|
|
||||||
// Per-element expiry (epoch ms). A bubble fades once its expiry passes —
|
|
||||||
// but only in stack order (see _reapBubbles). Refreshing the expiry keeps a
|
|
||||||
// bubble alive while it updates (e.g. the live user utterance).
|
|
||||||
/** @type {WeakMap<HTMLElement, number>} */
|
|
||||||
this._bubbleExpiry = new WeakMap();
|
|
||||||
// Single pending reaper handle: one timer for the whole stack (not one per
|
|
||||||
// bubble) so dismissal is strictly oldest-first regardless of per-bubble delays.
|
|
||||||
this._reaperHandle = 0;
|
|
||||||
|
|
||||||
// ── New inline-chat handlers: only wire when old overlay buttons exist.
|
|
||||||
if (this._chatBtn) this._chatBtn.addEventListener("click", () => (this._panelOpen ? this._closePanel() : this._openPanel()));
|
|
||||||
if (this._chatPanelClose) this._chatPanelClose.addEventListener("click", () => this._closePanel());
|
|
||||||
if (this._chatPanelBackdrop) this._chatPanelBackdrop.addEventListener("click", () => this._closePanel());
|
|
||||||
document.addEventListener("keydown", (e) => {
|
|
||||||
// Old behaviour (ESC closes overlay) is a no-op on the inline permanent
|
|
||||||
// panel; only execute it when a close button actually exists.
|
|
||||||
if (e.key === "Escape" && this._chatPanelClose && this._panelOpen) this._closePanel();
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── User name / initials helpers ────────────────────────────────────────
|
|
||||||
/** @returns {string} display name for the current user (e.g. "Alessandro").
|
|
||||||
* Reuses extractVoceroUserName + initialsForName exposed on window by main.js
|
|
||||||
* so we don't duplicate the nested-object lookup logic (profile/user/customer
|
|
||||||
* keys, first+last name fallback, etc.). Falls back to "Usuario" (Spanish)
|
|
||||||
* when no profile has been resolved yet — never returns "" because the
|
|
||||||
* white-theme chat label is rendered into a pseudo-element that would show
|
|
||||||
* a lonely timestamp otherwise.
|
|
||||||
*/
|
|
||||||
_resolveUserName() {
|
|
||||||
const ex = typeof window._s2sExtractName === "function" ? window._s2sExtractName : null;
|
|
||||||
const profile =
|
|
||||||
/** @type {Record<string, unknown> | null} */ (/** @type {unknown} */ (window.voceroUserProfile)) || null;
|
|
||||||
const full = ex ? ex(profile) : "";
|
|
||||||
if (!full) return "Usuario";
|
|
||||||
// Return ONLY the first token so the narrow chat label line fits:
|
|
||||||
// "Alessandro Valenti" → "Alessandro", matches the screenshot style.
|
|
||||||
const first = String(full).trim().split(/\s+/)[0] || full;
|
|
||||||
return first;
|
|
||||||
}
|
|
||||||
/** @returns {string} 2-letter uppercase initials, e.g. "Alessandro Valenti" → "AV". */
|
|
||||||
_resolveUserInitials() {
|
|
||||||
const mk = typeof window._s2sInitialsFor === "function" ? window._s2sInitialsFor : null;
|
|
||||||
const profile =
|
|
||||||
/** @type {Record<string, unknown> | null} */ (/** @type {unknown} */ (window.voceroUserProfile)) || null;
|
|
||||||
const ex = typeof window._s2sExtractName === "function" ? window._s2sExtractName : null;
|
|
||||||
const full = ex ? ex(profile) : "";
|
|
||||||
if (mk && full) {
|
|
||||||
const init = mk(full);
|
|
||||||
if (init) return init;
|
|
||||||
}
|
|
||||||
return "AV";
|
|
||||||
}
|
|
||||||
/** @returns {string} "HH:MM" 24h using the current locale, no seconds. */
|
|
||||||
_nowStamp() {
|
|
||||||
try {
|
|
||||||
return new Date().toLocaleTimeString([], { hour: "2-digit", minute: "2-digit" });
|
|
||||||
} catch (_) {
|
|
||||||
const d = new Date();
|
|
||||||
const hh = String(d.getHours()).padStart(2, "0");
|
|
||||||
const mm = String(d.getMinutes()).padStart(2, "0");
|
|
||||||
return `${hh}:${mm}`;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/** Assistant display name — fixed Spanish label, same as the topbar brand. */
|
|
||||||
get _assistantName() {
|
|
||||||
return "Vocero";
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Panel ───────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
_openPanel() {
|
|
||||||
this._panelOpen = true;
|
|
||||||
if (this._chatPanel) this._chatPanel.classList.add("open");
|
|
||||||
if (this._chatBadge) this._chatBadge.classList.remove("visible");
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
_closePanel() {
|
|
||||||
// Closing disabled on the new inline-permanent layout; guard so if the
|
|
||||||
// overlay UI is still in use by another variant it continues to work.
|
|
||||||
if (!this._chatPanelClose) return;
|
|
||||||
this._panelOpen = false;
|
|
||||||
if (this._chatPanel) this._chatPanel.classList.remove("open");
|
|
||||||
}
|
|
||||||
|
|
||||||
// Scroll the chat history to place the LAST message's FINAL LINE fully above
|
|
||||||
// the sticky footer bar.
|
|
||||||
//
|
|
||||||
// DESIGN NOTES (2026-09-04, after user report "scroll non scrolla
|
|
||||||
// sufficientemente"):
|
|
||||||
// 1. NO early-return on "already queued". The previous coalescing flag was
|
|
||||||
// the root cause of tail-clipping: the 1st incoming delta would queue a
|
|
||||||
// rAF, every subsequent delta (up to ~30/frame) was SKIPPED, then the
|
|
||||||
// rAF ran against the scrollHeight of the 1st delta only — so the tail
|
|
||||||
// words appended later were never accounted for.
|
|
||||||
// 2. Instead we CANCEL any pending timer chain and RESCHEDULE from scratch
|
|
||||||
// every single time `_scrollToBottom()` is called. This means the very
|
|
||||||
// last append/update call always "wins" the chain — so we scroll
|
|
||||||
// against the final grown content, not an intermediate snapshot.
|
|
||||||
// 3. We run 4 scroll attempts staggered at [0ms, 30ms, 100ms, 250ms].
|
|
||||||
// Layout is async: line-wrap, font metrics, the scroll anchoring reset,
|
|
||||||
// and tool-row height expansions all resolve in later micro/task frames.
|
|
||||||
// Attempt 0 = immediate post-append; 30 = after one layout-pass; 100 =
|
|
||||||
// after any Chrome scroll-anchor reflow; 250 = after React-like deferred
|
|
||||||
// re-renders (if any). At 250ms we are guaranteed stable scrollHeight.
|
|
||||||
// 4. Instead of scrollTop = scrollHeight we use scrollTo({top: 9999999}).
|
|
||||||
// The browser clamps automatically; using a giant literal avoids any
|
|
||||||
// race where scrollHeight was read before a layout commit propagated.
|
|
||||||
_scrollToBottom() {
|
|
||||||
if (!this._chatHistory) return;
|
|
||||||
const h = this._chatHistory;
|
|
||||||
// Cancel any in-flight attempt chain and start fresh — guarantees the LAST
|
|
||||||
// caller wins (so if 100 tokens arrive in 1 frame we don't run 100 chains,
|
|
||||||
// we only run ONE chain: the one scheduled from the 100th token append).
|
|
||||||
if (this._scrollTimerId != null) {
|
|
||||||
clearTimeout(this._scrollTimerId);
|
|
||||||
this._scrollTimerId = null;
|
|
||||||
}
|
|
||||||
// Also reset any per-chain tracking so we re-run all 4 steps from the new
|
|
||||||
// point-in-time content height.
|
|
||||||
this._scrollStep = 0;
|
|
||||||
|
|
||||||
const snap = () => {
|
|
||||||
// Use Number.MAX_SAFE_INTEGER: browsers clamp to the real max
|
|
||||||
// (scrollHeight - clientHeight) automatically. This avoids ANY race
|
|
||||||
// where we read scrollHeight at an intermediate layout commit, then
|
|
||||||
// text wraps to more lines later and the scroll delta is short by a
|
|
||||||
// few pixels (the exact bug reported by the user where the last line
|
|
||||||
// was ½ visible behind the floating footer bar).
|
|
||||||
const BIG = Number.MAX_SAFE_INTEGER;
|
|
||||||
try { h.scrollTo({ top: BIG, behavior: "instant" }); }
|
|
||||||
catch (_) {
|
|
||||||
// Fallback for browsers without instant behavior option support
|
|
||||||
h.scrollTop = BIG;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
const step = () => {
|
|
||||||
snap();
|
|
||||||
this._scrollStep++;
|
|
||||||
// 5 attempts, staggered: next delay in ms per step index
|
|
||||||
// step 0 (already run above): immediate
|
|
||||||
// step 1: +30ms after one layout pass
|
|
||||||
// step 2: +70ms after line-wrap reflow (most-common "one word left on
|
|
||||||
// line 7" tail-clipping gets fixed here)
|
|
||||||
// step 3: +150ms after Chrome scroll-anchor reflow
|
|
||||||
// step 4: +250ms final after deferred tool-row / image layout
|
|
||||||
// stop: step 5 reached → exit.
|
|
||||||
const nextDelay = [30, 70, 150, 250][this._scrollStep];
|
|
||||||
if (nextDelay != null) {
|
|
||||||
this._scrollTimerId = setTimeout(step, nextDelay);
|
|
||||||
} else {
|
|
||||||
// Step 4 done → release guard. The id is kept so a new incoming call
|
|
||||||
// can cancel a mid-flight chain cleanly.
|
|
||||||
this._scrollTimerId = null;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Attempt 0 — IMMEDIATELY (same microtask as append — catches 90% of cases).
|
|
||||||
step();
|
|
||||||
}
|
|
||||||
|
|
||||||
_markUnread() {
|
|
||||||
// New inline layout: the chat is always visible so no "unread badge" UX.
|
|
||||||
// If we're running the old overlay variant we still show it.
|
|
||||||
if (this._panelOpen) {
|
|
||||||
this._scrollToBottom();
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
if (this._chatBadge) this._chatBadge.classList.add("visible");
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Shared rendering ──────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Build a role-labelled message element. Ephemeral bubbles and persistent
|
|
||||||
* history rows share the same shape and differ only in their class prefix
|
|
||||||
* (`bubble`/`bubble-*` vs `hist-msg`/`hist-*`).
|
|
||||||
*
|
|
||||||
* White-theme label + avatar data attributes (populated on both the outer
|
|
||||||
* wrapper and the inner body):
|
|
||||||
* wrapper .{hist-msg|bubble}.user → data-user-name / data-user-initials / data-timestamp
|
|
||||||
* wrapper .{hist-msg|bubble}.assistant → data-asst-name / data-timestamp
|
|
||||||
* inner .{hist|bubble}-body → data-user-name | data-asst-name + data-timestamp
|
|
||||||
*
|
|
||||||
* The .{hist|bubble}-body::before pseudo-element in style.css uses attr()
|
|
||||||
* to render the "Nombre HH:MM" label aligned to the bubble corner; the
|
|
||||||
* wrapper .user::before pseudo uses attr(data-user-initials) to paint the
|
|
||||||
* circular avatar's 2-letter monogram.
|
|
||||||
*
|
|
||||||
* @param {{ container: string, prefix: string, role: "user"|"assistant", text: string, partial?: boolean }} o
|
|
||||||
* @returns {HTMLElement}
|
|
||||||
*/
|
|
||||||
_buildMessageEl({ container, prefix, role, text, partial = false }) {
|
|
||||||
const el = document.createElement("div");
|
|
||||||
el.className = `${container} ${role}`;
|
|
||||||
const label = role === "user" ? "You" : "Assistant";
|
|
||||||
const stamp = this._nowStamp();
|
|
||||||
if (role === "user") {
|
|
||||||
const uName = this._resolveUserName();
|
|
||||||
const uInit = this._resolveUserInitials();
|
|
||||||
el.setAttribute("data-user-name", uName);
|
|
||||||
el.setAttribute("data-user-initials", uInit);
|
|
||||||
el.setAttribute("data-timestamp", stamp);
|
|
||||||
} else {
|
|
||||||
el.setAttribute("data-asst-name", this._assistantName);
|
|
||||||
el.setAttribute("data-timestamp", stamp);
|
|
||||||
}
|
|
||||||
el.innerHTML = `<div class="${prefix}-role">${label}</div><div class="${prefix}-body${partial ? " partial" : ""}">${escHtml(text)}</div>`;
|
|
||||||
const body = /** @type {HTMLDivElement} */ (el.lastElementChild);
|
|
||||||
if (body) {
|
|
||||||
if (role === "user") {
|
|
||||||
body.setAttribute("data-user-name", el.getAttribute("data-user-name") || "");
|
|
||||||
} else {
|
|
||||||
body.setAttribute("data-asst-name", this._assistantName);
|
|
||||||
}
|
|
||||||
body.setAttribute("data-timestamp", stamp);
|
|
||||||
}
|
|
||||||
return el;
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Ephemeral bubbles ───────────────────────────────────────────────────
|
|
||||||
|
|
||||||
/** @param {"user"|"assistant"|"tool"} role @param {string} text @returns {HTMLElement} */
|
|
||||||
_spawnBubble(role, text) {
|
|
||||||
let el;
|
|
||||||
if (role === "tool") {
|
|
||||||
el = document.createElement("div");
|
|
||||||
el.className = "bubble tool";
|
|
||||||
el.innerHTML = `<svg class="bubble-tool-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">${WRENCH_PATH}</svg><span class="bubble-tool-text">${escHtml(text)}</span>`;
|
|
||||||
} else {
|
|
||||||
el = this._buildMessageEl({ container: "bubble", prefix: "bubble", role, text });
|
|
||||||
}
|
|
||||||
this._bubbleStack.appendChild(el);
|
|
||||||
// Cap the stack at 3, but never evict the bubble the caller is still
|
|
||||||
// actively updating (the live user bubble) — drop the next-oldest instead.
|
|
||||||
const visible = /** @type {HTMLElement[]} */ ([...this._bubbleStack.querySelectorAll(".bubble:not(.out)")]);
|
|
||||||
if (visible.length > 3) {
|
|
||||||
this._dismissBubble(visible.find((b) => b !== this._activeUserBubble) ?? visible[0]);
|
|
||||||
}
|
|
||||||
requestAnimationFrame(() => el.classList.add("in"));
|
|
||||||
return el;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {HTMLElement} el @param {string} text */
|
|
||||||
_updateBubbleText(el, text) {
|
|
||||||
const t = el.querySelector(".bubble-body");
|
|
||||||
if (t) t.textContent = text;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {HTMLElement} el */
|
|
||||||
_dismissBubble(el) {
|
|
||||||
if (!el || el.classList.contains("out")) return; // idempotent
|
|
||||||
this._bubbleExpiry.delete(el);
|
|
||||||
el.classList.remove("in");
|
|
||||||
el.classList.add("out");
|
|
||||||
const remove = () => el.remove();
|
|
||||||
el.addEventListener("transitionend", remove, { once: true });
|
|
||||||
// Fallback: transitionend never fires if the bubble's visual state didn't
|
|
||||||
// change (dismissed pre-paint) or the tab is backgrounded. The transition
|
|
||||||
// is 0.3s, so force removal a little after.
|
|
||||||
setTimeout(remove, 400);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Fade bubbles whose expiry has passed — strictly oldest-first. We walk the
|
|
||||||
* stack top (oldest) to bottom (newest) and stop at the first bubble still
|
|
||||||
* alive: nothing newer may leave while an older bubble is still on screen. A
|
|
||||||
* bubble that keeps updating pushes its own expiry forward, so it (and
|
|
||||||
* everything behind it) stays put until it finally goes quiet.
|
|
||||||
*/
|
|
||||||
_reapBubbles() {
|
|
||||||
this._reaperHandle = 0;
|
|
||||||
const now = Date.now();
|
|
||||||
const visible = /** @type {HTMLElement[]} */ ([...this._bubbleStack.querySelectorAll(".bubble:not(.out)")]);
|
|
||||||
let nextWake = Infinity;
|
|
||||||
for (const el of visible) {
|
|
||||||
const exp = this._bubbleExpiry.get(el) ?? now; // no expiry recorded → treat as due
|
|
||||||
if (exp <= now) {
|
|
||||||
this._dismissBubble(el);
|
|
||||||
} else {
|
|
||||||
// Oldest survivor isn't due yet; stop so nothing newer leaves before it.
|
|
||||||
nextWake = exp;
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (nextWake !== Infinity) {
|
|
||||||
this._reaperHandle = setTimeout(() => this._reapBubbles(), Math.max(50, nextWake - Date.now()));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* (Re)arm a bubble's auto-dismiss by pushing its expiry out by `delay`.
|
|
||||||
* Calling it again resets the countdown — so a bubble that keeps updating
|
|
||||||
* stays on screen and only fades once it goes quiet. Removal is ordered by the
|
|
||||||
* shared reaper, so the oldest bubble always disappears first.
|
|
||||||
* @param {HTMLElement} el @param {number} [delay]
|
|
||||||
*/
|
|
||||||
_bumpDismiss(el, delay = 4000) {
|
|
||||||
this._bubbleExpiry.set(el, Date.now() + delay);
|
|
||||||
if (!this._reaperHandle) this._reaperHandle = setTimeout(() => this._reapBubbles(), delay);
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── History ───────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
/** Render the empty-state placeholder into the history panel. */
|
|
||||||
renderEmptyState() {
|
|
||||||
this._chatHistory.innerHTML = EMPTY_STATE_HTML;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Reset the panel to the empty state and clear the unread badge. */
|
|
||||||
clear() {
|
|
||||||
this.renderEmptyState();
|
|
||||||
if (this._chatBadge) this._chatBadge.classList.remove("visible");
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {"user"|"assistant"} role @param {string} text @param {boolean} partial @returns {HTMLElement} */
|
|
||||||
_appendHistMsg(role, text, partial) {
|
|
||||||
const empty = this._chatHistory.querySelector(".chat-empty");
|
|
||||||
if (empty) empty.remove();
|
|
||||||
const el = this._buildMessageEl({ container: "hist-msg", prefix: "hist", role, text, partial });
|
|
||||||
this._chatHistory.appendChild(el);
|
|
||||||
this._scrollToBottom();
|
|
||||||
return el;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {HTMLElement | null} el @param {string} text @param {boolean} partial */
|
|
||||||
_updateHistMsg(el, text, partial) {
|
|
||||||
if (!el) return;
|
|
||||||
const body = /** @type {HTMLElement | null} */ (el.querySelector(".hist-body"));
|
|
||||||
if (!body) return;
|
|
||||||
body.textContent = text;
|
|
||||||
body.classList.toggle("partial", partial);
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Append a tool-call row to the conversation. We only add it once the tool
|
|
||||||
* has run, so the expandable toggle carries BOTH the call input and its result.
|
|
||||||
* @param {string} name @param {string} argsJson @param {string} output
|
|
||||||
*/
|
|
||||||
_appendHistTool(name, argsJson, output) {
|
|
||||||
const empty = this._chatHistory.querySelector(".chat-empty");
|
|
||||||
if (empty) empty.remove();
|
|
||||||
let pretty = argsJson;
|
|
||||||
try { pretty = JSON.stringify(JSON.parse(argsJson), null, 2); } catch {}
|
|
||||||
const el = document.createElement("div");
|
|
||||||
el.className = "hist-msg tool";
|
|
||||||
el.innerHTML = `
|
|
||||||
<div class="hist-role">Tool call</div>
|
|
||||||
<button class="hist-tool-header" aria-expanded="false">
|
|
||||||
<svg class="hist-tool-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">${WRENCH_PATH}</svg>
|
|
||||||
<span class="hist-tool-name">${escHtml(name)}</span>
|
|
||||||
<svg class="hist-tool-chevron" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="6 9 12 15 18 9"/></svg>
|
|
||||||
</button>
|
|
||||||
<div class="hist-tool-body">
|
|
||||||
<div class="hist-tool-label">Input</div>
|
|
||||||
<div class="hist-tool-block">${escHtml(pretty)}</div>
|
|
||||||
<div class="hist-tool-label">Output</div>
|
|
||||||
<div class="hist-tool-block hist-tool-output">${escHtml(output || "(no output)")}</div>
|
|
||||||
</div>
|
|
||||||
`;
|
|
||||||
const header = /** @type {HTMLButtonElement} */ (el.querySelector(".hist-tool-header"));
|
|
||||||
const body = /** @type {HTMLDivElement} */ (el.querySelector(".hist-tool-body"));
|
|
||||||
header.addEventListener("click", () => {
|
|
||||||
const expanded = header.getAttribute("aria-expanded") === "true";
|
|
||||||
header.setAttribute("aria-expanded", String(!expanded));
|
|
||||||
body.classList.toggle("open", !expanded);
|
|
||||||
});
|
|
||||||
this._chatHistory.appendChild(el);
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Tag an assistant history row as interrupted (user barged in mid-reply).
|
|
||||||
* @param {HTMLElement | null} hist */
|
|
||||||
_markHistInterrupted(hist) {
|
|
||||||
if (!hist || hist.querySelector(".hist-note")) return;
|
|
||||||
hist.classList.add("interrupted");
|
|
||||||
const note = document.createElement("div");
|
|
||||||
note.className = "hist-note";
|
|
||||||
note.textContent = "Interrupted";
|
|
||||||
hist.appendChild(note);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Render a captured webcam frame in the transcript (the camera tool result).
|
|
||||||
* @param {string} dataUrl */
|
|
||||||
_appendHistImage(dataUrl) {
|
|
||||||
const empty = this._chatHistory.querySelector(".chat-empty");
|
|
||||||
if (empty) empty.remove();
|
|
||||||
const el = document.createElement("div");
|
|
||||||
el.className = "hist-msg tool";
|
|
||||||
el.innerHTML = `<div class="hist-role">Snapshot</div><img class="hist-image" alt="Webcam snapshot sent to the model" />`;
|
|
||||||
const img = /** @type {HTMLImageElement} */ (el.querySelector("img"));
|
|
||||||
img.src = dataUrl;
|
|
||||||
this._chatHistory.appendChild(el);
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Reset all streaming bookkeeping for session start / teardown. Pass
|
|
||||||
* `dismiss` to also fade any bubbles still on screen.
|
|
||||||
* @param {{ dismiss?: boolean }} [opts]
|
|
||||||
*/
|
|
||||||
reset(opts) {
|
|
||||||
if (opts?.dismiss) {
|
|
||||||
if (this._activeUserBubble) this._dismissBubble(this._activeUserBubble);
|
|
||||||
for (const { bubble } of this._asstByResp.values()) this._dismissBubble(bubble);
|
|
||||||
}
|
|
||||||
this._userHistByItem.clear();
|
|
||||||
this._activeUserBubble = null;
|
|
||||||
this._activeUserItemId = "";
|
|
||||||
this._asstByResp.clear();
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Export / download conversation ───────────────────────────────────────
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Serialize the current persistent chat history (.hist-msg rows inside
|
|
||||||
* #chat-history) into a structured JSON object. Covers:
|
|
||||||
* - user / assistant text messages (role, text, timestamp, name, initials)
|
|
||||||
* - tool-call expandable rows (role=tool, name, args, output)
|
|
||||||
* - webcam / snapshot images (role=image, dataUrl)
|
|
||||||
* Empty state rows (.chat-empty) and ephemeral orb bubbles are ignored.
|
|
||||||
* @returns {{
|
|
||||||
* exportedAt: string,
|
|
||||||
* messageCount: number,
|
|
||||||
* messages: Array<{ type: string; timestamp?: string; [k: string]: unknown }>,
|
|
||||||
* }}
|
|
||||||
*/
|
|
||||||
_serializeConversation() {
|
|
||||||
/** @type {Array<{ type: string; timestamp?: string; [k: string]: unknown }>} */
|
|
||||||
const messages = [];
|
|
||||||
if (!this._chatHistory) return { exportedAt: new Date().toISOString(), messageCount: 0, messages };
|
|
||||||
const rows = /** @type {HTMLElement[]} */ ([...this._chatHistory.querySelectorAll(":scope > .hist-msg")]);
|
|
||||||
for (const row of rows) {
|
|
||||||
const ts = row.getAttribute("data-timestamp") || undefined;
|
|
||||||
if (row.classList.contains("user") || row.classList.contains("assistant")) {
|
|
||||||
const body = /** @type {HTMLElement | null} */ (row.querySelector(".hist-body"));
|
|
||||||
if (!body) continue;
|
|
||||||
const role = row.classList.contains("user") ? "user" : "assistant";
|
|
||||||
const out = /** @type {{ type: string; text: string; timestamp?: string; name?: string; initials?: string; interrupted?: boolean }} */ ({
|
|
||||||
type: role,
|
|
||||||
text: body.textContent || "",
|
|
||||||
});
|
|
||||||
if (ts) out.timestamp = ts;
|
|
||||||
if (role === "user") {
|
|
||||||
const n = row.getAttribute("data-user-name");
|
|
||||||
const i = row.getAttribute("data-user-initials");
|
|
||||||
if (n) out.name = n;
|
|
||||||
if (i) out.initials = i;
|
|
||||||
}
|
|
||||||
if (row.classList.contains("interrupted")) out.interrupted = true;
|
|
||||||
messages.push(out);
|
|
||||||
} else if (row.classList.contains("tool")) {
|
|
||||||
const img = /** @type {HTMLImageElement | null} */ (row.querySelector("img.hist-image"));
|
|
||||||
if (img) {
|
|
||||||
messages.push({ type: "image", timestamp: ts, dataUrl: img.src || "" });
|
|
||||||
} else {
|
|
||||||
const nameEl = /** @type {HTMLElement | null} */ (row.querySelector(".hist-tool-name"));
|
|
||||||
const blocks = /** @type {HTMLElement[]} */ ([...row.querySelectorAll(".hist-tool-block")]);
|
|
||||||
const args = blocks[0]?.textContent || "";
|
|
||||||
const output = blocks[1]?.textContent || "";
|
|
||||||
messages.push({
|
|
||||||
type: "tool",
|
|
||||||
timestamp: ts,
|
|
||||||
name: nameEl?.textContent || "",
|
|
||||||
args,
|
|
||||||
output,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
exportedAt: new Date().toISOString(),
|
|
||||||
messageCount: messages.length,
|
|
||||||
messages,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Pretty-print the JSON conversation structure to a human-readable plain
|
|
||||||
* text log (one entry per "HH:MM Nombre: texto"). Tool calls are rendered
|
|
||||||
* as indented blocks ("TOOL: nombre" followed by Input/Output sections).
|
|
||||||
* @param {ReturnType<ChatView["_serializeConversation"]>} conv
|
|
||||||
* @returns {string}
|
|
||||||
*/
|
|
||||||
_conversationToText(conv) {
|
|
||||||
const lines = [];
|
|
||||||
const date = (() => { try { return new Date(conv.exportedAt).toLocaleString(); } catch { return conv.exportedAt; } })();
|
|
||||||
lines.push(`Conversación Vocero — ${date}`);
|
|
||||||
lines.push(`Mensajes totales: ${conv.messageCount}`);
|
|
||||||
lines.push("—".repeat(52));
|
|
||||||
lines.push("");
|
|
||||||
for (const m of conv.messages) {
|
|
||||||
if (m.type === "user" || m.type === "assistant") {
|
|
||||||
const label = m.type === "user" ? (m.name ? String(m.name) : "Usuario") : "Vocero";
|
|
||||||
const head = [m.timestamp, label].filter(Boolean).join(" ");
|
|
||||||
lines.push(`${head}${m.interrupted ? " [interrumpida]" : ""}:`);
|
|
||||||
lines.push(String(m.text || "").trim());
|
|
||||||
lines.push("");
|
|
||||||
} else if (m.type === "tool") {
|
|
||||||
const head = [m.timestamp, `HERRAMIENTA: ${String(m.name || "").trim()}`].filter(Boolean).join(" ");
|
|
||||||
lines.push(head);
|
|
||||||
const args = String(m.args || "").trim();
|
|
||||||
const output = String(m.output || "").trim();
|
|
||||||
if (args) { lines.push(" Input:"); lines.push(" " + args.split("\n").join("\n ")); }
|
|
||||||
if (output) { lines.push(" Output:"); lines.push(" " + output.split("\n").join("\n ")); }
|
|
||||||
lines.push("");
|
|
||||||
} else if (m.type === "image") {
|
|
||||||
const head = [m.timestamp, "SNAPSHOT (imagen web-cámara enviada al modelo)"].filter(Boolean).join(" ");
|
|
||||||
lines.push(head);
|
|
||||||
lines.push(" " + String(m.dataUrl || "").slice(0, 80) + (String(m.dataUrl || "").length > 80 ? "…" : ""));
|
|
||||||
lines.push("");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return lines.join("\n");
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Trigger a browser "download" for the given string content. Uses the
|
|
||||||
* classic Blob+URL+a.click pattern — no backend call, fully client-side,
|
|
||||||
* works offline and never leaks the conversation content anywhere.
|
|
||||||
* @param {string} content
|
|
||||||
* @param {string} filename
|
|
||||||
* @param {string} mime
|
|
||||||
*/
|
|
||||||
_downloadBlob(content, filename, mime) {
|
|
||||||
const blob = new Blob([content], { type: mime });
|
|
||||||
const url = URL.createObjectURL(blob);
|
|
||||||
try {
|
|
||||||
const a = document.createElement("a");
|
|
||||||
a.href = url;
|
|
||||||
a.download = filename;
|
|
||||||
a.rel = "noopener";
|
|
||||||
document.body.appendChild(a);
|
|
||||||
a.click();
|
|
||||||
setTimeout(() => a.remove(), 0);
|
|
||||||
} finally {
|
|
||||||
setTimeout(() => URL.revokeObjectURL(url), 4000);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Public entry-point wired to the #chat-export-btn in the chat header.
|
|
||||||
* Serializes the current conversation and produces TWO downloads:
|
|
||||||
* 1. vocero-conversacion-YYYYMMDD-HHMMSS.json (structured data)
|
|
||||||
* 2. vocero-conversacion-YYYYMMDD-HHMMSS.txt (log legibile para humanos)
|
|
||||||
* UI feedback: briefly swaps the download icon for a ✓ "hecho" tick then
|
|
||||||
* restores the original SVG so the user gets a visual ack that the export
|
|
||||||
* completed (idempotent / errors are logged to console — never throws).
|
|
||||||
*/
|
|
||||||
exportConversation() {
|
|
||||||
try {
|
|
||||||
const conv = this._serializeConversation();
|
|
||||||
const profile = /** @type {Record<string, unknown> | null} */ (/** @type {unknown} */ (window.voceroUserProfile)) || null;
|
|
||||||
const ex = typeof window._s2sExtractName === "function" ? window._s2sExtractName : null;
|
|
||||||
const userName = ex && profile ? String(ex(profile) || "").trim() : "";
|
|
||||||
const stampBase = (() => {
|
|
||||||
const d = new Date();
|
|
||||||
const pad = (n, w = 2) => String(n).padStart(w, "0");
|
|
||||||
return `${d.getFullYear()}${pad(d.getMonth() + 1)}${pad(d.getDate())}-${pad(d.getHours())}${pad(d.getMinutes())}${pad(d.getSeconds())}`;
|
|
||||||
})();
|
|
||||||
const prefix = _safeName(`vocero-conversacion${userName ? "-" + userName : ""}`, "-");
|
|
||||||
const jsonName = `${prefix}-${stampBase}.json`;
|
|
||||||
const txtName = `${prefix}-${stampBase}.txt`;
|
|
||||||
const jsonText = JSON.stringify(conv, null, 2);
|
|
||||||
const plainText = this._conversationToText(conv);
|
|
||||||
if (conv.messageCount === 0) {
|
|
||||||
// Don't offer downloads of empty logs; flash the button briefly to
|
|
||||||
// signal "pressed" without spamming the user with empty files.
|
|
||||||
if (this._exportBtn) {
|
|
||||||
this._exportBtn.classList.add("chat-export-empty");
|
|
||||||
setTimeout(() => this._exportBtn?.classList.remove("chat-export-empty"), 600);
|
|
||||||
}
|
|
||||||
if (typeof window !== "undefined" && typeof console !== "undefined") {
|
|
||||||
console.info("[ui/chat.js:exportConversation] No hay mensajes aún — exportación omitida.");
|
|
||||||
}
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
this._downloadBlob(jsonText, jsonName, "application/json;charset=utf-8");
|
|
||||||
this._downloadBlob(plainText, txtName, "text/plain;charset=utf-8");
|
|
||||||
// Quick visual OK: paint the button green for 650ms. Avoids modifying
|
|
||||||
// innerHTML (we'd otherwise lose the SVG on mobile fast-clicks).
|
|
||||||
if (this._exportBtn) {
|
|
||||||
this._exportBtn.classList.add("chat-export-ok");
|
|
||||||
setTimeout(() => this._exportBtn?.classList.remove("chat-export-ok"), 650);
|
|
||||||
}
|
|
||||||
} catch (err) {
|
|
||||||
if (typeof window !== "undefined" && typeof console !== "undefined") {
|
|
||||||
console.error("[ui/chat.js:exportConversation] Error exportando conversación:", err);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Client event handlers ─────────────────────────────────────────────────
|
|
||||||
|
|
||||||
/**
|
|
||||||
* A streamed transcript delta (user or assistant).
|
|
||||||
* @param {{ role: "user" | "assistant"; text: string; partial: boolean; itemId?: string; responseId?: string }} d
|
|
||||||
*/
|
|
||||||
onTranscript(d) {
|
|
||||||
if (DEBUG) console.debug(`[ui] transcript role=${d.role} partial=${d.partial} item=${d.itemId} resp=${d.responseId} text=${JSON.stringify(d.text)}`);
|
|
||||||
|
|
||||||
if (d.role === "user") {
|
|
||||||
// Group by item_id: a speculative continuation reuses the same id, so it
|
|
||||||
// updates the same row/bubble. A missing id falls back to the active item
|
|
||||||
// (same utterance) or a fresh unique key, never a shared sentinel that
|
|
||||||
// would collapse distinct turns into one row.
|
|
||||||
const id = d.itemId || this._activeUserItemId || `_u${++this._anonSeq}`;
|
|
||||||
const text = d.text;
|
|
||||||
|
|
||||||
let hist = this._userHistByItem.get(id);
|
|
||||||
if (!hist) {
|
|
||||||
hist = this._appendHistMsg("user", text, d.partial);
|
|
||||||
this._userHistByItem.set(id, hist);
|
|
||||||
} else {
|
|
||||||
this._updateHistMsg(hist, text, d.partial);
|
|
||||||
}
|
|
||||||
|
|
||||||
// One ephemeral bubble per active item. Purely timer-based: the timer is
|
|
||||||
// refreshed on every delta, so it stays while the user keeps talking and
|
|
||||||
// fades a few seconds after they stop — no dependency on a response ever
|
|
||||||
// arriving, so it can never get stuck.
|
|
||||||
if (this._activeUserItemId !== id || !this._activeUserBubble) {
|
|
||||||
this._activeUserBubble = this._spawnBubble("user", text);
|
|
||||||
this._activeUserItemId = id;
|
|
||||||
} else {
|
|
||||||
this._updateBubbleText(this._activeUserBubble, text);
|
|
||||||
}
|
|
||||||
this._bumpDismiss(this._activeUserBubble, 6000);
|
|
||||||
this._markUnread();
|
|
||||||
} else if (d.role === "assistant") {
|
|
||||||
// Assistant transcript arrives once, as the full text, keyed by
|
|
||||||
// response_id so a cancelled speculative response can be removed later. A
|
|
||||||
// missing id gets a unique key so two id-less replies never collide.
|
|
||||||
const rid = d.responseId || `_a${++this._anonSeq}`;
|
|
||||||
const entry = this._asstByResp.get(rid);
|
|
||||||
if (!entry) {
|
|
||||||
const bubble = this._spawnBubble("assistant", d.text);
|
|
||||||
this._asstByResp.set(rid, { bubble, hist: this._appendHistMsg("assistant", d.text, false) });
|
|
||||||
this._bumpDismiss(bubble);
|
|
||||||
} else {
|
|
||||||
this._updateBubbleText(entry.bubble, d.text);
|
|
||||||
this._updateHistMsg(entry.hist, d.text, false);
|
|
||||||
this._bumpDismiss(entry.bubble);
|
|
||||||
}
|
|
||||||
this._markUnread();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* A response closed (completed or cancelled).
|
|
||||||
* @param {{ responseId: string; status: string; audible?: boolean; transcript?: string }} detail
|
|
||||||
*/
|
|
||||||
onResponseFinished(detail) {
|
|
||||||
const { responseId, status, audible, transcript } = detail;
|
|
||||||
if (DEBUG) console.debug(`[ui] response-finished resp=${responseId} status=${status} audible=${audible} known=${this._asstByResp.has(responseId)}`);
|
|
||||||
// Without an id we can't target a specific response; the bubble will
|
|
||||||
// auto-dismiss on its own timer regardless.
|
|
||||||
if (!responseId) return;
|
|
||||||
const entry = this._asstByResp.get(responseId);
|
|
||||||
|
|
||||||
if (status === "cancelled") {
|
|
||||||
// Keep every transcript that was received — mark it interrupted rather
|
|
||||||
// than erasing it. If the `*.transcript.done` never fired, build the row
|
|
||||||
// from the text carried in response.done.
|
|
||||||
let hist = entry?.hist ?? null;
|
|
||||||
if (!hist && transcript) {
|
|
||||||
hist = this._appendHistMsg("assistant", transcript, false);
|
|
||||||
} else if (hist && transcript) {
|
|
||||||
this._updateHistMsg(hist, transcript, false);
|
|
||||||
}
|
|
||||||
if (hist) this._markHistInterrupted(hist);
|
|
||||||
this._asstByResp.delete(responseId);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Any other terminal close (completed / failed / incomplete / …): just
|
|
||||||
// release the map entry. The bubble already auto-dismisses on its timer and
|
|
||||||
// the history row persists as the conversation log. Crucially we do NOT
|
|
||||||
// touch user state here — that lifecycle is fully independent.
|
|
||||||
this._asstByResp.delete(responseId);
|
|
||||||
// ALWAYS re-anchor to the last line when a response ends, since the final
|
|
||||||
// punctuation line + any trailing tool-result row pushes the tail further
|
|
||||||
// down than the last streamed delta did.
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The model called a tool — show an ephemeral "running" bubble.
|
|
||||||
* @param {string} name */
|
|
||||||
onToolCall(name) {
|
|
||||||
this._bumpDismiss(this._spawnBubble("tool", name));
|
|
||||||
this._markUnread();
|
|
||||||
this._scrollToBottom();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The tool finished — append its call+result row (and any captured image).
|
|
||||||
* @param {string} name @param {string} argsJson @param {string} output @param {string} [image] */
|
|
||||||
onToolResult(name, argsJson, output, image) {
|
|
||||||
const startedAt = performance.now();
|
|
||||||
this._appendHistTool(name, argsJson, output);
|
|
||||||
if (image) this._appendHistImage(image);
|
|
||||||
// Tool rows can be tall (big JSON payloads, tables). Force a scroll after
|
|
||||||
// insert so the start of the result isn't hidden off-screen above the
|
|
||||||
// sticky footer; we want the LAST line of the answer always in view.
|
|
||||||
this._scrollToBottom();
|
|
||||||
this._markUnread();
|
|
||||||
// #region debug-point C:chat-tool-render
|
|
||||||
reportChatDebug("C", "ui/chat.js:onToolResult", "rendered tool result row", {
|
|
||||||
name,
|
|
||||||
argsLength: argsJson.length,
|
|
||||||
outputLength: output.length,
|
|
||||||
renderMs: Math.round(performance.now() - startedAt),
|
|
||||||
});
|
|
||||||
// #endregion
|
|
||||||
if (image) this._appendHistImage(image); // show the captured frame below the call
|
|
||||||
this._markUnread();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/** Small shared helpers used across the UI modules: a strict query selector,
|
|
||||||
* HTML escaping for text we drop into innerHTML, error-string trimming, and
|
|
||||||
* the opt-in debug flag. */
|
|
||||||
|
|
||||||
/** Opt-in tracing: `localStorage.setItem("s2s.debug", "1")` then reload. */
|
|
||||||
export const DEBUG = (() => {
|
|
||||||
try {
|
|
||||||
return localStorage.getItem("s2s.debug") === "1";
|
|
||||||
} catch {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
})();
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Query a single element, throwing if it's missing (so a broken selector fails
|
|
||||||
* loudly at startup rather than as a later null-deref).
|
|
||||||
* @template {HTMLElement} T
|
|
||||||
* @param {string} selector
|
|
||||||
* @returns {T}
|
|
||||||
*/
|
|
||||||
export function $(selector) {
|
|
||||||
const el = document.querySelector(selector);
|
|
||||||
if (!el) throw new Error(`Missing element: ${selector}`);
|
|
||||||
return /** @type {T} */ (el);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {string} s @returns {string} */
|
|
||||||
export function escHtml(s) {
|
|
||||||
return s.replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">");
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Trim a long error message to fit the orb caption. @param {string} text */
|
|
||||||
export function truncateError(text) {
|
|
||||||
if (text.length <= 90) return text;
|
|
||||||
return text.slice(0, 87) + "…";
|
|
||||||
}
|
|
||||||
@@ -1,186 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* AudioWorkletProcessor that plays back Float32 mono samples received from
|
|
||||||
* the main thread, upsampling whatever incoming rate the server uses
|
|
||||||
* (typically 24 kHz PCM16) to the AudioContext rate (typically 48 kHz).
|
|
||||||
*
|
|
||||||
* Lifecycle / messaging:
|
|
||||||
*
|
|
||||||
* main -> worklet:
|
|
||||||
* { kind: "config", inputRate: 24000 } one-shot at startup
|
|
||||||
* { kind: "audio", samples: Float32Array } (transferable) per chunk
|
|
||||||
* { kind: "clear" } wipe queue (barge-in)
|
|
||||||
*
|
|
||||||
* worklet -> main:
|
|
||||||
* { kind: "stats", queuedMs, played } every ~250 ms
|
|
||||||
* { kind: "underrun" } every time the queue
|
|
||||||
* runs dry mid-playback
|
|
||||||
*
|
|
||||||
* Underrun strategy: output silence. We do NOT hold the last sample (that
|
|
||||||
* tends to produce audible clicks/buzzes when long gaps appear between
|
|
||||||
* TTS chunks). A short ramp-out + ramp-in at boundaries would be nicer but
|
|
||||||
* the server's 30 ms cadence makes underruns visible only at end of turn.
|
|
||||||
*/
|
|
||||||
|
|
||||||
const STATS_INTERVAL_FRAMES = 12000;
|
|
||||||
const FADE_FRAMES = 32;
|
|
||||||
const START_BUFFER_MS = 90;
|
|
||||||
|
|
||||||
class AudioPlaybackProcessor extends AudioWorkletProcessor {
|
|
||||||
constructor() {
|
|
||||||
super();
|
|
||||||
this._inputRate = 24000;
|
|
||||||
this._stepRatio = this._inputRate / sampleRate;
|
|
||||||
this._queue = [];
|
|
||||||
this._readIdx = 0;
|
|
||||||
this._fracPos = 0;
|
|
||||||
this._playing = false;
|
|
||||||
this._framesSinceStats = 0;
|
|
||||||
this._totalPlayed = 0;
|
|
||||||
this._fadeIn = 0;
|
|
||||||
this._fadeOut = 0;
|
|
||||||
this._lastSample = 0;
|
|
||||||
this._startBuffered = false;
|
|
||||||
|
|
||||||
this.port.onmessage = (e) => {
|
|
||||||
const data = e.data;
|
|
||||||
if (!data || typeof data !== "object") return;
|
|
||||||
switch (data.kind) {
|
|
||||||
case "config":
|
|
||||||
if (typeof data.inputRate === "number" && data.inputRate > 0) {
|
|
||||||
this._inputRate = data.inputRate;
|
|
||||||
this._stepRatio = this._inputRate / sampleRate;
|
|
||||||
}
|
|
||||||
break;
|
|
||||||
case "audio":
|
|
||||||
if (data.samples instanceof Float32Array && data.samples.length > 0) {
|
|
||||||
this._queue.push(data.samples);
|
|
||||||
if (!this._playing && this._queuedMs() >= START_BUFFER_MS) {
|
|
||||||
this._playing = true;
|
|
||||||
this._startBuffered = true;
|
|
||||||
this._fadeIn = FADE_FRAMES;
|
|
||||||
this._fadeOut = 0;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
break;
|
|
||||||
case "clear":
|
|
||||||
this._queue.length = 0;
|
|
||||||
this._readIdx = 0;
|
|
||||||
this._fracPos = 0;
|
|
||||||
this._fadeOut = FADE_FRAMES;
|
|
||||||
this._startBuffered = false;
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
_queuedSamples() {
|
|
||||||
let total = -this._readIdx;
|
|
||||||
for (const buf of this._queue) total += buf.length;
|
|
||||||
return Math.max(0, total);
|
|
||||||
}
|
|
||||||
|
|
||||||
_queuedMs() {
|
|
||||||
return (this._queuedSamples() / this._inputRate) * 1000;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Linear-interp read at the current fractional position. */
|
|
||||||
_readInterpolated() {
|
|
||||||
if (this._queue.length === 0) return null;
|
|
||||||
const head = this._queue[0];
|
|
||||||
const idx = this._readIdx;
|
|
||||||
const frac = this._fracPos;
|
|
||||||
|
|
||||||
let a = head[idx];
|
|
||||||
let b;
|
|
||||||
if (idx + 1 < head.length) {
|
|
||||||
b = head[idx + 1];
|
|
||||||
} else if (this._queue.length > 1) {
|
|
||||||
b = this._queue[1][0];
|
|
||||||
} else {
|
|
||||||
b = a;
|
|
||||||
}
|
|
||||||
return a + (b - a) * frac;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Advance the read position by `stepRatio`; pop consumed buffers. */
|
|
||||||
_advance() {
|
|
||||||
this._fracPos += this._stepRatio;
|
|
||||||
while (this._fracPos >= 1) {
|
|
||||||
this._fracPos -= 1;
|
|
||||||
this._readIdx += 1;
|
|
||||||
}
|
|
||||||
while (this._queue.length > 0 && this._readIdx >= this._queue[0].length) {
|
|
||||||
this._readIdx -= this._queue[0].length;
|
|
||||||
this._queue.shift();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
process(_, outputs) {
|
|
||||||
const channels = outputs[0];
|
|
||||||
if (!channels || channels.length === 0) return true;
|
|
||||||
const out = channels[0];
|
|
||||||
const stereo = channels.length > 1 ? channels[1] : null;
|
|
||||||
|
|
||||||
for (let i = 0; i < out.length; i++) {
|
|
||||||
let sample = 0;
|
|
||||||
|
|
||||||
if (!this._playing && this._queue.length > 0 && this._queuedMs() >= START_BUFFER_MS) {
|
|
||||||
this._playing = true;
|
|
||||||
this._startBuffered = true;
|
|
||||||
this._fadeIn = FADE_FRAMES;
|
|
||||||
this._fadeOut = 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (this._playing) {
|
|
||||||
const v = this._readInterpolated();
|
|
||||||
if (v === null) {
|
|
||||||
// Underrun: try to ramp out cleanly to avoid clicks.
|
|
||||||
sample = this._lastSample * Math.max(0, 1 - 1 / FADE_FRAMES);
|
|
||||||
this._lastSample = sample;
|
|
||||||
if (Math.abs(sample) < 1e-4) {
|
|
||||||
this._playing = false;
|
|
||||||
this._startBuffered = false;
|
|
||||||
this._lastSample = 0;
|
|
||||||
this.port.postMessage({ kind: "underrun" });
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
sample = v;
|
|
||||||
this._lastSample = v;
|
|
||||||
this._advance();
|
|
||||||
}
|
|
||||||
|
|
||||||
if (this._fadeIn > 0) {
|
|
||||||
const gain = 1 - this._fadeIn / FADE_FRAMES;
|
|
||||||
sample *= gain;
|
|
||||||
this._fadeIn -= 1;
|
|
||||||
}
|
|
||||||
if (this._fadeOut > 0) {
|
|
||||||
const gain = this._fadeOut / FADE_FRAMES;
|
|
||||||
sample *= gain;
|
|
||||||
this._fadeOut -= 1;
|
|
||||||
if (this._fadeOut === 0) {
|
|
||||||
this._playing = false;
|
|
||||||
this._startBuffered = false;
|
|
||||||
this._lastSample = 0;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
this._totalPlayed += 1;
|
|
||||||
}
|
|
||||||
|
|
||||||
out[i] = sample;
|
|
||||||
if (stereo) stereo[i] = sample;
|
|
||||||
}
|
|
||||||
|
|
||||||
this._framesSinceStats += out.length;
|
|
||||||
if (this._framesSinceStats >= STATS_INTERVAL_FRAMES) {
|
|
||||||
this._framesSinceStats = 0;
|
|
||||||
this.port.postMessage({ kind: "stats", queuedMs: this._queuedMs(), played: this._totalPlayed });
|
|
||||||
}
|
|
||||||
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
registerProcessor("audio-playback", AudioPlaybackProcessor);
|
|
||||||
@@ -1,159 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* AudioWorkletProcessor that resamples the AudioContext rate (typically 48 kHz)
|
|
||||||
* down to 16 kHz, packs the result as little-endian Int16 PCM, and posts it
|
|
||||||
* back to the main thread in fixed-size chunks.
|
|
||||||
*
|
|
||||||
* The Hugging Face speech-to-speech WebSocket route expects the
|
|
||||||
* `input_audio_buffer.append` payload at 16 kHz PCM16 mono.
|
|
||||||
*
|
|
||||||
* Design notes:
|
|
||||||
* - 48 -> 16 is an exact 3:1 ratio so we use a 3-tap boxcar average as a
|
|
||||||
* cheap low-pass before decimating. Good enough for voice STT; we lose
|
|
||||||
* a tiny bit of >8 kHz content which the pipeline discards anyway.
|
|
||||||
* - Output frames are emitted at the cadence dictated by `chunkMs`
|
|
||||||
* (default 40 ms = 640 samples = 1280 bytes). The OpenAI Realtime
|
|
||||||
* server batches incoming audio so the cadence is flexible; 20-100 ms
|
|
||||||
* is the sweet spot.
|
|
||||||
* - Float -> Int16 saturates to [-1, 1] before scaling.
|
|
||||||
* - Optional noise gate: per-chunk RMS decides open/closed against a
|
|
||||||
* threshold; the gain ramps (fast attack, hold, slow release) so word
|
|
||||||
* onsets aren't clipped and quiet tails don't click. The gate only
|
|
||||||
* affects the audio we SEND; the main-thread visualiser taps the raw
|
|
||||||
* mic separately. We post the chunk RMS up every frame so the Settings
|
|
||||||
* mic meter can show the live level against the threshold.
|
|
||||||
*/
|
|
||||||
|
|
||||||
const TARGET_RATE = 16000;
|
|
||||||
const DEFAULT_CHUNK_MS = 40;
|
|
||||||
// Gate envelope timing (fixed; only the threshold is user-tunable).
|
|
||||||
const GATE_ATTACK_MS = 5; // open almost instantly so word onsets survive
|
|
||||||
const GATE_HOLD_MS = 250; // stay open this long after the level drops back under
|
|
||||||
const GATE_RELEASE_MS = 80; // then fade closed over this long (no click)
|
|
||||||
|
|
||||||
class MicCaptureProcessor extends AudioWorkletProcessor {
|
|
||||||
constructor(options) {
|
|
||||||
super();
|
|
||||||
const chunkMs = options?.processorOptions?.chunkMs ?? DEFAULT_CHUNK_MS;
|
|
||||||
this._inputRate = sampleRate;
|
|
||||||
this._ratio = this._inputRate / TARGET_RATE;
|
|
||||||
this._chunkSamples16k = Math.round((TARGET_RATE * chunkMs) / 1000);
|
|
||||||
this._scratch = new Float32Array(0);
|
|
||||||
this._decimated = new Float32Array(this._chunkSamples16k);
|
|
||||||
this._enabled = true;
|
|
||||||
|
|
||||||
// Noise gate state. Disabled by default (pure passthrough).
|
|
||||||
this._gateEnabled = false;
|
|
||||||
this._thresholdLin = 0; // linear amplitude; signal RMS must exceed this to open
|
|
||||||
this._gateGain = 1; // smoothed gain currently applied
|
|
||||||
this._holdRemaining = 0; // samples left before the gate may start closing
|
|
||||||
this._attackCoef = Math.exp(-1 / ((GATE_ATTACK_MS / 1000) * TARGET_RATE));
|
|
||||||
this._releaseCoef = Math.exp(-1 / ((GATE_RELEASE_MS / 1000) * TARGET_RATE));
|
|
||||||
this._holdSamples = Math.round((GATE_HOLD_MS / 1000) * TARGET_RATE);
|
|
||||||
|
|
||||||
this.port.onmessage = (e) => {
|
|
||||||
const data = e.data;
|
|
||||||
if (data?.kind === "enable") this._enabled = !!data.value;
|
|
||||||
else if (data?.kind === "gate") {
|
|
||||||
this._gateEnabled = !!data.enabled;
|
|
||||||
// dB -> linear amplitude. When off, threshold 0 keeps the gate open.
|
|
||||||
this._thresholdLin = data.enabled ? Math.pow(10, data.thresholdDb / 20) : 0;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Append `incoming` to the internal scratch buffer, then emit as many
|
|
||||||
* full output chunks as we have material for.
|
|
||||||
* @param {Float32Array} incoming
|
|
||||||
*/
|
|
||||||
_ingest(incoming) {
|
|
||||||
if (incoming.length === 0) return;
|
|
||||||
const next = new Float32Array(this._scratch.length + incoming.length);
|
|
||||||
next.set(this._scratch, 0);
|
|
||||||
next.set(incoming, this._scratch.length);
|
|
||||||
this._scratch = next;
|
|
||||||
this._maybeEmit();
|
|
||||||
}
|
|
||||||
|
|
||||||
_maybeEmit() {
|
|
||||||
const r = this._ratio;
|
|
||||||
const n = this._chunkSamples16k;
|
|
||||||
const needIn = Math.ceil(n * r);
|
|
||||||
const dec = this._decimated;
|
|
||||||
while (this._scratch.length >= needIn) {
|
|
||||||
// 1. Decimate to 16 kHz floats and accumulate energy for the gate/meter.
|
|
||||||
let sumSq = 0;
|
|
||||||
if (Math.abs(r - 3) < 1e-6) {
|
|
||||||
// 48 kHz -> 16 kHz fast path with boxcar lowpass.
|
|
||||||
for (let i = 0; i < n; i++) {
|
|
||||||
const idx = i * 3;
|
|
||||||
const s = (this._scratch[idx] + this._scratch[idx + 1] + this._scratch[idx + 2]) / 3;
|
|
||||||
dec[i] = s;
|
|
||||||
sumSq += s * s;
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
// Generic path: linear interpolation. Slower but works at any rate
|
|
||||||
// (e.g. some Windows boxes report sampleRate=44100).
|
|
||||||
for (let i = 0; i < n; i++) {
|
|
||||||
const srcPos = i * r;
|
|
||||||
const idx = Math.floor(srcPos);
|
|
||||||
const frac = srcPos - idx;
|
|
||||||
const a = this._scratch[idx];
|
|
||||||
const b = this._scratch[idx + 1] ?? a;
|
|
||||||
const s = a + (b - a) * frac;
|
|
||||||
dec[i] = s;
|
|
||||||
sumSq += s * s;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
const rms = Math.sqrt(sumSq / n);
|
|
||||||
|
|
||||||
// 2. Decide the gate target for this chunk, then ramp sample-by-sample.
|
|
||||||
let target = 1;
|
|
||||||
if (this._gateEnabled) {
|
|
||||||
if (rms >= this._thresholdLin) {
|
|
||||||
this._holdRemaining = this._holdSamples; // re-arm the hold
|
|
||||||
} else if (this._holdRemaining > 0) {
|
|
||||||
this._holdRemaining -= n; // coasting through the hold window
|
|
||||||
} else {
|
|
||||||
target = 0;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// 3. Apply the (smoothed) gain and pack to Int16.
|
|
||||||
const out = new Int16Array(n);
|
|
||||||
let gain = this._gateGain;
|
|
||||||
for (let i = 0; i < n; i++) {
|
|
||||||
const coef = target > gain ? this._attackCoef : this._releaseCoef;
|
|
||||||
gain = target + (gain - target) * coef;
|
|
||||||
const s = dec[i] * gain;
|
|
||||||
const clamped = s < -1 ? -1 : s > 1 ? 1 : s;
|
|
||||||
out[i] = clamped < 0 ? clamped * 0x8000 : clamped * 0x7fff;
|
|
||||||
}
|
|
||||||
this._gateGain = gain;
|
|
||||||
|
|
||||||
// Shift the scratch buffer to keep only the trailing unused samples.
|
|
||||||
const consumed = Math.floor(n * r);
|
|
||||||
this._scratch = this._scratch.slice(consumed);
|
|
||||||
|
|
||||||
// Live input level for the Settings meter (raw RMS, pre-gate).
|
|
||||||
this.port.postMessage({ kind: "level", rms });
|
|
||||||
|
|
||||||
if (this._enabled) {
|
|
||||||
this.port.postMessage(out.buffer, [out.buffer]);
|
|
||||||
}
|
|
||||||
// When disabled (mic muted) we silently consume input so the worklet
|
|
||||||
// stays alive and the buffer never grows unbounded.
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
process(inputs) {
|
|
||||||
const input = inputs[0];
|
|
||||||
if (!input || input.length === 0 || !input[0]) return true;
|
|
||||||
const mono = input[0];
|
|
||||||
if (mono.length > 0) this._ingest(mono);
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
registerProcessor("mic-capture", MicCaptureProcessor);
|
|
||||||
@@ -1,57 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* Pure, stateless helpers for the WebSocket realtime client: base64 <-> PCM
|
|
||||||
* conversion for the audio frames on the wire, transcript extraction from a
|
|
||||||
* `response.done` payload, and a tiny URL helper. Kept separate from the client
|
|
||||||
* so the protocol/state logic stays readable.
|
|
||||||
*/
|
|
||||||
|
|
||||||
/** @param {string} url */
|
|
||||||
export function trimTrailingSlash(url) {
|
|
||||||
return url.endsWith("/") ? url.slice(0, -1) : url;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Pull the assistant transcript out of a `response.done` payload. The text
|
|
||||||
* lives in `response.output[].content[].transcript` (audio) or `.text`. Used as
|
|
||||||
* the source of truth for interrupted replies, where the dedicated
|
|
||||||
* `*.transcript.done` event may never arrive.
|
|
||||||
* @param {any} response
|
|
||||||
* @returns {string}
|
|
||||||
*/
|
|
||||||
export function extractResponseTranscript(response) {
|
|
||||||
const output = response?.output;
|
|
||||||
if (!Array.isArray(output)) return "";
|
|
||||||
/** @type {string[]} */
|
|
||||||
const parts = [];
|
|
||||||
for (const item of output) {
|
|
||||||
for (const part of item?.content ?? []) {
|
|
||||||
const text = part?.transcript ?? part?.text;
|
|
||||||
if (typeof text === "string" && text.trim()) parts.push(text.trim());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return parts.join(" ").trim();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {ArrayBuffer} buf */
|
|
||||||
export function base64FromArrayBuffer(buf) {
|
|
||||||
const bytes = new Uint8Array(buf);
|
|
||||||
// Chunked encoding so we don't blow up the call stack on long buffers.
|
|
||||||
let binary = "";
|
|
||||||
const chunk = 0x8000;
|
|
||||||
for (let i = 0; i < bytes.length; i += chunk) {
|
|
||||||
binary += String.fromCharCode.apply(null, /** @type {number[]} */ (
|
|
||||||
/** @type {unknown} */ (bytes.subarray(i, i + chunk))
|
|
||||||
));
|
|
||||||
}
|
|
||||||
return btoa(binary);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {string} b64 */
|
|
||||||
export function base64ToBytes(b64) {
|
|
||||||
const binary = atob(b64);
|
|
||||||
const len = binary.length;
|
|
||||||
const out = new Uint8Array(len);
|
|
||||||
for (let i = 0; i < len; i++) out[i] = binary.charCodeAt(i);
|
|
||||||
return out;
|
|
||||||
}
|
|
||||||
@@ -1,98 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
/**
|
|
||||||
* Orb spectrum visualiser. Each animation frame it reads two AnalyserNodes (the
|
|
||||||
* mic input and the TTS output) and maps the low-frequency speech energy onto
|
|
||||||
* the orb's CSS custom properties:
|
|
||||||
* - `--bar0`..`--bar4` the 5-band level meter
|
|
||||||
* - `--ai-audio-level` the global "Reachy talks" glow / scale pulse
|
|
||||||
*
|
|
||||||
* The bottom of the FFT is where speech energy lives, so the band edges stay
|
|
||||||
* low — that keeps the bars dancing on voice rather than on noise. While the AI
|
|
||||||
* is speaking we source the bars from the OUTPUT analyser so the orb pulses with
|
|
||||||
* Reachy's voice instead of sitting dead while the user is silent.
|
|
||||||
*/
|
|
||||||
|
|
||||||
// Exported so the client can size its AnalyserNodes to match our buffer.
|
|
||||||
export const VIS_FFT_SIZE = 256;
|
|
||||||
const VIS_BAND_COUNT = 5;
|
|
||||||
const VIS_BAND_EDGES = [2, 5, 9, 16, 28, 52];
|
|
||||||
const VIS_ATTACK = 0.6; // weight for new sample on upswing (snappy)
|
|
||||||
const VIS_RELEASE = 0.18; // weight for new sample on decay (gentle fade)
|
|
||||||
|
|
||||||
export class OrbVisualiser {
|
|
||||||
/**
|
|
||||||
* @param {AnalyserNode} micAnalyser
|
|
||||||
* @param {AnalyserNode} outAnalyser
|
|
||||||
* @param {() => boolean} isAiSpeaking Source the bars from the AI output when
|
|
||||||
* true, otherwise from the mic.
|
|
||||||
*/
|
|
||||||
constructor(micAnalyser, outAnalyser, isAiSpeaking) {
|
|
||||||
this._mic = micAnalyser;
|
|
||||||
this._out = outAnalyser;
|
|
||||||
this._isAiSpeaking = isAiSpeaking;
|
|
||||||
this._buf = new Uint8Array(micAnalyser.frequencyBinCount);
|
|
||||||
this._bands = new Float32Array(VIS_BAND_COUNT);
|
|
||||||
this._aiLevel = 0;
|
|
||||||
/** @type {number | null} */
|
|
||||||
this._frame = null;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Begin the rAF loop (idempotent). */
|
|
||||||
start() {
|
|
||||||
if (this._frame !== null) return;
|
|
||||||
const root = document.documentElement;
|
|
||||||
const tick = () => {
|
|
||||||
this._frame = requestAnimationFrame(tick);
|
|
||||||
this._update(root);
|
|
||||||
};
|
|
||||||
this._frame = requestAnimationFrame(tick);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Stop the loop and clear the CSS vars so the orb returns to rest. */
|
|
||||||
stop() {
|
|
||||||
if (this._frame !== null) {
|
|
||||||
cancelAnimationFrame(this._frame);
|
|
||||||
this._frame = null;
|
|
||||||
}
|
|
||||||
const root = document.documentElement;
|
|
||||||
for (let i = 0; i < VIS_BAND_COUNT; i++) root.style.removeProperty(`--bar${i}`);
|
|
||||||
root.style.removeProperty("--ai-audio-level");
|
|
||||||
}
|
|
||||||
|
|
||||||
/** @param {HTMLElement} root */
|
|
||||||
_update(root) {
|
|
||||||
// Mic bars: split FFT into 5 log-ish bands, smooth, write CSS vars.
|
|
||||||
const source = this._isAiSpeaking() ? this._out : this._mic;
|
|
||||||
source.getByteFrequencyData(this._buf);
|
|
||||||
|
|
||||||
for (let b = 0; b < VIS_BAND_COUNT; b++) {
|
|
||||||
const lo = VIS_BAND_EDGES[b];
|
|
||||||
const hi = VIS_BAND_EDGES[b + 1];
|
|
||||||
let sum = 0;
|
|
||||||
let n = 0;
|
|
||||||
for (let i = lo; i < hi && i < this._buf.length; i++) {
|
|
||||||
sum += this._buf[i];
|
|
||||||
n += 1;
|
|
||||||
}
|
|
||||||
const target = n > 0 ? sum / (n * 255) : 0;
|
|
||||||
const prev = this._bands[b];
|
|
||||||
const k = target > prev ? VIS_ATTACK : VIS_RELEASE;
|
|
||||||
const next = prev + (target - prev) * k;
|
|
||||||
this._bands[b] = next;
|
|
||||||
root.style.setProperty(`--bar${b}`, next.toFixed(3));
|
|
||||||
}
|
|
||||||
|
|
||||||
// Global AI audio level: peak of the output analyser, used by the CSS to
|
|
||||||
// make the orb's glow / scale react to Reachy's voice.
|
|
||||||
this._out.getByteFrequencyData(this._buf);
|
|
||||||
let peak = 0;
|
|
||||||
const limit = Math.min(this._buf.length, VIS_BAND_EDGES[VIS_BAND_COUNT]);
|
|
||||||
for (let i = 0; i < limit; i++) {
|
|
||||||
if (this._buf[i] > peak) peak = this._buf[i];
|
|
||||||
}
|
|
||||||
const aiTarget = peak / 255;
|
|
||||||
const k = aiTarget > this._aiLevel ? VIS_ATTACK : VIS_RELEASE;
|
|
||||||
this._aiLevel = this._aiLevel + (aiTarget - this._aiLevel) * k;
|
|
||||||
root.style.setProperty("--ai-audio-level", this._aiLevel.toFixed(3));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,39 +0,0 @@
|
|||||||
# Configurazione ambiente per s2s-reasoning-engine
|
|
||||||
#
|
|
||||||
# Copia questo file in .env nella root del progetto o nella cartella s2s-reasoning-engine/
|
|
||||||
# e modifica i valori secondo il tuo deployment.
|
|
||||||
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
|
||||||
# 1) LLM (OpenAI-compatible)
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
|
||||||
OPENAI_API_KEY=sk-....................................
|
|
||||||
OPENAI_BASE_URL=https://api.openai.com/v1
|
|
||||||
OPENAI_REASONING_MODEL=gpt-4.1-mini
|
|
||||||
OPENAI_ORGANIZATION=
|
|
||||||
OPENAI_PROJECT=
|
|
||||||
OPENAI_REASONING_TEMPERATURE=0.2
|
|
||||||
OPENAI_REASONING_MAX_TOKENS=512
|
|
||||||
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
|
||||||
# 2) Aquachile (profilo agente, tool MCP, autenticazione)
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
|
||||||
S2S_AGENT_PROFILE=https://aquachile.azurianlab.com/api/v1/agents/profile
|
|
||||||
S2S_USER_PHONE_AUTH_URL=https://aquachile.azurianlab.com/api/v1/auth/user-by-phone
|
|
||||||
S2S_TOOL_EXECUTION_MCP_URL=https://aquachile.azurianlab.com/api/v1/mcp
|
|
||||||
S2S_TOOL_EXECUTION_MCP_API_KEY=....................................
|
|
||||||
S2S_TOOL_EXECUTION_MCP_API_SECRET=....................................
|
|
||||||
S2S_SEARCH_MCP=true
|
|
||||||
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
|
||||||
# 3) Node-RED runtime
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
|
||||||
NODE_RED_PORT=1881
|
|
||||||
NODE_RED_LOG_LEVEL=info
|
|
||||||
NODE_RED_CREDENTIAL_SECRET=change-me-in-production
|
|
||||||
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
|
||||||
# 4) Integrazione con s2s UI (server.py)
|
|
||||||
# ─────────────────────────────────────────────────────────────
|
|
||||||
# URL raggiungibile dal container Python della UI
|
|
||||||
# (se usi rete docker condivisa è http://s2s-reasoning:1880)
|
|
||||||
S2S_REASONING_NODE_RED_URL=http://localhost:1881
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
# s2s-reasoning-engine: Node-RED 5.0 custom image
|
|
||||||
|
|
||||||
# Usa Node-RED 5.x ufficiale (immagine Alpine per dimensioni ridotte)
|
|
||||||
FROM nodered/node-red:5.0.3
|
|
||||||
|
|
||||||
# Porta di default di Node-RED (esposta in reti containerizzate)
|
|
||||||
EXPOSE 1880
|
|
||||||
|
|
||||||
# 1) Installa nodi aggiuntivi (a livello utente node-red nella cartella dati)
|
|
||||||
# - node-red-contrib-openai: integrazione nativa Chat Completions / Embeddings
|
|
||||||
# - node-red-contrib-http-logger: log HTTP opzionale
|
|
||||||
# - node-red-contrib-cache: cache in-memory per dati Aquachile (opzionale)
|
|
||||||
# - node-red-node-rbe: report-by-exception per deduplicare eventi
|
|
||||||
RUN cd /data && npm install --no-audit --no-fund \
|
|
||||||
node-red-contrib-openai@0.11.0 \
|
|
||||||
node-red-contrib-cache@1.1.1 \
|
|
||||||
node-red-node-rbe@0.5.0
|
|
||||||
|
|
||||||
# 2) Copia file di configurazione e flows iniziali
|
|
||||||
COPY settings.js /data/settings.js
|
|
||||||
COPY flows.json /data/flows.json
|
|
||||||
|
|
||||||
# 3) Imposta Node-RED per caricare flows statici + disabilita progetto e auth
|
|
||||||
# (per ambiente di sviluppo; in produzione abilitare adminAuth in settings.js)
|
|
||||||
ENV NODE_RED_FLOWS_FILE=/data/flows.json
|
|
||||||
ENV NODE_RED_CREDENTIAL_SECRET=s2s-reasoning-dev-secret
|
|
||||||
ENV FLOWS=/data/flows.json
|
|
||||||
|
|
||||||
# Avvio Node-RED
|
|
||||||
CMD ["npm", "start", "--", "--userDir", "/data", "--settings", "/data/settings.js"]
|
|
||||||
@@ -1,254 +0,0 @@
|
|||||||
[
|
|
||||||
{
|
|
||||||
"id": "s2s-reasoning-tab",
|
|
||||||
"type": "tab",
|
|
||||||
"label": "S2S Reasoning",
|
|
||||||
"disabled": false,
|
|
||||||
"info": "# Flusso di reasoning S2S — FASE 1: LOG-ONLY\n\nModalità attuale: il flusso NON produce risposte, NON modifica il prompt.\nSi limita a:\n 1. Ricevere POST /api/reasoning dal server demo\n 2. Estrarre transcript, lingua, phoneNumber\n 3. Loggare la richiesta utente su console + sidebar debug\n 4. Risondere HTTP 200 con contratto standard e flag `log_only: true`\n\nContratto risposta (fase 1):\n { answer: \"\", log_only: true, intent: \"log_only\", usage: {prompt_tokens:0,completion_tokens:0,total_tokens:0}, latency_ms: <ms> }\n",
|
|
||||||
"wires": []
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "http-in-reasoning",
|
|
||||||
"type": "http in",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "POST /api/reasoning",
|
|
||||||
"url": "/api/reasoning",
|
|
||||||
"method": "post",
|
|
||||||
"upload": false,
|
|
||||||
"swaggerDoc": "",
|
|
||||||
"x": 160,
|
|
||||||
"y": 160,
|
|
||||||
"wires": [
|
|
||||||
[
|
|
||||||
"parse-request"
|
|
||||||
]
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "parse-request",
|
|
||||||
"type": "function",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "Parse richiesta (log-only)",
|
|
||||||
"func": "// FASE 1 — LOG-ONLY: nessuna answer generata.\n// Valida e normalizza il payload in arrivo da server.py.\n// msg.req.body = { transcript, instructions, language, user_context, phoneNumber? }\n\nconst body = msg.req?.body || {};\nconst startedAt = Date.now();\n\nconst transcript = String(body.transcript || \"\").trim();\nconst instructions = String(body.instructions || \"\").trim();\nconst language = String(body.language || \"\").trim();\nconst userContext = String(body.user_context || body.user_context || \"\").trim();\nconst phoneNumber = String(body.phoneNumber || body.phone_number || msg.req?.query?.phoneNumber || \"\").trim();\n\nif (!transcript) {\n msg.statusCode = 400;\n msg.payload = { error: \"missing_transcript\", detail: \"transcript field is required\" };\n return [null, msg]; // uscita 2 = errore\n}\n\nconst detectedLang = language || (transcript.match(/[áéíóúñ¿¡]/i) ? \"es-CL\" : \"en-US\");\n\nmsg.state = {\n startedAt,\n transcript,\n instructionsLen: instructions.length,\n language: detectedLang,\n userContextLen: userContext.length,\n phoneNumber: phoneNumber || null,\n intent: \"log_only\",\n};\n\nmsg.payload = {\n transcript,\n language: detectedLang,\n phone: phoneNumber || \"none\",\n};\n\nreturn [msg, null];",
|
|
||||||
"outputs": 2,
|
|
||||||
"timeout": 0,
|
|
||||||
"noerr": 0,
|
|
||||||
"initialize": "",
|
|
||||||
"finalize": "",
|
|
||||||
"libs": [],
|
|
||||||
"x": 400,
|
|
||||||
"y": 160,
|
|
||||||
"wires": [
|
|
||||||
[
|
|
||||||
"log-transcript-console",
|
|
||||||
"debug-parsed"
|
|
||||||
],
|
|
||||||
[
|
|
||||||
"http-response-error"
|
|
||||||
]
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "log-transcript-console",
|
|
||||||
"type": "function",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "[LOG] richiesta utente",
|
|
||||||
"func": "// FASE 1 — Stampa la richiesta utente sul log server (Node-RED console)\n// e sul pannello debug sidebar.\n// NON produce alcuna risposta: quella arriva dal backend S2S nativo.\n\nconst state = msg.state || {};\nconst transcript = state.transcript || \"\";\nconst lang = state.language || \"\";\nconst phone = state.phoneNumber || \"none\";\nconst ts = new Date(state.startedAt || Date.now()).toISOString();\n\nconst line = `[S2S NODE_RED LOG_ONLY] ts=${ts} lang=${lang} phone=${phone} transcript=\"${transcript.replace(/\"/g, '\\\\\"')}\"`;\n\nnode.warn(line);\n\nmsg.logLine = line;\nreturn msg;",
|
|
||||||
"outputs": 1,
|
|
||||||
"timeout": 0,
|
|
||||||
"noerr": 0,
|
|
||||||
"initialize": "",
|
|
||||||
"finalize": "",
|
|
||||||
"libs": [],
|
|
||||||
"x": 660,
|
|
||||||
"y": 120,
|
|
||||||
"wires": [
|
|
||||||
[
|
|
||||||
"build-response"
|
|
||||||
]
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "debug-parsed",
|
|
||||||
"type": "debug",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "[Debug] parsed state",
|
|
||||||
"active": true,
|
|
||||||
"tosidebar": true,
|
|
||||||
"console": true,
|
|
||||||
"tostatus": false,
|
|
||||||
"complete": "state",
|
|
||||||
"targetType": "msg",
|
|
||||||
"status": "",
|
|
||||||
"x": 650,
|
|
||||||
"y": 200,
|
|
||||||
"wires": []
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "build-response",
|
|
||||||
"type": "function",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "Build 200 (log_only)",
|
|
||||||
"func": "// FASE 1 — Costruisci la risposta HTTP 200.\n// Contratto: answer VUOTA + flag log_only=true.\n// Il client (main.js) vedrà log_only e NON interverrà:\n// - NON chiama cancelActiveResponse()\n// - NON usa appendAssistantText()\n// - NON usa speechSynthesis\n// => la risposta vocale arriva dal backend S2S nativo.\n\nconst state = msg.state || {};\nconst latencyMs = Math.max(0, Date.now() - (state.startedAt || Date.now()));\n\nmsg.statusCode = 200;\nmsg.headers = {\n \"Content-Type\": \"application/json; charset=utf-8\",\n \"Cache-Control\": \"no-store\",\n};\nmsg.payload = {\n answer: \"\",\n log_only: true,\n intent: \"log_only\",\n usage: {\n prompt_tokens: 0,\n completion_tokens: 0,\n total_tokens: 0,\n },\n latency_ms: latencyMs,\n mcp_tools_used: [],\n caller_profile_keys: [],\n debug: {\n phase: \"log_only_fase1\",\n language: state.language || \"\",\n phone: state.phoneNumber || null,\n transcript_len: (state.transcript || \"\").length,\n },\n};\n\nreturn msg;",
|
|
||||||
"outputs": 1,
|
|
||||||
"timeout": 0,
|
|
||||||
"noerr": 0,
|
|
||||||
"initialize": "",
|
|
||||||
"finalize": "",
|
|
||||||
"libs": [],
|
|
||||||
"x": 920,
|
|
||||||
"y": 160,
|
|
||||||
"wires": [
|
|
||||||
[
|
|
||||||
"http-response-ok",
|
|
||||||
"debug-response"
|
|
||||||
]
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "http-response-ok",
|
|
||||||
"type": "http response",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "HTTP 200 OK",
|
|
||||||
"statusCode": "",
|
|
||||||
"headers": {},
|
|
||||||
"x": 1160,
|
|
||||||
"y": 120,
|
|
||||||
"wires": []
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "debug-response",
|
|
||||||
"type": "debug",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "[Debug] response payload",
|
|
||||||
"active": true,
|
|
||||||
"tosidebar": true,
|
|
||||||
"console": false,
|
|
||||||
"tostatus": false,
|
|
||||||
"complete": "payload",
|
|
||||||
"targetType": "msg",
|
|
||||||
"status": "",
|
|
||||||
"x": 1180,
|
|
||||||
"y": 200,
|
|
||||||
"wires": []
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "http-response-error",
|
|
||||||
"type": "http response",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "HTTP Error",
|
|
||||||
"statusCode": "",
|
|
||||||
"headers": {},
|
|
||||||
"x": 400,
|
|
||||||
"y": 240,
|
|
||||||
"wires": []
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "http-in-log-response",
|
|
||||||
"type": "http in",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "POST /api/log/system-response",
|
|
||||||
"url": "/api/log/system-response",
|
|
||||||
"method": "post",
|
|
||||||
"upload": false,
|
|
||||||
"swaggerDoc": "",
|
|
||||||
"x": 160,
|
|
||||||
"y": 360,
|
|
||||||
"wires": [
|
|
||||||
[
|
|
||||||
"parse-log-response"
|
|
||||||
]
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "parse-log-response",
|
|
||||||
"type": "function",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "Parse system-response",
|
|
||||||
"func": "// Riceve dal server.py la risposta TTS emessa dal backend S2S nativo.\n// msg.req.body = { tts, response_id?, item_id?, client_ip?, role?, source_event?, user_transcript?, phone? }\n// FASE 1 LOG-ONLY: nessun side-effect oltre al log.\n\nconst body = msg.req?.body || {};\nconst startedAt = Date.now();\n\nconst tts = String(body.tts || body.text || body.response || \"\").trim();\nconst responseId = String(body.response_id || body.responseId || \"\").trim() || null;\nconst itemId = String(body.item_id || body.itemId || \"\").trim() || null;\nconst clientIp = String(body.client_ip || body.clientIp || body.ip || \"\").trim() || null;\nconst role = String(body.role || \"assistant\").trim() || \"assistant\";\nconst sourceEvent = String(body.source_event || body.sourceEvent || \"\").trim() || null;\nconst userTranscript = String(body.user_transcript || body.userTranscript || \"\").trim() || null;\nconst phone = String(body.phone || body.phoneNumber || \"\").trim() || null;\n\nif (!tts) {\n msg.statusCode = 400;\n msg.headers = { \"Content-Type\": \"application/json\" };\n msg.payload = { error: \"missing_tts\", detail: \"field 'tts' (or 'text'/'response') is required\" };\n return [null, msg];\n}\n\nmsg.state = {\n startedAt,\n kind: \"system_response_log\",\n role,\n tts,\n responseId,\n itemId,\n clientIp,\n sourceEvent,\n userTranscript,\n phone,\n};\n\nmsg.payload = { tts, role, source: sourceEvent, client: clientIp };\nreturn [msg, null];",
|
|
||||||
"outputs": 2,
|
|
||||||
"timeout": 0,
|
|
||||||
"noerr": 0,
|
|
||||||
"initialize": "",
|
|
||||||
"finalize": "",
|
|
||||||
"libs": [],
|
|
||||||
"x": 420,
|
|
||||||
"y": 360,
|
|
||||||
"wires": [
|
|
||||||
[
|
|
||||||
"log-system-response",
|
|
||||||
"debug-system-response"
|
|
||||||
],
|
|
||||||
[
|
|
||||||
"http-response-error"
|
|
||||||
]
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "log-system-response",
|
|
||||||
"type": "function",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "[LOG] risposta sistema",
|
|
||||||
"func": "// Stampa la risposta del sistema (TTS dal backend nativo) sul log Node-RED.\n// Formato riga unica ricercabile con grep.\nconst state = msg.state || {};\nconst tts = state.tts || \"\";\nconst role = state.role || \"assistant\";\nconst src = state.sourceEvent || \"\";\nconst user = state.userTranscript || \"\";\nconst phone = state.phone || \"none\";\nconst ts = new Date(state.startedAt || Date.now()).toISOString();\n\nconst userShort = user.length > 120 ? user.slice(0, 117) + \"...\" : user;\nconst ttsShort = tts.length > 240 ? tts.slice(0, 237) + \"...\" : tts;\nconst esc = (s) => String(s || \"\").replace(/\\\\/g, \"\\\\\\\\\").replace(/\"/g, '\\\\\"');\n\nconst line = `[S2S NODE_RED SYSTEM_RESPONSE] ts=${ts} role=${role} phone=${esc(phone)} user=\"${esc(userShort)}\" tts=\"${esc(ttsShort)}\"`;\n\nnode.warn(line);\nmsg.logLine = line;\nreturn msg;",
|
|
||||||
"outputs": 1,
|
|
||||||
"timeout": 0,
|
|
||||||
"noerr": 0,
|
|
||||||
"initialize": "",
|
|
||||||
"finalize": "",
|
|
||||||
"libs": [],
|
|
||||||
"x": 690,
|
|
||||||
"y": 320,
|
|
||||||
"wires": [
|
|
||||||
[
|
|
||||||
"build-log-ack"
|
|
||||||
]
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "debug-system-response",
|
|
||||||
"type": "debug",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "[Debug] system-response state",
|
|
||||||
"active": true,
|
|
||||||
"tosidebar": true,
|
|
||||||
"console": true,
|
|
||||||
"tostatus": false,
|
|
||||||
"complete": "state",
|
|
||||||
"targetType": "msg",
|
|
||||||
"status": "",
|
|
||||||
"x": 710,
|
|
||||||
"y": 400,
|
|
||||||
"wires": []
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "build-log-ack",
|
|
||||||
"type": "function",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "Build 200 (ack)",
|
|
||||||
"func": "// Risposta minimale: 200 OK + ricevuto=true.\n// Non ritorna dati.\nconst state = msg.state || {};\nconst latencyMs = Math.max(0, Date.now() - (state.startedAt || Date.now()));\nmsg.statusCode = 200;\nmsg.headers = {\n \"Content-Type\": \"application/json; charset=utf-8\",\n \"Cache-Control\": \"no-store\",\n};\nmsg.payload = {\n ricevuto: true,\n log_only: true,\n intent: \"system_response_log\",\n latency_ms: latencyMs,\n};\nreturn msg;",
|
|
||||||
"outputs": 1,
|
|
||||||
"timeout": 0,
|
|
||||||
"noerr": 0,
|
|
||||||
"initialize": "",
|
|
||||||
"finalize": "",
|
|
||||||
"libs": [],
|
|
||||||
"x": 950,
|
|
||||||
"y": 360,
|
|
||||||
"wires": [
|
|
||||||
[
|
|
||||||
"http-response-log-ack"
|
|
||||||
]
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "http-response-log-ack",
|
|
||||||
"type": "http response",
|
|
||||||
"z": "s2s-reasoning-tab",
|
|
||||||
"name": "HTTP 200 ACK",
|
|
||||||
"statusCode": "",
|
|
||||||
"headers": {},
|
|
||||||
"x": 1180,
|
|
||||||
"y": 360,
|
|
||||||
"wires": []
|
|
||||||
}
|
|
||||||
]
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "s2s-reasoning-engine",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"description": "Node-RED 5.0 reasoning engine per S2S/Aquachile",
|
|
||||||
"main": "settings.js",
|
|
||||||
"scripts": {
|
|
||||||
"test": "echo \"Error: no test specified\" && exit 1"
|
|
||||||
},
|
|
||||||
"keywords": [],
|
|
||||||
"author": "",
|
|
||||||
"license": "ISC",
|
|
||||||
"type": "commonjs",
|
|
||||||
"private": "true",
|
|
||||||
"engines": {
|
|
||||||
"node": ">=18"
|
|
||||||
},
|
|
||||||
"dependencies": {
|
|
||||||
"dotenv": "^16.4.7",
|
|
||||||
"node-red": "^5.0.3",
|
|
||||||
"node-red-contrib-cache": "^2.0.3",
|
|
||||||
"node-red-contrib-openai": "^1.0.3",
|
|
||||||
"node-red-node-rbe": "^0.5.0"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,121 +0,0 @@
|
|||||||
/**
|
|
||||||
* Node-RED 5.0 settings for s2s-reasoning-engine — LOCAL INSTALL (non Docker).
|
|
||||||
* File: <project_root>/s2s-reasoning-engine/.node-red/settings.js
|
|
||||||
* oppure <project_root>/s2s-reasoning-engine/settings.js (master)
|
|
||||||
*
|
|
||||||
* Copiato automaticamente in .node-red/ da start.sh se mancante o più vecchio.
|
|
||||||
*/
|
|
||||||
|
|
||||||
const path = require("path");
|
|
||||||
const fs = require("fs");
|
|
||||||
|
|
||||||
// userDir è sempre <s2s-reasoning-engine>/.node-red (lo script start.sh)
|
|
||||||
const ENGINE_ROOT = path.resolve(__dirname, "..");
|
|
||||||
const LOCAL_NODE_MODULES = path.join(ENGINE_ROOT, "node_modules");
|
|
||||||
const USERDIR_NODE_MODULES = path.join(__dirname, "node_modules");
|
|
||||||
|
|
||||||
// ── Fix per editor "tslib.es6.js 404 / net::ERR_ABORTED" ─────────────────
|
|
||||||
// Installazioni npm locali flat (default npm 3+) spostano tslib in
|
|
||||||
// node_modules/tslib/ (root) invece che dentro
|
|
||||||
// @node-red/editor-client/node_modules/tslib/. L'editor di Node-RED 5
|
|
||||||
// carica `/tslib.es6.js` dalla root HTTP: se la cartella nested non esiste,
|
|
||||||
// torna 404. Qui aggiorniamo editor-client/public/ con i file tslib mancanti
|
|
||||||
// a ogni avvio (sopravvive a npm reinstall se tslib è in root).
|
|
||||||
(function _installTslibVendor() {
|
|
||||||
try {
|
|
||||||
const tslibRoot = path.join(LOCAL_NODE_MODULES, "tslib");
|
|
||||||
if (!fs.existsSync(path.join(tslibRoot, "tslib.es6.js"))) return;
|
|
||||||
const pub = path.join(LOCAL_NODE_MODULES, "@node-red", "editor-client", "public");
|
|
||||||
if (!fs.existsSync(pub)) return;
|
|
||||||
for (const f of ["tslib.es6.js", "tslib.js", "tslib.mjs"]) {
|
|
||||||
const src = path.join(tslibRoot, f);
|
|
||||||
const dst = path.join(pub, f);
|
|
||||||
if (!fs.existsSync(src)) continue;
|
|
||||||
try {
|
|
||||||
const s = fs.statSync(src);
|
|
||||||
const d = fs.existsSync(dst) ? fs.statSync(dst) : null;
|
|
||||||
if (!d || d.size !== s.size || d.mtimeMs < s.mtimeMs) {
|
|
||||||
fs.copyFileSync(src, dst);
|
|
||||||
}
|
|
||||||
} catch (e) { /* ignore */ }
|
|
||||||
}
|
|
||||||
} catch (e) { /* no-op */ }
|
|
||||||
})();
|
|
||||||
|
|
||||||
module.exports = {
|
|
||||||
uiPort: process.env.PORT || process.env.NODE_RED_PORT || 1881,
|
|
||||||
uiHost: "0.0.0.0",
|
|
||||||
|
|
||||||
httpRequestTimeout: 120000,
|
|
||||||
httpRequestLegacyMsgOverrides: true,
|
|
||||||
|
|
||||||
// ── Native HTTP helper per function node (SENZA externalModules) ─────
|
|
||||||
// NOTA: non usare functionExternalModules: true — in Node-RED v5 quel
|
|
||||||
// flag può bloccare il caricamento palette (es. "37/41 nodes loading"
|
|
||||||
// durante la verifica di tutti i pkg npm).
|
|
||||||
// Al suo posto usiamo functionGlobalContext: i moduli core Node.js sono
|
|
||||||
// require'd QUI nel settings.js (prima dell'avvio NR) e poi i function
|
|
||||||
// node li recuperano via global.get("_core_https") ecc.
|
|
||||||
functionModuleRequireAllowList: ["https","http","url","path","util","fs","os","buffer","zlib","stream","crypto"],
|
|
||||||
functionGlobalContext: {
|
|
||||||
_core_https: require("https"),
|
|
||||||
_core_http: require("http"),
|
|
||||||
_core_url: require("url"),
|
|
||||||
},
|
|
||||||
|
|
||||||
// adminAuth: {...} ← scommentare e valorizzare in produzione
|
|
||||||
|
|
||||||
// HTTPS (opzionale in produzione)
|
|
||||||
// https: {
|
|
||||||
// key: require("fs").readFileSync(path.join(__dirname, "privkey.pem")),
|
|
||||||
// cert: require("fs").readFileSync(path.join(__dirname, "fullchain.pem"))
|
|
||||||
// },
|
|
||||||
|
|
||||||
flowFile: process.env.FLOWS || path.join(__dirname, "flows.json"),
|
|
||||||
flowFilePretty: true,
|
|
||||||
credentialSecret: process.env.NODE_RED_CREDENTIAL_SECRET || "s2s-reasoning-local-dev",
|
|
||||||
|
|
||||||
httpNodeCors: {
|
|
||||||
origin: "*",
|
|
||||||
methods: "GET,PUT,POST,DELETE,OPTIONS",
|
|
||||||
credentials: true,
|
|
||||||
},
|
|
||||||
csrf: false,
|
|
||||||
|
|
||||||
// ── IMPORTANTE: caricamento nodi extra ─────────────────────────────
|
|
||||||
// Node-RED 5 carica i plugin NODI da:
|
|
||||||
// 1) {userDir}/node_modules (nostro .node-red/node_modules)
|
|
||||||
// 2) {nodesDir} (installazione root del progetto)
|
|
||||||
// 3) npm global
|
|
||||||
// Senza nodesDir i pacchetti installati nella ROOT del progetto non
|
|
||||||
// vengono trovati.
|
|
||||||
nodesDir: LOCAL_NODE_MODULES,
|
|
||||||
|
|
||||||
externalModules: {
|
|
||||||
autoInstall: true, // auto-installa nodi mancanti al boot
|
|
||||||
autoInstallRetry: 3,
|
|
||||||
palette: {
|
|
||||||
allowInstall: true, // permette Install dall'editor
|
|
||||||
allowUpload: true,
|
|
||||||
modules: ["node-red-contrib-openai", "node-red-contrib-cache", "node-red-node-rbe"],
|
|
||||||
allowList: ["node-red-contrib-*", "node-red-node-*"],
|
|
||||||
denyList: []
|
|
||||||
}
|
|
||||||
},
|
|
||||||
|
|
||||||
editorTheme: { projects: { enabled: false } },
|
|
||||||
|
|
||||||
logging: {
|
|
||||||
console: {
|
|
||||||
level: process.env.NODE_RED_LOG_LEVEL || process.env.LOG_LEVEL || "info",
|
|
||||||
metrics: false,
|
|
||||||
audit: false,
|
|
||||||
timestamp: true,
|
|
||||||
}
|
|
||||||
},
|
|
||||||
|
|
||||||
contextStorage: {
|
|
||||||
default: { module: "memory" },
|
|
||||||
file: { module: "localfilesystem" }
|
|
||||||
},
|
|
||||||
};
|
|
||||||
@@ -1,84 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# =============================================================
|
|
||||||
# start.sh — Avvia Node-RED 5.0 in locale per s2s-reasoning-engine
|
|
||||||
#
|
|
||||||
# Uso:
|
|
||||||
# ./start.sh # porta default 1881
|
|
||||||
# PORT=1900 ./start.sh # porta custom
|
|
||||||
# LOG_LEVEL=debug ./start.sh # log più dettagliato
|
|
||||||
# DEBUG=nodes:* ./start.sh # debug Node-RED interno
|
|
||||||
# Stop: Ctrl+C
|
|
||||||
# =============================================================
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
PARENT_DIR="$(dirname "$ROOT_DIR")"
|
|
||||||
|
|
||||||
cd "$ROOT_DIR"
|
|
||||||
|
|
||||||
# ----------------------------------------------------------
|
|
||||||
# 1) Carica env: prima locale .env (se esiste) poi padre ../.env
|
|
||||||
# Il file locale sovrascrive il padre se c'è lo stesso var.
|
|
||||||
# ----------------------------------------------------------
|
|
||||||
set -a
|
|
||||||
if [ -f "$ROOT_DIR/.env" ]; then
|
|
||||||
echo "[env] loaded $ROOT_DIR/.env"
|
|
||||||
# shellcheck disable=SC1091
|
|
||||||
source "$ROOT_DIR/.env"
|
|
||||||
fi
|
|
||||||
if [ -f "$PARENT_DIR/.env" ]; then
|
|
||||||
echo "[env] loaded $PARENT_DIR/.env"
|
|
||||||
# shellcheck disable=SC1091
|
|
||||||
source "$PARENT_DIR/.env"
|
|
||||||
fi
|
|
||||||
set +a
|
|
||||||
|
|
||||||
# ----------------------------------------------------------
|
|
||||||
# 2) Parametri runtime Node-RED
|
|
||||||
# ----------------------------------------------------------
|
|
||||||
PORT="${PORT:-${NODE_RED_PORT:-1881}}"
|
|
||||||
export PORT
|
|
||||||
export NODE_RED_PORT="$PORT"
|
|
||||||
export NODE_RED_LOG_LEVEL="${LOG_LEVEL:-${NODE_RED_LOG_LEVEL:-info}}"
|
|
||||||
export NODE_RED_CREDENTIAL_SECRET="${NODE_RED_CREDENTIAL_SECRET:-s2s-reasoning-local-dev}"
|
|
||||||
export FLOWS="${FLOWS:-$ROOT_DIR/.node-red/flows.json}"
|
|
||||||
|
|
||||||
# UserDir locale (tutto dentro la cartella corrente, NO ~/.node-red)
|
|
||||||
USERDIR="${USERDIR:-$ROOT_DIR/.node-red}"
|
|
||||||
SETTINGS="${SETTINGS:-$USERDIR/settings.js}"
|
|
||||||
|
|
||||||
# Assicura che la userDir esista e i file base siano presenti.
|
|
||||||
mkdir -p "$USERDIR"
|
|
||||||
if [ ! -f "$SETTINGS" ] || [ "$ROOT_DIR/settings.js" -nt "$SETTINGS" ]; then
|
|
||||||
cp -f "$ROOT_DIR/settings.js" "$SETTINGS"
|
|
||||||
echo "[init] copied settings.js -> $SETTINGS"
|
|
||||||
fi
|
|
||||||
if [ ! -f "$FLOWS" ] || [ "$ROOT_DIR/flows.json" -nt "$FLOWS" ]; then
|
|
||||||
cp -f "$ROOT_DIR/flows.json" "$FLOWS"
|
|
||||||
echo "[init] copied flows.json -> $FLOWS"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Verifica node_modules installati
|
|
||||||
if [ ! -d "$ROOT_DIR/node_modules/node-red" ]; then
|
|
||||||
echo "[init] node_modules non trovati. Avvio npm install..."
|
|
||||||
npm install --no-audit --no-fund
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo ""
|
|
||||||
echo "╔════════════════════════════════════════════════════════════╗"
|
|
||||||
echo "║ s2s-reasoning-engine — Node-RED v5 locale ║"
|
|
||||||
echo "╠════════════════════════════════════════════════════════════╣"
|
|
||||||
echo "║ Editor: http://localhost:${PORT}/ ║"
|
|
||||||
echo "║ API: http://localhost:${PORT}/api/reasoning ║"
|
|
||||||
echo "║ UserDir: ${USERDIR}"
|
|
||||||
echo "║ LogLevel: ${NODE_RED_LOG_LEVEL}"
|
|
||||||
echo "╚════════════════════════════════════════════════════════════╝"
|
|
||||||
echo ""
|
|
||||||
|
|
||||||
# ----------------------------------------------------------
|
|
||||||
# 3) Avvio Node-RED
|
|
||||||
# ----------------------------------------------------------
|
|
||||||
exec "$ROOT_DIR/node_modules/.bin/node-red" \
|
|
||||||
--userDir "$USERDIR" \
|
|
||||||
--settings "$SETTINGS" \
|
|
||||||
--port "$PORT"
|
|
||||||
@@ -1,139 +1,6 @@
|
|||||||
# LLM Summary
|
# Language model handlers
|
||||||
|
|
||||||
## Available LLM backends (`--llm_backend`)
|
This directory contains local Transformers/MLX and remote Responses/Chat
|
||||||
|
Completions handlers. See [language model backends](../../docs/speech/llm.md)
|
||||||
Runtime-supported values in `s2s_pipeline.py`:
|
for configuration and [realtime API](../../docs/speech/realtime-api.md) for the
|
||||||
|
tool-call and conversation flow.
|
||||||
- `transformers` → `language_model.py` (Transformers backend)
|
|
||||||
- `mlx-lm` → `language_model.py` (MLX backend)
|
|
||||||
- `responses-api` → `responses_api_language_model.py`
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
### 1) Transformers (`--llm_backend transformers`)
|
|
||||||
|
|
||||||
- Handler: `LanguageModelHandler`
|
|
||||||
- Typical use: local GPU/CPU inference using Hugging Face Transformers
|
|
||||||
- Backend-specific args prefix: `--llm_*`
|
|
||||||
- Shared args (from base): `--model_name`, `--chat_size`, `--init_chat_prompt`, `--enable_lang_prompt`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--llm_backend transformers \
|
|
||||||
--model_name Qwen/Qwen3-4B-Instruct-2507 \
|
|
||||||
--llm_device cuda \
|
|
||||||
--llm_torch_dtype float16 \
|
|
||||||
--llm_gen_max_new_tokens 128
|
|
||||||
```
|
|
||||||
|
|
||||||
Common options:
|
|
||||||
- `--llm_gen_min_new_tokens`
|
|
||||||
- `--llm_gen_temperature`
|
|
||||||
- `--llm_gen_do_sample`
|
|
||||||
- `--chat_size`
|
|
||||||
- `--init_chat_prompt`
|
|
||||||
|
|
||||||
### 2) MLX-LM (`--llm_backend mlx-lm`)
|
|
||||||
|
|
||||||
- Handler: `LanguageModelHandler`
|
|
||||||
- Typical use: Apple Silicon local inference
|
|
||||||
- Backend-specific args prefix: same as Transformers (`--llm_*`)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--llm_backend mlx-lm \
|
|
||||||
--model_name mlx-community/Qwen3-4B-Instruct-2507-bf16 \
|
|
||||||
--llm_device mps \
|
|
||||||
--llm_gen_max_new_tokens 128
|
|
||||||
```
|
|
||||||
|
|
||||||
Common options:
|
|
||||||
- `--llm_gen_temperature`
|
|
||||||
- `--llm_gen_do_sample`
|
|
||||||
- `--chat_size`
|
|
||||||
- `--init_chat_prompt`
|
|
||||||
|
|
||||||
### 3) OpenAI-compatible API (`--llm_backend responses-api`)
|
|
||||||
|
|
||||||
- Handler: `ResponsesApiModelHandler`
|
|
||||||
- Typical use: remote model serving via OpenAI-compatible endpoints
|
|
||||||
- Backend-specific args prefix: `--responses_api_*`
|
|
||||||
- Shared args (from base): `--model_name`, `--chat_size`, `--init_chat_prompt`, `--enable_lang_prompt`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--llm_backend responses-api \
|
|
||||||
--model_name gpt-5.4-mini \
|
|
||||||
--responses_api_api_key YOUR_API_KEY \
|
|
||||||
--responses_api_base_url https://api.example.com/v1 \
|
|
||||||
--responses_api_stream true
|
|
||||||
```
|
|
||||||
|
|
||||||
Common options:
|
|
||||||
- `--chat_size`
|
|
||||||
- `--init_chat_prompt`
|
|
||||||
- `--user_role`
|
|
||||||
|
|
||||||
## LLM Behavior
|
|
||||||
|
|
||||||
When STT is set to language auto-detection (`--language auto`), LLM handlers can receive `(text, language_code)` and prepend a language control instruction like:
|
|
||||||
|
|
||||||
- `Please reply to my message in <language>.`
|
|
||||||
|
|
||||||
This helps the assistant respond in the detected language. The behavior is opt-in via `--enable_lang_prompt` (shared across all backends); it defaults to `False`.
|
|
||||||
|
|
||||||
## Setup
|
|
||||||
|
|
||||||
### CUDA setup
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--llm_backend transformers \
|
|
||||||
--model_name microsoft/Phi-3-mini-4k-instruct
|
|
||||||
```
|
|
||||||
|
|
||||||
### Local Mac setup
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--local_mac_optimal_settings \
|
|
||||||
--model_name mlx-community/Qwen3-4B-Instruct-2507-bf16
|
|
||||||
```
|
|
||||||
|
|
||||||
`--local_mac_optimal_settings` already sets `--llm_backend mlx-lm` and will default the model to `mlx-community/Qwen3-4B-Instruct-2507-bf16` if not overridden.
|
|
||||||
|
|
||||||
### Realtime (OpenAI-compatible) setup
|
|
||||||
|
|
||||||
Run the server in realtime mode, then connect with the realtime client:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Start the pipeline in realtime mode
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--mode realtime \
|
|
||||||
--llm_backend mlx-lm \
|
|
||||||
--model_name mlx-community/Qwen3-4B-Instruct-2507-bf16 \
|
|
||||||
--ws_host 0.0.0.0 \
|
|
||||||
--ws_port 8765
|
|
||||||
|
|
||||||
# 2. Connect with the realtime client
|
|
||||||
python listen_and_play_realtime.py --host 127.0.0.1 --port 8765
|
|
||||||
```
|
|
||||||
|
|
||||||
Or with `--local_mac_optimal_settings` on Apple Silicon:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--local_mac_optimal_settings \
|
|
||||||
--mode realtime \
|
|
||||||
--ws_host 0.0.0.0 \
|
|
||||||
--ws_port 8765
|
|
||||||
```
|
|
||||||
|
|
||||||
### Remote API setup
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--llm_backend responses-api \
|
|
||||||
--model_name gpt-5.4-mini \
|
|
||||||
--responses_api_api_key YOUR_API_KEY
|
|
||||||
```
|
|
||||||
|
|||||||
@@ -196,7 +196,7 @@ class ChatCompletionsApiModelHandler(BaseOpenAICompatibleHandler):
|
|||||||
if self.stream:
|
if self.stream:
|
||||||
create_kwargs["stream_options"] = {"include_usage": True}
|
create_kwargs["stream_options"] = {"include_usage": True}
|
||||||
|
|
||||||
# Svuota i parametri dei tool per evitare il blocco 400 di NVIDIA NIM
|
# Remove tool parameters to avoid NVIDIA NIM's HTTP 400 response.
|
||||||
create_kwargs.pop("tools", None)
|
create_kwargs.pop("tools", None)
|
||||||
create_kwargs.pop("tool_choice", None)
|
create_kwargs.pop("tool_choice", None)
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ LLM Output Processor
|
|||||||
Intercepts LLM output to:
|
Intercepts LLM output to:
|
||||||
1. Extract tool calls and send them via text_output_queue
|
1. Extract tool calls and send them via text_output_queue
|
||||||
2. Forward clean text to TTS pipeline
|
2. Forward clean text to TTS pipeline
|
||||||
3. 2026-09-25: SANITIZE APIPILOT internal markers before clients / TTS (anti-leak tecnico)
|
3. Sanitize ApiPilot internal markers before sending output to clients or TTS
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -17,16 +17,15 @@ from queue import Queue
|
|||||||
from speech_to_speech.baseHandler import BaseHandler
|
from speech_to_speech.baseHandler import BaseHandler
|
||||||
from speech_to_speech.pipeline.events import AssistantTextEvent, ResponseFailedEvent, TokenUsageEvent
|
from speech_to_speech.pipeline.events import AssistantTextEvent, ResponseFailedEvent, TokenUsageEvent
|
||||||
|
|
||||||
# 2026-09-25 SANITIZE APIPILOT CALL/RESULT markers in LLM output BEFORE:
|
# Sanitize ApiPilot CALL/RESULT markers before emitting AssistantTextEvent
|
||||||
# (a) AssistantTextEvent viene mandato ai client WS (e loggato come <<< TEXT:/ASSISTANT:)
|
# to WebSocket clients or TTSInput to Qwen3-TTS, which would speak the markers.
|
||||||
# (b) TTSInput viene mandato a Qwen3-TTS (che lo pronuncerebbe ad alta voce).
|
# Observed formats from upstream Vocero start_pipeline_rag:
|
||||||
# Formati osservati reali oggi (upstream Vocero start_pipeline_rag):
|
# 1) <<<APIPILOT_CALL:NAME=tool,ARGUMENTS=json>>> standard triple brackets
|
||||||
# 1) <<<APIPILOT_CALL:NAME=tool,ARGUMENTS=json>>> standard 3x
|
|
||||||
# 2) <APIPILOT_CALL:NAME=get_user_briefing,ARGUMENTS={"format":"voice"}
|
# 2) <APIPILOT_CALL:NAME=get_user_briefing,ARGUMENTS={"format":"voice"}
|
||||||
# singolo '<' '>' + args inventati da AI (format=voice)
|
# single brackets and model-generated arguments
|
||||||
# 3) <<<APIPILOT_CALL:...}> \n>> spezzato multi-riga
|
# 3) <<<APIPILOT_CALL:...}> \n>> split across lines
|
||||||
# 4) >>>APIPILOT_CALL:...<<< invertito (errore sintassi AI)
|
# 4) >>>APIPILOT_CALL:...<<< inverted brackets
|
||||||
# 5) <<<APIPILOT_RESULT:PASS2>>> + blocco result.
|
# 5) <<<APIPILOT_RESULT:PASS2>>> followed by a result block
|
||||||
_APIPILOT_CALL_NORM = re.compile(
|
_APIPILOT_CALL_NORM = re.compile(
|
||||||
r"[<]{1,3}\s*APIPILOT_CALL:NAME=([A-Za-z0-9_.-]+),ARGUMENTS=(.*?)[>]{1,3}",
|
r"[<]{1,3}\s*APIPILOT_CALL:NAME=([A-Za-z0-9_.-]+),ARGUMENTS=(.*?)[>]{1,3}",
|
||||||
re.IGNORECASE | re.DOTALL,
|
re.IGNORECASE | re.DOTALL,
|
||||||
@@ -64,7 +63,7 @@ def _sanitize_apipilot_before_client_and_tts(text: str) -> tuple[str, bool, str]
|
|||||||
if _APIPILOT_RESULT.search(new_text):
|
if _APIPILOT_RESULT.search(new_text):
|
||||||
new_text = _APIPILOT_RESULT.sub(" ", new_text)
|
new_text = _APIPILOT_RESULT.sub(" ", new_text)
|
||||||
found_result = True
|
found_result = True
|
||||||
# safety catchall frammenti marker senza wrapper (se marker troncato mid-stream)
|
# Catch marker fragments without a wrapper, including mid-stream truncation.
|
||||||
low = new_text.lower()
|
low = new_text.lower()
|
||||||
if "apipilot" in low and ("call" in low or "result" in low or "name=" in low or "arguments=" in low):
|
if "apipilot" in low and ("call" in low or "result" in low or "name=" in low or "arguments=" in low):
|
||||||
new_text = re.sub(r"(?i)\bAPIPILOT(?:_CALL|_RESULT)?\b", " ", new_text)
|
new_text = re.sub(r"(?i)\bAPIPILOT(?:_CALL|_RESULT)?\b", " ", new_text)
|
||||||
@@ -184,8 +183,8 @@ class LMOutputProcessor(BaseHandler[LLMOut, TTSIn]):
|
|||||||
|
|
||||||
logger.debug(f"LM processor: text='{lm_output.text}', tools={lm_output.tools}")
|
logger.debug(f"LM processor: text='{lm_output.text}', tools={lm_output.tools}")
|
||||||
|
|
||||||
# 2026-09-25 SANITIZE SINGOLO PUNTO CENTRALE: PRIMA di AssistantTextEvent (client/log)
|
# Sanitize centrally before AssistantTextEvent and TTSInput so neither
|
||||||
# E PRIMA di TTSInput (Qwen3-TTS che pronuncerebbe marker APIPILOT CALL).
|
# clients nor Qwen3-TTS receive internal ApiPilot markers.
|
||||||
clean_text = lm_output.text
|
clean_text = lm_output.text
|
||||||
any_sanitized = False
|
any_sanitized = False
|
||||||
sanitize_tag = ""
|
sanitize_tag = ""
|
||||||
@@ -193,14 +192,14 @@ class LMOutputProcessor(BaseHandler[LLMOut, TTSIn]):
|
|||||||
clean_text, any_sanitized, sanitize_tag = _sanitize_apipilot_before_client_and_tts(clean_text)
|
clean_text, any_sanitized, sanitize_tag = _sanitize_apipilot_before_client_and_tts(clean_text)
|
||||||
if any_sanitized:
|
if any_sanitized:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"[LM-OUTPUT-APIPILOT-SANITIZE] %s — tool=%s rawLen=%d cleanLen=%d (rimosso / sostituito con placeholder spagnolo PRIMA di client + TTS).",
|
"[LM-OUTPUT-APIPILOT-SANITIZE] %s — tool=%s rawLen=%d cleanLen=%d (removed or replaced with the Spanish placeholder before client/TTS output).",
|
||||||
sanitize_tag or "-",
|
sanitize_tag or "-",
|
||||||
(sanitize_tag if sanitize_tag else "marker"),
|
(sanitize_tag if sanitize_tag else "marker"),
|
||||||
len(lm_output.text or ""),
|
len(lm_output.text or ""),
|
||||||
len(clean_text or ""),
|
len(clean_text or ""),
|
||||||
)
|
)
|
||||||
# evita che lo stesso chunk (marker solo) arrivi a TTS come "" → Hello.
|
# Use the placeholder when sanitization removes the entire chunk,
|
||||||
# Se dopo sanitize testo vuoto MA avevamo trovato marker → placeholder.
|
# avoiding empty text that would trigger the TTS greeting fallback.
|
||||||
if not clean_text.strip():
|
if not clean_text.strip():
|
||||||
clean_text = _APIPILOT_SPANISH_PLACEHOLDER
|
clean_text = _APIPILOT_SPANISH_PLACEHOLDER
|
||||||
|
|
||||||
|
|||||||
@@ -1,163 +1,5 @@
|
|||||||
# STT Summary
|
# Speech-to-text handlers
|
||||||
|
|
||||||
This document summarizes the Speech-to-Text (STT) implementations in the `STT/` folder, including language support, language abbreviations, and usage in `s2s_pipeline.py`.
|
This directory contains Whisper, Parakeet, and Paraformer handlers and live
|
||||||
|
transcription helpers. See [speech-to-text backends](../../docs/speech/stt.md)
|
||||||
## Available STT Modes (`--stt`)
|
for supported modes, language codes, and configuration.
|
||||||
|
|
||||||
- `whisper` → `STT/whisper_stt_handler.py`
|
|
||||||
- `whisper-mlx` → `STT/lightning_whisper_mlx_handler.py`
|
|
||||||
- `mlx-audio-whisper` → `STT/mlx_audio_whisper_handler.py`
|
|
||||||
- `faster-whisper` → `STT/faster_whisper_handler.py`
|
|
||||||
- `parakeet-tdt` → `STT/parakeet_tdt_handler.py`
|
|
||||||
- `paraformer` → `STT/paraformer_handler.py`
|
|
||||||
|
|
||||||
## Language Support by Handler
|
|
||||||
|
|
||||||
### 1) Whisper (`--stt whisper`)
|
|
||||||
|
|
||||||
- Handler: `WhisperSTTHandler`
|
|
||||||
- Language input flag: `--language` (from shared Whisper args)
|
|
||||||
- Supports fixed language (e.g. `en`) or `auto`
|
|
||||||
- Internal supported language list:
|
|
||||||
- `en`, `fr`, `es`, `zh`, `ja`, `ko`, `hi`, `de`, `pt`, `pl`, `it`, `nl`
|
|
||||||
- Behavior:
|
|
||||||
- Detects language from token output
|
|
||||||
- If detected language is outside the supported list, it falls back to the previous language
|
|
||||||
|
|
||||||
### 2) Lightning Whisper MLX (`--stt whisper-mlx`)
|
|
||||||
|
|
||||||
- Handler: `LightningWhisperSTTHandler`
|
|
||||||
- Uses same shared `--language` argument as Whisper
|
|
||||||
- Internal supported language list:
|
|
||||||
- `en`, `fr`, `es`, `zh`, `ja`, `ko`, `hi`, `de`, `pt`, `pl`, `it`, `nl`
|
|
||||||
- Behavior:
|
|
||||||
- If `--language auto`, model auto-detects each utterance
|
|
||||||
- If detected language is unsupported, falls back to last supported language
|
|
||||||
|
|
||||||
### 3) MLX Audio Whisper (`--stt mlx-audio-whisper`)
|
|
||||||
|
|
||||||
- Handler: `MLXAudioWhisperSTTHandler`
|
|
||||||
- Model flag: `--mlx_audio_whisper_model_name`
|
|
||||||
- Language still comes from shared `--language` flag (wired in pipeline)
|
|
||||||
- Internal supported language list:
|
|
||||||
- `en`, `fr`, `es`, `zh`, `ja`, `ko`, `hi`, `de`, `pt`, `pl`, `it`, `nl`
|
|
||||||
- Behavior:
|
|
||||||
- Uses fixed language unless `--language auto`
|
|
||||||
- Falls back to last known supported language when needed
|
|
||||||
|
|
||||||
### 4) Faster-Whisper (`--stt faster-whisper`)
|
|
||||||
|
|
||||||
- Handler: `FasterWhisperSTTHandler`
|
|
||||||
- Language flag: `--faster_whisper_stt_gen_language`
|
|
||||||
- Default language: `en`
|
|
||||||
- Note:
|
|
||||||
- This handler passes generation kwargs directly to `faster_whisper.WhisperModel.transcribe(...)`
|
|
||||||
- Effective language coverage depends on selected Faster-Whisper/OpenAI Whisper model
|
|
||||||
|
|
||||||
### 5) Parakeet TDT (`--stt parakeet-tdt`)
|
|
||||||
|
|
||||||
- Handler: `ParakeetTDTSTTHandler`
|
|
||||||
- Language flag: `--parakeet_tdt_language` (optional)
|
|
||||||
- Supports auto language detection when language not specified
|
|
||||||
- Declared supported language list (25 European languages):
|
|
||||||
- `en`, `de`, `fr`, `es`, `it`, `pt`, `nl`, `pl`, `ru`, `uk`, `cs`, `sk`, `hu`, `ro`, `bg`, `hr`, `sl`, `sr`, `da`, `no`, `sv`, `fi`, `et`, `lv`, `lt`
|
|
||||||
- Backend behavior:
|
|
||||||
- On macOS/MPS: MLX (`mlx-community/parakeet-tdt-0.6b-v3`)
|
|
||||||
- On CUDA/CPU: nano-parakeet (`nvidia/parakeet-tdt-0.6b-v3`)
|
|
||||||
|
|
||||||
### 6) Paraformer (`--stt paraformer`)
|
|
||||||
|
|
||||||
- Handler: `ParaformerSTTHandler`
|
|
||||||
- Model flag: `--paraformer_stt_model_name`
|
|
||||||
- Default model: `paraformer-zh`
|
|
||||||
- No dedicated language flag in current args class
|
|
||||||
- Practical support:
|
|
||||||
- Depends on selected FunASR model checkpoint
|
|
||||||
- Default setup is Chinese-oriented (`zh`)
|
|
||||||
|
|
||||||
## Language Abbreviations (ISO-style codes seen in STT handlers)
|
|
||||||
|
|
||||||
| Code | Language |
|
|
||||||
|---|---|
|
|
||||||
| `en` | English |
|
|
||||||
| `fr` | French |
|
|
||||||
| `es` | Spanish |
|
|
||||||
| `zh` | Chinese |
|
|
||||||
| `ja` | Japanese |
|
|
||||||
| `ko` | Korean |
|
|
||||||
| `hi` | Hindi |
|
|
||||||
| `de` | German |
|
|
||||||
| `pt` | Portuguese |
|
|
||||||
| `pl` | Polish |
|
|
||||||
| `it` | Italian |
|
|
||||||
| `nl` | Dutch |
|
|
||||||
| `ru` | Russian |
|
|
||||||
| `uk` | Ukrainian |
|
|
||||||
| `cs` | Czech |
|
|
||||||
| `sk` | Slovak |
|
|
||||||
| `hu` | Hungarian |
|
|
||||||
| `ro` | Romanian |
|
|
||||||
| `bg` | Bulgarian |
|
|
||||||
| `hr` | Croatian |
|
|
||||||
| `sl` | Slovenian |
|
|
||||||
| `sr` | Serbian |
|
|
||||||
| `da` | Danish |
|
|
||||||
| `no` | Norwegian |
|
|
||||||
| `sv` | Swedish |
|
|
||||||
| `fi` | Finnish |
|
|
||||||
| `et` | Estonian |
|
|
||||||
| `lv` | Latvian |
|
|
||||||
| `lt` | Lithuanian |
|
|
||||||
| `auto` | Per-utterance automatic language detection |
|
|
||||||
|
|
||||||
## Usage Examples
|
|
||||||
|
|
||||||
### Whisper (Transformers)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py --stt whisper --language en
|
|
||||||
python s2s_pipeline.py --stt whisper --language auto
|
|
||||||
```
|
|
||||||
|
|
||||||
### Whisper MLX (LightningWhisperMLX)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py --stt whisper-mlx --language auto --device mps
|
|
||||||
```
|
|
||||||
|
|
||||||
### MLX Audio Whisper
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py --stt mlx-audio-whisper \
|
|
||||||
--mlx_audio_whisper_model_name mlx-community/whisper-large-v3-turbo \
|
|
||||||
--language auto
|
|
||||||
```
|
|
||||||
|
|
||||||
### Faster-Whisper
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py --stt faster-whisper \
|
|
||||||
--faster_whisper_stt_model_name large-v3 \
|
|
||||||
--faster_whisper_stt_gen_language en
|
|
||||||
```
|
|
||||||
|
|
||||||
### Parakeet TDT
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py --stt parakeet-tdt --parakeet_tdt_device auto
|
|
||||||
python s2s_pipeline.py --stt parakeet-tdt --parakeet_tdt_language de
|
|
||||||
```
|
|
||||||
|
|
||||||
With live transcription (MLX or CUDA/nano-parakeet backend):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py --stt parakeet-tdt \
|
|
||||||
--enable_live_transcription \
|
|
||||||
--live_transcription_update_interval 0.25
|
|
||||||
```
|
|
||||||
|
|
||||||
### Paraformer
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py --stt paraformer --paraformer_stt_model_name paraformer-zh
|
|
||||||
```
|
|
||||||
|
|||||||
@@ -1,155 +1,5 @@
|
|||||||
# TTS Summary
|
# Text-to-speech handlers
|
||||||
|
|
||||||
## Available TTS modes (`--tts`)
|
This directory contains ChatTTS, Facebook MMS, Pocket, Kokoro, and Qwen3-TTS
|
||||||
|
handlers. See [text-to-speech backends](../../docs/speech/tts.md) for configuration,
|
||||||
Runtime-supported values in `s2s_pipeline.py`:
|
voice selection, and the optional reference-audio example.
|
||||||
|
|
||||||
- `chatTTS` → `chatTTS_handler.py`
|
|
||||||
- `facebookMMS` → `facebookmms_handler.py`
|
|
||||||
- `pocket` → `pocket_tts_handler.py`
|
|
||||||
- `kokoro` → `kokoro_handler.py`
|
|
||||||
- `qwen3` → `qwen3_tts_handler.py`
|
|
||||||
|
|
||||||
Deprecated TTS implementations, including MeloTTS, live in [`../../../archive/TTS`](../../../archive/TTS) and are no longer wired into `s2s_pipeline.py`.
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
### 1) ChatTTS (`--tts chatTTS`)
|
|
||||||
|
|
||||||
Primary args prefix: `--chat_tts_*`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--tts chatTTS \
|
|
||||||
--chat_tts_device cuda \
|
|
||||||
--chat_tts_stream true \
|
|
||||||
--chat_tts_chunk_size 512
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2) Facebook MMS (`--tts facebookMMS`)
|
|
||||||
|
|
||||||
Primary args prefix: `--facebook_mms_*` plus `--tts_language`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--tts facebookMMS \
|
|
||||||
--facebook_mms_device cuda \
|
|
||||||
--tts_language en
|
|
||||||
```
|
|
||||||
|
|
||||||
This handler maps STT language codes (e.g. `en`, `fr`, `es`) to MMS model suffixes (e.g. `eng`, `fra`, `spa`) and reloads the model on language changes.
|
|
||||||
|
|
||||||
### 3) Pocket TTS (`--tts pocket`)
|
|
||||||
|
|
||||||
Primary args prefix: `--pocket_tts_*`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--tts pocket \
|
|
||||||
--pocket_tts_voice jean \
|
|
||||||
--pocket_tts_device cpu \
|
|
||||||
--pocket_tts_sample_rate 16000
|
|
||||||
```
|
|
||||||
|
|
||||||
Available preset voices include:
|
|
||||||
`alba`, `marius`, `javert`, `jean`, `fantine`, `cosette`, `eponine`, `azelma`.
|
|
||||||
|
|
||||||
### 4) Kokoro (`--tts kokoro`)
|
|
||||||
|
|
||||||
Primary args prefix: `--kokoro_*`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--tts kokoro \
|
|
||||||
--kokoro_device auto \
|
|
||||||
--kokoro_voice bm_fable \
|
|
||||||
--kokoro_lang_code b
|
|
||||||
```
|
|
||||||
|
|
||||||
Behavior:
|
|
||||||
- Uses MLX backend on Apple Silicon (`mlx-community/Kokoro-82M-bf16`)
|
|
||||||
- Uses native kokoro pipeline otherwise (`hexgrad/Kokoro-82M`)
|
|
||||||
- Can auto-switch voice/language based on STT language code mapping
|
|
||||||
|
|
||||||
### 5) Qwen3-TTS (`--tts qwen3`)
|
|
||||||
|
|
||||||
Primary args prefix: `--qwen3_tts_*`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--tts qwen3 \
|
|
||||||
--qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
|
|
||||||
--qwen3_tts_device cuda \
|
|
||||||
--qwen3_tts_backend ggml \
|
|
||||||
--qwen3_tts_speaker Aiden \
|
|
||||||
--qwen3_tts_language auto \
|
|
||||||
--qwen3_tts_non_streaming_mode True
|
|
||||||
```
|
|
||||||
|
|
||||||
Behavior:
|
|
||||||
- Uses `faster-qwen3-tts` on non-macOS platforms, defaulting to the GGML backend. Pass `--qwen3_tts_backend torch` to use the CUDA-graphs backend instead.
|
|
||||||
- Uses `mlx-audio` on Apple Silicon and auto-maps `Qwen/...` model IDs to `mlx-community/...`, defaulting to the `6bit` MLX variant unless the model name already pins a suffix.
|
|
||||||
- Supports MLX quantization overrides on Apple Silicon via `--qwen3_tts_mlx_quantization bf16|4bit|6bit|8bit`.
|
|
||||||
- Keeps the existing voice-clone/custom-voice/voice-design handler flow intact.
|
|
||||||
- Defaults to the CustomVoice model with speaker `Aiden`, so no reference audio is required. Voice-clone/base models can still use `--qwen3_tts_ref_audio`.
|
|
||||||
|
|
||||||
Install notes for Linux GGML:
|
|
||||||
- The default PyPI `qwentts-cpp-python` wheel targets CUDA 12.8.
|
|
||||||
- If that wheel does not match your CUDA runtime, install one of the Hugging Face wheelhouse builds before installing `speech-to-speech`.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install "qwentts-cpp-python==0.3.1+cu130" \
|
|
||||||
-f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu130
|
|
||||||
pip install speech-to-speech
|
|
||||||
```
|
|
||||||
|
|
||||||
Available wheelhouse directories include `cu124`, `cu128`, `cu130`, and `cpu`.
|
|
||||||
|
|
||||||
Example for Apple Silicon using the default 6-bit MLX variant:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--tts qwen3 \
|
|
||||||
--qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
|
|
||||||
--qwen3_tts_speaker Aiden
|
|
||||||
```
|
|
||||||
|
|
||||||
You can override the default and select `bf16`, `4bit`, or `8bit` explicitly:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--tts qwen3 \
|
|
||||||
--qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
|
|
||||||
--qwen3_tts_mlx_quantization 4bit \
|
|
||||||
--qwen3_tts_speaker Aiden
|
|
||||||
```
|
|
||||||
|
|
||||||
To benchmark the Apple Silicon MLX variants side by side:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
.venv/bin/python benchmark_tts.py \
|
|
||||||
--handlers qwen3 \
|
|
||||||
--iterations 3 \
|
|
||||||
--qwen3_mlx_quantizations bf16 4bit 6bit 8bit
|
|
||||||
```
|
|
||||||
|
|
||||||
This will run separate benchmark entries for `qwen3[bf16]`, `qwen3[4bit]`, `qwen3[6bit]`, and `qwen3[8bit]`.
|
|
||||||
|
|
||||||
## Setup
|
|
||||||
|
|
||||||
### Low-latency GPU setup
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--stt whisper \
|
|
||||||
--tts pocket
|
|
||||||
```
|
|
||||||
|
|
||||||
### Apple Silicon setup
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python s2s_pipeline.py \
|
|
||||||
--local_mac_optimal_settings
|
|
||||||
```
|
|
||||||
|
|
||||||
`--tts pocket` and `--tts kokoro` are also valid options on macOS.
|
|
||||||
|
|||||||
@@ -13,16 +13,15 @@ import re
|
|||||||
import tempfile
|
import tempfile
|
||||||
import unicodedata
|
import unicodedata
|
||||||
|
|
||||||
# 2026-09-25 SANITIZE APIPILOT MARKERS PRIMA DI TTS (anti-leak pronunciato da Qwen3).
|
# Sanitize ApiPilot markers directly before TTS as an additional safeguard
|
||||||
# Lato server S2S Python abbiamo gia' gate + sanitize ma come DOPPIA RETE
|
# alongside the S2S relay's audio gating and sanitization.
|
||||||
# di sicurezza sanitizziamo DIRETTAMENTE il testo che entra in Qwen3-TTS.
|
# Observed formats:
|
||||||
# Formati osservati reali oggi:
|
# (a) <<<APIPILOT_CALL:NAME=tool,ARGUMENTS=json>>> standard triple brackets
|
||||||
# (a) <<<APIPILOT_CALL:NAME=tool,ARGUMENTS=json>>> standard 3 parentesi
|
|
||||||
# (b) <APIPILOT_CALL:NAME=tool,ARGUMENTS={"format":"voice"}
|
# (b) <APIPILOT_CALL:NAME=tool,ARGUMENTS={"format":"voice"}
|
||||||
# singolo < > (caso user oggi)
|
# single brackets
|
||||||
# (c) <<<APIPILOT_CALL:...}> \n>> spezzato piu' righe
|
# (c) <<<APIPILOT_CALL:...}> \n>> split across lines
|
||||||
# (d) >>>APIPILOT_CALL:...<<< invertito
|
# (d) >>>APIPILOT_CALL:...<<< inverted brackets
|
||||||
# Regola: CALL -> placeholder spagnolo (frase naturale utente), RESULT -> strip.
|
# Replace CALL with a natural Spanish utterance; strip RESULT markers.
|
||||||
_APIPILOT_CALL_NORMAL_RE = re.compile(
|
_APIPILOT_CALL_NORMAL_RE = re.compile(
|
||||||
r"[<]{1,3}\s*APIPILOT_CALL:NAME=([A-Za-z0-9_.-]+),ARGUMENTS=(.*?)[>]{1,3}",
|
r"[<]{1,3}\s*APIPILOT_CALL:NAME=([A-Za-z0-9_.-]+),ARGUMENTS=(.*?)[>]{1,3}",
|
||||||
re.IGNORECASE | re.DOTALL,
|
re.IGNORECASE | re.DOTALL,
|
||||||
@@ -39,7 +38,7 @@ _APIPILOT_SPANISH_PLACEHOLDER = "Voy a consultar tus datos, un momento por favor
|
|||||||
|
|
||||||
|
|
||||||
def _tts_sanitize_apipilot_markers(text: str) -> tuple[str, bool, str]:
|
def _tts_sanitize_apipilot_markers(text: str) -> tuple[str, bool, str]:
|
||||||
"""Rimuove marker APIPILOT dal testo TTS.
|
"""Remove ApiPilot markers from TTS text.
|
||||||
|
|
||||||
Returns (sanitized_text, found_any, log_tag).
|
Returns (sanitized_text, found_any, log_tag).
|
||||||
"""
|
"""
|
||||||
@@ -48,7 +47,7 @@ def _tts_sanitize_apipilot_markers(text: str) -> tuple[str, bool, str]:
|
|||||||
found_call = False
|
found_call = False
|
||||||
found_result = False
|
found_result = False
|
||||||
tool_name = ""
|
tool_name = ""
|
||||||
# 1) marker CALL standard + invertito -> placeholder spagnolo
|
# 1) Replace standard/inverted CALL markers with the Spanish placeholder.
|
||||||
def _repl_call(m: re.Match) -> str:
|
def _repl_call(m: re.Match) -> str:
|
||||||
nonlocal found_call, tool_name
|
nonlocal found_call, tool_name
|
||||||
found_call = True
|
found_call = True
|
||||||
@@ -62,11 +61,11 @@ def _tts_sanitize_apipilot_markers(text: str) -> tuple[str, bool, str]:
|
|||||||
text2 = _APIPILOT_CALL_NORMAL_RE.sub(_repl_call, text)
|
text2 = _APIPILOT_CALL_NORMAL_RE.sub(_repl_call, text)
|
||||||
if not found_call:
|
if not found_call:
|
||||||
text2 = _APIPILOT_CALL_INVERTED_RE.sub(_repl_call, text2)
|
text2 = _APIPILOT_CALL_INVERTED_RE.sub(_repl_call, text2)
|
||||||
# 2) marker RESULT -> strip silenzioso (non sono pronunciabili)
|
# 2) Strip RESULT markers silently; they must not be spoken.
|
||||||
if _APIPILOT_RESULT_RE.search(text2):
|
if _APIPILOT_RESULT_RE.search(text2):
|
||||||
found_result = True
|
found_result = True
|
||||||
text2 = _APIPILOT_RESULT_RE.sub(" ", text2)
|
text2 = _APIPILOT_RESULT_RE.sub(" ", text2)
|
||||||
# 3) safety: FRAMMENTI marker singoli che non hanno wrapper completo
|
# 3) Catch marker fragments without a complete wrapper.
|
||||||
low = text2.lower()
|
low = text2.lower()
|
||||||
if "apipilot" in low and ("call" in low or "result" in low):
|
if "apipilot" in low and ("call" in low or "result" in low):
|
||||||
import re as _re_sf
|
import re as _re_sf
|
||||||
@@ -782,18 +781,16 @@ class Qwen3TTSHandler(BaseHandler[TTSIn, TTSOut]):
|
|||||||
coalesced_text, _language_code, _saw_end_of_response = self._coalesce_pending_tts_input(tts_input)
|
coalesced_text, _language_code, _saw_end_of_response = self._coalesce_pending_tts_input(tts_input)
|
||||||
|
|
||||||
raw_text = coalesced_text or "Hello."
|
raw_text = coalesced_text or "Hello."
|
||||||
# 2026-09-25 SANITIZE APIPILOT PRIMA DI TTS (doppia rete sicurezza):
|
# A race or a new marker format can bypass relay sanitization. Strip or
|
||||||
# Anche se lato server S2s Python abbiamo gia' gate audio drop + sanitize,
|
# replace markers again before passing text to the TTS model.
|
||||||
# Qwen3-TTS puo' ricevere marker per race / nuovi formati. Quindi QUI
|
|
||||||
# eliminiamo / sostituiamo con placeholder PRIMA di chiamare generate().
|
|
||||||
text, found_marker, sanitize_tag = _tts_sanitize_apipilot_markers(raw_text)
|
text, found_marker, sanitize_tag = _tts_sanitize_apipilot_markers(raw_text)
|
||||||
if found_marker:
|
if found_marker:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"[TTS-APIPILOT-SANITIZE] marker=%s rawLen=%d sanitizedLen=%d → sostituito con placeholder spagnolo / strip.",
|
"[TTS-APIPILOT-SANITIZE] marker=%s rawLen=%d sanitizedLen=%d → replaced with the Spanish placeholder or stripped.",
|
||||||
sanitize_tag, len(raw_text or ""), len(text or ""),
|
sanitize_tag, len(raw_text or ""), len(text or ""),
|
||||||
)
|
)
|
||||||
if not text.strip():
|
if not text.strip():
|
||||||
# Safety se il marker era l'unica cosa: ripetiamo placeholder invece di Hello.
|
# Use the Spanish placeholder when the marker was the entire utterance.
|
||||||
text = _APIPILOT_SPANISH_PLACEHOLDER
|
text = _APIPILOT_SPANISH_PLACEHOLDER
|
||||||
|
|
||||||
model_type = self._model_type()
|
model_type = self._model_type()
|
||||||
|
|||||||
@@ -1,216 +1,6 @@
|
|||||||
## Realtime Engine -- High-Level Architecture
|
# Realtime API implementation
|
||||||
|
|
||||||
The server runs as a handler thread inside the existing pipeline. A FastAPI/uvicorn instance serves the WebSocket endpoint at `/v1/realtime`, while the same queue-driven handler chain (VAD, STT, LLM, TTS) processes audio in parallel threads.
|
This directory implements the OpenAI Realtime-compatible WebSocket service,
|
||||||
|
session configuration, event handlers, and pipeline routing. See the
|
||||||
```mermaid
|
[realtime API guide](../../../docs/speech/realtime-api.md) for architecture,
|
||||||
flowchart LR
|
supported events, tool calling, cancellation, and startup examples.
|
||||||
subgraph client [Client]
|
|
||||||
WS["WebSocket /v1/realtime"]
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph server [Realtime Server]
|
|
||||||
Router["WebSocket Router"]
|
|
||||||
Service["RealtimeService"]
|
|
||||||
Config["RuntimeConfig"]
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph pipeline [Pipeline Threads]
|
|
||||||
VAD["VAD"]
|
|
||||||
STT["STT"]
|
|
||||||
TN["TranscriptionNotifier"]
|
|
||||||
LLM["LLM"]
|
|
||||||
Proc["LMOutputProcessor"]
|
|
||||||
TTS["TTS"]
|
|
||||||
end
|
|
||||||
|
|
||||||
WS -- "client events (JSON)" --> Router
|
|
||||||
Router -- "parse / dispatch" --> Service
|
|
||||||
Service -- "PCM chunks" --> VAD
|
|
||||||
Service -- "session config" --> Config
|
|
||||||
Config -. "read by" .-> VAD & LLM & TTS
|
|
||||||
VAD -- "speech segments" --> STT
|
|
||||||
STT -- "transcript" --> TN
|
|
||||||
TN -- "transcript + notify" --> LLM
|
|
||||||
LLM -- "text + tools" --> Proc
|
|
||||||
Proc -- "clean text" --> TTS
|
|
||||||
TTS -- "PCM audio" --> Router
|
|
||||||
Proc -- "assistant_text / tools" --> Router
|
|
||||||
VAD -- "speech_started/stopped" --> Router
|
|
||||||
TN -- "transcription events" --> Router
|
|
||||||
Router -- "server events (JSON)" --> WS
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key flow:**
|
|
||||||
|
|
||||||
1. **Inbound audio**: Client sends `input_audio_buffer.append` with base64 PCM. `RealtimeService` decodes, resamples to 16 kHz, splits into 512-sample chunks, and puts them on the `recv_audio_chunks_queue` for VAD.
|
|
||||||
2. **Speech detection**: VAD detects speech boundaries and emits `speech_started` / `speech_stopped` events on the `text_output_queue`. Full utterance audio goes to STT.
|
|
||||||
3. **Transcription**: STT output passes through `TranscriptionNotifier`, which taps the transcript for `transcription.delta` / `transcription.completed` events before forwarding to the LLM.
|
|
||||||
4. **Generation**: The LLM generates text (and optional tool calls). `LMOutputProcessor` splits the output: clean text goes to TTS, and `assistant_text` + tool call dicts go to the `text_output_queue`.
|
|
||||||
5. **Outbound audio**: TTS writes PCM chunks to `send_audio_chunks_queue`. The router's async `_send_loop` drains both queues, encoding PCM as `response.output_audio.delta` events and translating internal messages into protocol events.
|
|
||||||
6. **Session config**: `session.update` events deep-merge into `RuntimeConfig`, which is a shared Pydantic model read by VAD (turn detection thresholds), LLM (instructions, tools), and TTS (voice) at processing time.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Supported OpenAI Realtime Events
|
|
||||||
|
|
||||||
### Client -> Server
|
|
||||||
|
|
||||||
| Event | Description |
|
|
||||||
|---|---|
|
|
||||||
| `input_audio_buffer.append` | Stream base64 PCM audio. Decoded, resampled to 16 kHz, and chunked for the VAD. |
|
|
||||||
| `session.update` | Deep-merge session config (instructions, tools, voice, turn detection, audio format). |
|
|
||||||
| `conversation.item.create` | Inject `input_text` or `function_call_output` into the LLM context without triggering generation. |
|
|
||||||
| `response.create` | Trigger LLM generation. Supports per-response `instructions` and `tool_choice` overrides. |
|
|
||||||
| `response.cancel` | Cancel the in-progress response and re-enable listening. |
|
|
||||||
|
|
||||||
### Server -> Client
|
|
||||||
|
|
||||||
| Event | Description |
|
|
||||||
|---|---|
|
|
||||||
| `session.created` | Sent on connection with current session config. |
|
|
||||||
| `error` | Protocol errors (`session_limit_reached`, `unknown_or_invalid_event`, `invalid_session_type`, `conversation_already_has_active_response`, etc.) |
|
|
||||||
| `input_audio_buffer.speech_started` | VAD detected user speech. |
|
|
||||||
| `input_audio_buffer.speech_stopped` | End of user speech segment. |
|
|
||||||
| `conversation.item.created` | Acknowledges injected `input_text` from `conversation.item.create`. |
|
|
||||||
| `conversation.item.input_audio_transcription.delta` | Streaming partial transcript (when live transcription is enabled). |
|
|
||||||
| `conversation.item.input_audio_transcription.completed` | Final transcript for the user turn (with duration usage). |
|
|
||||||
| `response.created` | Emitted on the first outbound audio chunk (response is `in_progress`). |
|
|
||||||
| `response.output_audio.delta` | Base64 PCM audio chunk from TTS. |
|
|
||||||
| `response.output_audio.done` | Audio stream complete for the current output item. |
|
|
||||||
| `response.output_audio_transcript.done` | Full assistant text transcript for the turn. |
|
|
||||||
| `response.function_call_arguments.done` | Tool call with `call_id`, `name`, and JSON `arguments`. |
|
|
||||||
| `response.done` | Response finished (`completed`, `cancelled` with reason `turn_detected` or `client_cancelled`). |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Tool Calling Design
|
|
||||||
|
|
||||||
Tool calling works through two distinct paths depending on the LLM backend, but both converge to the same wire protocol for the client.
|
|
||||||
|
|
||||||
### Local LLM path (`LanguageModelHandler` -- transformers / mlx-lm)
|
|
||||||
|
|
||||||
Tools defined in `session.update` are converted to `FunctionTool` objects. Each tool's JSON Schema `parameters` are turned into a Python `inspect.Signature` (via `signature_from_schema`), and `to_code_prompt()` renders a human-readable `def name(...): """docstring"""` block.
|
|
||||||
|
|
||||||
These tool signatures are injected into the system prompt using a Jinja2 template (`tool_prompt.py`) that instructs the model to wrap tool calls in `<code>...</code>` delimiters:
|
|
||||||
|
|
||||||
```
|
|
||||||
<code>
|
|
||||||
function_name(arg_name_1=value1, arg_name_2='string_value')
|
|
||||||
</code>
|
|
||||||
```
|
|
||||||
|
|
||||||
After generation, `_extract_tools` uses a regex to find `<code>` blocks, then `extract_function_calls_from_text` parses each `name(kwargs)` call and validates it against registered tools. Valid calls become `ResponseFunctionToolCall` dicts with generated `call_id`s.
|
|
||||||
|
|
||||||
### OpenAI API path (`ResponsesApiModelHandler`)
|
|
||||||
|
|
||||||
Tools are passed natively as the `tools=` parameter to `client.responses.create`. The API returns structured `function_call` items directly -- no prompt engineering or regex parsing needed. Per-response `tool_choice` overrides from `response.create` are supported.
|
|
||||||
|
|
||||||
### Common output path
|
|
||||||
|
|
||||||
Both handlers yield `(text, language_code, tools)` tuples. `LMOutputProcessor` forwards clean text to TTS and puts `{"type": "assistant_text", "text": ..., "tools": [...]}` on the `text_output_queue`. The router's `_send_loop` translates these into:
|
|
||||||
- `response.output_audio_transcript.done` for the text
|
|
||||||
- `response.function_call_arguments.done` for each tool call
|
|
||||||
|
|
||||||
### Tool result flow
|
|
||||||
|
|
||||||
1. Client executes the tool and sends `conversation.item.create` with `type: "function_call_output"` and `output: "<result>"`
|
|
||||||
2. `RealtimeService` appends the tool output to the chat context and emits `conversation.item.created`; this does not trigger generation.
|
|
||||||
3. If the tool result needs to be spoken to the user, such as camera/search/data results, the client sends `response.create` to trigger follow-up generation.
|
|
||||||
4. For fire-and-forget robot actions such as dance, emotion, head movement, stop, or idle tools, the client can stop after `conversation.item.created`; the assistant should already have spoken the natural lead-in before the tool call.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Interruption Handling
|
|
||||||
|
|
||||||
Barge-in (user speaks while the assistant is playing audio) is handled cooperatively between the VAD, the `_send_loop`, and the LLM/TTS handlers via a shared `CancelScope` object (`cancel_scope.py`).
|
|
||||||
|
|
||||||
### CancelScope design
|
|
||||||
|
|
||||||
`CancelScope` replaces the old two-signal pattern (`cancel_response` Event + `discard_stale_output` boolean) with a single object that manages:
|
|
||||||
|
|
||||||
- **Generation counter** (`cancel_scope.generation`): pipeline threads (LLM, TTS) capture the current generation at the start of each response and check `cancel_scope.is_stale(gen)` on every streaming token. When `cancel()` is called, the generation increments and all prior generations become stale -- no timing games required.
|
|
||||||
- **Discard flag** (`cancel_scope.discarding`): set by `cancel()`, checked by the async `_send_loop` to drop output from superseded generations that arrives between `cancel()` and `response_done()`. Cleared by `response_done(generation)` (only when the sentinel's generation matches the discarded or current one -- sentinels from unrelated older generations are ignored), by `new_response()` on an explicit `response.create`, or by `reset()` on session claim/release.
|
|
||||||
|
|
||||||
Pipeline output is **generation-tagged**: `AudioOutput` chunks and `AssistantTextEvent`s carry a `cancel_generation` field stamped by the handler that produced them. The send loop's `_generation_is_discardable` drops an item if its generation is stale, or if `discarding` is set and the item is not from the current generation. Output from the *current* generation always passes through, so a fresh response is never swallowed by a lingering discard window (e.g. a superseded speculative turn whose TTS never emitted a `__RESPONSE_DONE__` sentinel).
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
sequenceDiagram
|
|
||||||
participant User
|
|
||||||
participant VAD
|
|
||||||
participant SendLoop as _send_loop
|
|
||||||
participant LLM
|
|
||||||
participant TTS
|
|
||||||
participant Client
|
|
||||||
|
|
||||||
Note over TTS,Client: Response active or pending (in_response / response_pending)
|
|
||||||
User->>VAD: speaks
|
|
||||||
VAD->>SendLoop: speech_started on text_output_queue
|
|
||||||
SendLoop->>Client: input_audio_buffer.speech_started
|
|
||||||
SendLoop->>Client: response.done (status=cancelled, reason=turn_detected)
|
|
||||||
SendLoop->>SendLoop: cancel_scope.cancel() (gen++ & discarding=True)
|
|
||||||
SendLoop->>SendLoop: flush output_queue + text_output_queue
|
|
||||||
SendLoop->>SendLoop: response_playing.clear()
|
|
||||||
LLM->>LLM: is_stale(gen) → True, aborts generation
|
|
||||||
TTS->>TTS: is_stale(gen) → True, aborts generation
|
|
||||||
TTS->>SendLoop: __RESPONSE_DONE__ (tagged with gen)
|
|
||||||
SendLoop->>SendLoop: cancel_scope.response_done(gen) (discarding=False)
|
|
||||||
Note over VAD,Client: Pipeline is now processing the new user utterance
|
|
||||||
```
|
|
||||||
|
|
||||||
**Step by step:**
|
|
||||||
|
|
||||||
1. **VAD detects speech**: puts a `SpeechStartedEvent` on `text_output_queue`.
|
|
||||||
2. **`_send_loop` processes text events first** (priority over audio): translates `speech_started` into protocol events. If an active response was in progress, `RealtimeService.dispatch_pipeline_event` emits `response.output_audio.done` + `response.done` with `status="cancelled"` and `reason="turn_detected"`.
|
|
||||||
3. **Cancel + queue flush**: if a response is active (`in_response`) *or* pending (`response_pending` -- accepted `response.create`, no audio yet), and interrupts are enabled (see step 4), the send loop calls `cancel_scope.cancel()` (increments generation, enables discard), clears `response_pending`, drains `output_queue` (preserving `__RESPONSE_DONE__` sentinels) and `text_output_queue` (preserving user-side events: `speech_stopped`, partial/completed transcriptions, token usage), then clears `response_playing`.
|
|
||||||
4. **Interrupt gating**: the cancel only fires if the `SpeechStartedEvent.interrupt_response` flag is set *and* the session config allows it (`turn_detection.interrupt_response`, read via `RuntimeConfig.interrupt_response_enabled`, default true). When disabled, user speech during a response is transcribed but the response keeps playing.
|
|
||||||
5. **LLM/TTS cancellation**: handlers capture `gen = cancel_scope.generation` at the start of each response and check `cancel_scope.is_stale(gen)` on every streaming token, aborting early when stale.
|
|
||||||
6. **Discard guard**: while `cancel_scope.discarding` is True, the send loop drops audio chunks and assistant text whose `cancel_generation` is not current (see `_generation_is_discardable` above). The guard clears when a `__RESPONSE_DONE__` with a matching generation arrives (via `cancel_scope.response_done(gen)`), or when an explicit `response.create` starts a new response (`cancel_scope.new_response()`).
|
|
||||||
7. **Client-initiated cancel**: `response.cancel` calls `cancel_scope.cancel()` (only if a response was active), flushes both queues with the same preservation rules, triggers `finish_response(status="cancelled", reason="client_cancelled")`, re-enables `should_listen`, and clears `response_playing`.
|
|
||||||
8. **Spurious cancel safety**: if no response is active, `cancel_scope.cancel()` is not called, preventing the discard guard from being set without a `__RESPONSE_DONE__` to clear it.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
### Local LLM with Transformers
|
|
||||||
|
|
||||||
```bash
|
|
||||||
uv run speech-to-speech \
|
|
||||||
--mode realtime \
|
|
||||||
--stt parakeet-tdt \
|
|
||||||
--llm_backend transformers \
|
|
||||||
--tts kokoro \
|
|
||||||
--model_name "Qwen/Qwen3-4B-Instruct-2507" \
|
|
||||||
--llm_device mps \
|
|
||||||
--llm_torch_dtype float16 \
|
|
||||||
--enable_live_transcription
|
|
||||||
```
|
|
||||||
|
|
||||||
### Local LLM with MLX-LM
|
|
||||||
|
|
||||||
```bash
|
|
||||||
uv run speech-to-speech \
|
|
||||||
--mode realtime \
|
|
||||||
--stt parakeet-tdt \
|
|
||||||
--llm_backend mlx-lm \
|
|
||||||
--tts kokoro \
|
|
||||||
--model_name "mlx-community/Qwen3-4B-Instruct-2507-bf16" \
|
|
||||||
--llm_device mps \
|
|
||||||
--llm_torch_dtype float16 \
|
|
||||||
--enable_live_transcription
|
|
||||||
```
|
|
||||||
|
|
||||||
### Remote LLM with OpenAI-compatible API
|
|
||||||
|
|
||||||
```bash
|
|
||||||
uv run speech-to-speech \
|
|
||||||
--mode realtime \
|
|
||||||
--stt parakeet-tdt \
|
|
||||||
--llm_backend responses-api \
|
|
||||||
--tts kokoro \
|
|
||||||
--model_name "openai/gpt-oss-20b:groq" \
|
|
||||||
--responses_api_base_url "https://router.huggingface.co/v1" \
|
|
||||||
--responses_api_api_key "$HF_TOKEN" \
|
|
||||||
--responses_api_stream \
|
|
||||||
--enable_live_transcription
|
|
||||||
```
|
|
||||||
|
|||||||
@@ -30,7 +30,7 @@ class SessionHandler(RealtimeBaseHandler):
|
|||||||
"""
|
"""
|
||||||
s = event.session
|
s = event.session
|
||||||
|
|
||||||
# Forza il bypass della voce inviata dal browser
|
# Ignore the voice supplied by the browser.
|
||||||
if hasattr(s, "voice") and s.voice is not None:
|
if hasattr(s, "voice") and s.voice is not None:
|
||||||
s.voice = None
|
s.voice = None
|
||||||
elif isinstance(s, dict) and "voice" in s:
|
elif isinstance(s, dict) and "voice" in s:
|
||||||
|
|||||||
@@ -277,11 +277,10 @@ class RealtimeService:
|
|||||||
try:
|
try:
|
||||||
return model_cls.model_validate(raw) # type: ignore[return-value]
|
return model_cls.model_validate(raw) # type: ignore[return-value]
|
||||||
except ValidationError as e:
|
except ValidationError as e:
|
||||||
# ── Retrocompatibilità session.update ─────────────────────────────────
|
# Backward-compatible session.update parsing.
|
||||||
# Il formato standard OpenAI Realtime GA annida instructions/tools/tool_choice
|
# OpenAI Realtime GA nests instructions/tools/tool_choice in "session".
|
||||||
# DENTRO "session": { ... }. Per tollerare anche client che li mandano
|
# Accept older clients that send them at the root by rebuilding the
|
||||||
# alla RADICE (es. bug prima della Fase 2 MCP), ricostruiamo qui il payload
|
# nested payload and retrying validation, only for session.update.
|
||||||
# corretto e riproviamo la validazione. Funziona SOLO per session.update.
|
|
||||||
if event_type == "session.update":
|
if event_type == "session.update":
|
||||||
try:
|
try:
|
||||||
raw_dict = dict(raw) if not isinstance(raw, dict) else raw
|
raw_dict = dict(raw) if not isinstance(raw, dict) else raw
|
||||||
@@ -290,7 +289,7 @@ class RealtimeService:
|
|||||||
normalized_inner = dict(session_inner)
|
normalized_inner = dict(session_inner)
|
||||||
else:
|
else:
|
||||||
normalized_inner = {}
|
normalized_inner = {}
|
||||||
# Promuovi campi noti dalla root → inner se mancanti (fallback flat)
|
# Promote known root fields into the inner session when missing.
|
||||||
SESSION_KNOWN_TOP_LEVEL = (
|
SESSION_KNOWN_TOP_LEVEL = (
|
||||||
"instructions",
|
"instructions",
|
||||||
"tools",
|
"tools",
|
||||||
|
|||||||
@@ -14,6 +14,6 @@ class ChatTTSHandlerArguments:
|
|||||||
chat_tts_chunk_size: int = field(
|
chat_tts_chunk_size: int = field(
|
||||||
default=512,
|
default=512,
|
||||||
metadata={
|
metadata={
|
||||||
"help": "Sets the size of the audio data chunk processed per cycle, balancing playback latency and CPU load.. Default is 512。."
|
"help": "Sets the size of the audio data chunk processed per cycle, balancing playback latency and CPU load. Default is 512."
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|||||||
+129
-19
@@ -2,44 +2,154 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
VENV_DIR="$ROOT_DIR/.venv"
|
API_DIR="$ROOT_DIR/vocero-api"
|
||||||
REQ_FILE="$ROOT_DIR/s2s-client/requirements.txt"
|
VENV_DIR="$API_DIR/.venv"
|
||||||
UI_PORT="${UI_PORT:-8420}"
|
FRONTEND_DIR="$ROOT_DIR/vocero-frontend"
|
||||||
if [ -f "$ROOT_DIR/.env" ]; then
|
if [ -f "$ROOT_DIR/.env" ]; then
|
||||||
set -a
|
set -a
|
||||||
source "$ROOT_DIR/.env"
|
source "$ROOT_DIR/.env"
|
||||||
set +a
|
set +a
|
||||||
fi
|
fi
|
||||||
|
UI_PORT="${UI_PORT:-8420}"
|
||||||
|
UI_HOST="${UI_HOST:-0.0.0.0}"
|
||||||
|
S2S_BACKEND_PORT="${S2S_BACKEND_PORT:-8421}"
|
||||||
S2S_PROXY_TARGET="${S2S_PROXY_TARGET:-ws://ai-server:12345/v1/realtime}"
|
S2S_PROXY_TARGET="${S2S_PROXY_TARGET:-ws://ai-server:12345/v1/realtime}"
|
||||||
|
|
||||||
if ! command -v python3 >/dev/null 2>&1; then
|
if (( BASH_VERSINFO[0] < 4 || (BASH_VERSINFO[0] == 4 && BASH_VERSINFO[1] < 4) )); then
|
||||||
echo "Errore: python3 non trovato."
|
echo "Error: start_ui.sh requires Bash 4.4 or newer."
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
if [ ! -d "$VENV_DIR" ]; then
|
for required_command in uv node npm setsid curl; do
|
||||||
echo "Creazione ambiente virtuale in .venv..."
|
if ! command -v "$required_command" >/dev/null 2>&1; then
|
||||||
python3 -m venv "$VENV_DIR"
|
echo "Error: $required_command was not found."
|
||||||
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
source "$VENV_DIR/bin/activate"
|
if ! node -e 'const [major, minor] = process.versions.node.split(".").map(Number); process.exit((major === 22 && minor >= 18) || major >= 24 ? 0 : 1)'; then
|
||||||
|
echo "Error: vocero-frontend requires Node.js 22.18+ on the 22.x line, or Node.js 24+."
|
||||||
if [ ! -f "$REQ_FILE" ]; then
|
|
||||||
echo "Errore: file requirements non trovato: $REQ_FILE"
|
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
if ! python -c "import fastapi, uvicorn, httpx, huggingface_hub, dotenv" >/dev/null 2>&1; then
|
for port_name in UI_PORT S2S_BACKEND_PORT; do
|
||||||
echo "Installazione dipendenze da s2s-client/requirements.txt..."
|
port_value="${!port_name}"
|
||||||
python -m pip install -r "$REQ_FILE"
|
if [[ ! "$port_value" =~ ^[0-9]{1,5}$ ]] || (( 10#$port_value < 1 || 10#$port_value > 65535 )); then
|
||||||
|
echo "Error: $port_name must be a port between 1 and 65535."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
# Normalize leading zeroes before passing ports to Node and Uvicorn.
|
||||||
|
printf -v "$port_name" '%d' "$((10#$port_value))"
|
||||||
|
done
|
||||||
|
if [ "$UI_PORT" = "$S2S_BACKEND_PORT" ]; then
|
||||||
|
echo "Error: UI_PORT and S2S_BACKEND_PORT must be different."
|
||||||
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
if [ ! -f "$API_DIR/pyproject.toml" ] || [ ! -f "$API_DIR/uv.lock" ]; then
|
||||||
|
echo "Error: vocero-api/pyproject.toml and vocero-api/uv.lock are required."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# uv manages only the API environment; an existing root speech venv is separate.
|
||||||
|
UV_PROJECT_ENVIRONMENT="$VENV_DIR" uv sync --project "$API_DIR" --locked
|
||||||
|
|
||||||
|
# Keep restarts offline when the installed frontend manifests have not changed.
|
||||||
|
if [ ! -f "$FRONTEND_DIR/node_modules/vite/bin/vite.js" ] || \
|
||||||
|
! cmp -s "$FRONTEND_DIR/package-lock.json" "$FRONTEND_DIR/node_modules/.s2s-package-lock.json" || \
|
||||||
|
! cmp -s "$FRONTEND_DIR/package.json" "$FRONTEND_DIR/node_modules/.s2s-package.json"; then
|
||||||
|
echo "Installing frontend dependencies from vocero-frontend/package-lock.json..."
|
||||||
|
npm --prefix "$FRONTEND_DIR" ci --include=dev
|
||||||
|
cp "$FRONTEND_DIR/package-lock.json" "$FRONTEND_DIR/node_modules/.s2s-package-lock.json"
|
||||||
|
cp "$FRONTEND_DIR/package.json" "$FRONTEND_DIR/node_modules/.s2s-package.json"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# The combined launcher always proxies to the backend it starts. An alternate
|
||||||
|
# S2S_BACKEND_URL is only used when running Vite independently.
|
||||||
|
export S2S_BACKEND_URL="http://127.0.0.1:$S2S_BACKEND_PORT"
|
||||||
export S2S_PROXY_TARGET
|
export S2S_PROXY_TARGET
|
||||||
|
|
||||||
echo "💻 Avvio dell'interfaccia S2S Client UI (Porta $UI_PORT)..."
|
child_pids=()
|
||||||
echo "🔌 Proxy realtime verso $S2S_PROXY_TARGET"
|
cleanup() {
|
||||||
echo " Premi Ctrl+C in questo terminale per fermarla."
|
local exit_status=$?
|
||||||
|
trap - EXIT
|
||||||
|
trap '' INT TERM HUP
|
||||||
|
for child_pid in "${child_pids[@]}"; do
|
||||||
|
kill -TERM -- "-$child_pid" 2>/dev/null || true
|
||||||
|
done
|
||||||
|
# Stop whole process groups, including Uvicorn's reload worker. Bound the
|
||||||
|
# graceful shutdown so a stuck connection cannot leave ports occupied.
|
||||||
|
for ((attempt = 0; attempt < 20; attempt++)); do
|
||||||
|
local alive=false
|
||||||
|
for child_pid in "${child_pids[@]}"; do
|
||||||
|
if kill -0 -- "-$child_pid" 2>/dev/null; then
|
||||||
|
alive=true
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ "$alive" = false ]; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 0.05
|
||||||
|
done
|
||||||
|
for child_pid in "${child_pids[@]}"; do
|
||||||
|
kill -KILL -- "-$child_pid" 2>/dev/null || true
|
||||||
|
wait "$child_pid" 2>/dev/null || true
|
||||||
|
done
|
||||||
|
exit "$exit_status"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
trap 'exit 130' INT
|
||||||
|
trap 'exit 143' TERM
|
||||||
|
trap 'exit 129' HUP
|
||||||
|
|
||||||
|
echo "💻 Starting the React UI at http://$UI_HOST:$UI_PORT (Vite hot reload)..."
|
||||||
|
echo "🔧 FastAPI backend: $S2S_BACKEND_URL (Python reload enabled)"
|
||||||
|
echo "🔌 Realtime proxy target: $S2S_PROXY_TARGET"
|
||||||
|
echo " Press Ctrl+C in this terminal to stop both servers."
|
||||||
echo "----------------------------------------------------"
|
echo "----------------------------------------------------"
|
||||||
|
|
||||||
cd "$ROOT_DIR"
|
cd "$ROOT_DIR"
|
||||||
PYTHONPATH=s2s-client python -m uvicorn server:app --host 0.0.0.0 --port "$UI_PORT" --reload
|
setsid "$VENV_DIR/bin/python" -m uvicorn vocero_api.main:app \
|
||||||
|
--app-dir "$API_DIR" \
|
||||||
|
--host 127.0.0.1 --port "$S2S_BACKEND_PORT" \
|
||||||
|
--reload --reload-dir "$API_DIR" &
|
||||||
|
backend_pid=$!
|
||||||
|
child_pids+=("$backend_pid")
|
||||||
|
|
||||||
|
backend_ready=false
|
||||||
|
startup_deadline=$((SECONDS + 30))
|
||||||
|
while (( SECONDS < startup_deadline )); do
|
||||||
|
if ! kill -0 "$backend_pid" 2>/dev/null; then
|
||||||
|
echo "Error: FastAPI exited before it was ready."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if curl --noproxy '*' --silent --fail --max-time 1 \
|
||||||
|
--output /dev/null "$S2S_BACKEND_URL/api/config"; then
|
||||||
|
backend_ready=true
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 0.2
|
||||||
|
done
|
||||||
|
if [ "$backend_ready" = false ]; then
|
||||||
|
echo "Error: FastAPI did not become ready within 30 seconds."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
(
|
||||||
|
cd "$FRONTEND_DIR"
|
||||||
|
# Keep React Fast Refresh enabled even if PM2 exports NODE_ENV=production.
|
||||||
|
export NODE_ENV=development
|
||||||
|
exec setsid node node_modules/vite/bin/vite.js --host "$UI_HOST" --port "$UI_PORT" --strictPort
|
||||||
|
) &
|
||||||
|
frontend_pid=$!
|
||||||
|
child_pids+=("$frontend_pid")
|
||||||
|
|
||||||
|
# PM2 supervises this script. If either server exits, stop its sibling and fail
|
||||||
|
# the launcher so PM2 can restart the complete pair instead of a partial UI.
|
||||||
|
exit_status=0
|
||||||
|
wait -n "$backend_pid" "$frontend_pid" || exit_status=$?
|
||||||
|
echo "Error: a UI server exited; stopping both servers."
|
||||||
|
if [ "$exit_status" -eq 0 ]; then
|
||||||
|
exit_status=1
|
||||||
|
fi
|
||||||
|
exit "$exit_status"
|
||||||
|
|||||||
@@ -6,7 +6,7 @@
|
|||||||
.idea/
|
.idea/
|
||||||
*.swp
|
*.swp
|
||||||
|
|
||||||
# Static HTTP server logs (we don't ship them)
|
# Server logs
|
||||||
*.log
|
*.log
|
||||||
|
|
||||||
# Python
|
# Python
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
minimum_pre_commit_version: "4.4.0"
|
||||||
|
default_install_hook_types: [pre-commit]
|
||||||
|
default_stages: [pre-commit]
|
||||||
|
|
||||||
|
repos:
|
||||||
|
- repo: local
|
||||||
|
hooks:
|
||||||
|
- id: ruff-check
|
||||||
|
name: Ruff lint
|
||||||
|
entry: uv run --project vocero-api --locked ruff check --force-exclude
|
||||||
|
language: unsupported
|
||||||
|
types_or: [python, pyi]
|
||||||
|
files: ^vocero-api/
|
||||||
|
require_serial: true
|
||||||
|
- id: ruff-format
|
||||||
|
name: Ruff formatting
|
||||||
|
entry: uv run --project vocero-api --locked ruff format --check --force-exclude
|
||||||
|
language: unsupported
|
||||||
|
types_or: [python, pyi]
|
||||||
|
files: ^vocero-api/
|
||||||
|
require_serial: true
|
||||||
|
- id: mypy
|
||||||
|
name: Python types
|
||||||
|
entry: uv run --directory vocero-api --locked mypy
|
||||||
|
language: unsupported
|
||||||
|
pass_filenames: false
|
||||||
|
files: ^vocero-api/(vocero_api/|server\.py|pyproject\.toml|uv\.lock)
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
3.12
|
||||||
@@ -0,0 +1,309 @@
|
|||||||
|
# Changelog — vocero-api (Azurian Vocero)
|
||||||
|
|
||||||
|
All significant client changes are recorded here. This English history
|
||||||
|
supersedes the former Italian and Spanish versions. Historical entries describe
|
||||||
|
the implementation at the recorded date; current operating instructions live in
|
||||||
|
[the project documentation](../docs/README.md).
|
||||||
|
|
||||||
|
## Unreleased
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Added authenticated report and episode link context, explicit conversation subject
|
||||||
|
selection, and report-title greetings in TALKFIRST. Searches and comparisons remain
|
||||||
|
unrestricted by the link; missing content does not select an unrelated podcast.
|
||||||
|
- Added optional cookie-bound Vocero conversations that keep each caller's AquaPortal
|
||||||
|
session on the server, with expiry, cleanup, and isolated MCP discovery and execution.
|
||||||
|
- Added bounded MCP tool loops to HTTP reasoning and the voice relay, preserving full
|
||||||
|
input schemas, structured results, source references, and conversation follow-ups.
|
||||||
|
Cancelled turns discard late results; the voice relay suppresses internal tool text
|
||||||
|
and audio without changing the speech pipeline.
|
||||||
|
- Added browser conversation bootstrap and cleanup, authenticated discovery, per-turn
|
||||||
|
reasoning headers, and tool progress states. MCP mode blocks failed login and skips
|
||||||
|
legacy unscoped content preloads; existing behavior remains available when disabled.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Consolidate responsive transcript layout, keep export controls visible on narrow screens,
|
||||||
|
and place feedback and version details below the conversation. Add jump-to-latest,
|
||||||
|
semantic message metadata, state-aware empty hints, shared sanitized Markdown rendering,
|
||||||
|
and reduced-motion support. Remove retired camera, account, and overlay styles.
|
||||||
|
|
||||||
|
- Use native dialogs for startup and mobile reports, restore focus on dismissal,
|
||||||
|
and make duplicate transcript bubbles inert. Require explicit wake-word opt-in,
|
||||||
|
show when it is listening, and turn it off when the conversation stops.
|
||||||
|
|
||||||
|
- Isolate microphone meter subscriptions from the application tree. Apply draft settings only
|
||||||
|
on Save, preserve explicitly live noise calibration, and persist/send only relevant changes.
|
||||||
|
|
||||||
|
- Keep the voice UI speaking until buffered playback finishes, flush short final audio chunks,
|
||||||
|
and exclude response processing from the user inactivity timeout.
|
||||||
|
|
||||||
|
- Make frontend configuration failures retryable, bound startup requests, and
|
||||||
|
load optional reports independently while waiting for caller context before
|
||||||
|
starting a session. Cancel superseded external reasoning and reject stale
|
||||||
|
answers after a new turn, interruption, or session teardown.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Removed the ApiPilot API bridge, configuration, credential overrides, browser
|
||||||
|
catalog/prompt handling, and two-pass HTTP/realtime execution. AquaPortal tools
|
||||||
|
use authenticated MCP; the separate speech pipeline retains its existing
|
||||||
|
marker sanitizers.
|
||||||
|
|
||||||
|
- Use the AquaPortal admin favicon in the Vocero browser frontend.
|
||||||
|
|
||||||
|
- Start `TALK` and `TALKFIRST` conversations as soon as configuration and caller
|
||||||
|
context are ready. Remove the startup dialog and two-second delay; use an
|
||||||
|
inline hint and the existing microphone button when a browser requires a tap.
|
||||||
|
|
||||||
|
- Replace the frontend legacy stylesheet with Tailwind v4 semantic theme tokens,
|
||||||
|
shared form/dialog controls and component-owned layout utilities. Preserve both
|
||||||
|
configured themes, responsive layouts, sanitized Markdown and audio-driven
|
||||||
|
visuals in focused stylesheets, and remove unused historical selectors.
|
||||||
|
|
||||||
|
- Renamed the API and browser projects to `vocero-api/` and `vocero-frontend/`.
|
||||||
|
Python uses the `vocero-api` distribution and `vocero_api` import package;
|
||||||
|
npm uses `vocero-frontend`. Updated the launcher, Docker setup, tooling,
|
||||||
|
tests, and documentation while preserving HTTP/WebSocket contracts and
|
||||||
|
existing environment variables, browser preferences, and PM2 process names.
|
||||||
|
- Removed the optional workflow reasoning service, its configuration, transcript
|
||||||
|
forwarding, and browser integration. Reasoning supports `PROXY` and `OPENAI`;
|
||||||
|
unsupported mode values fall back to `PROXY`.
|
||||||
|
- Filter the specific Starlette TestClient / AnyIO `BlockingPortal` deprecation
|
||||||
|
in pytest while retaining the runtime versions used for compatibility captures.
|
||||||
|
- Added a uv-managed Python 3.12 environment and Ruff, mypy, and pytest quality
|
||||||
|
checks, all contained in `vocero-api/`. Startup and Docker use the committed
|
||||||
|
lockfile. Split the FastAPI service into configuration, route, provider,
|
||||||
|
context/reasoning, and realtime modules while retaining `server:app` startup.
|
||||||
|
- Added Pydantic response contracts with permissive upstream fields and original
|
||||||
|
payload serialization. Captured HTTP, request-schema, and realtime regression
|
||||||
|
fixtures preserve existing API behavior, including known failures.
|
||||||
|
- Removed the legacy browser site in favor of the React application in
|
||||||
|
`vocero-frontend`. FastAPI now serves only backend and documentation routes.
|
||||||
|
- Retired daily talk-time accounting and the associated Hugging Face account
|
||||||
|
UI. Load-balancer allocation, queue lifecycle, caller profiles, upstream
|
||||||
|
authentication, and voice-session cleanup remain available.
|
||||||
|
- Consolidated technical documentation under the root `docs/` directory and
|
||||||
|
replaced translated changelog copies with this English history.
|
||||||
|
- Moved the agent-profile and reference-audio examples into `docs/examples/`.
|
||||||
|
- Moved the historical architecture proposal into `docs/proposals/` and retained
|
||||||
|
the existing document URLs as redirects.
|
||||||
|
- Removed the obsolete `index_video.html` page, which no longer matched the
|
||||||
|
elements required by the shared client script.
|
||||||
|
- Translated developer comments, diagnostics, internal model instructions, and
|
||||||
|
local identifiers into English. Sanitization accepts both the former Spanish
|
||||||
|
and new English internal tool-result wrappers.
|
||||||
|
- Standardized browser labels, accessibility text, account/profile dialogs,
|
||||||
|
session feedback, and built-in API errors in Spanish. The interface declares
|
||||||
|
`lang="es"`; internal instructions and diagnostics remain English.
|
||||||
|
|
||||||
|
## 2026-09-25
|
||||||
|
|
||||||
|
### Fixed — ApiPilot tool discovery in realtime sessions
|
||||||
|
|
||||||
|
- Fetch the tool catalog at the start of each realtime WebSocket connection,
|
||||||
|
before opening the upstream connection. Previously the catalog was injected
|
||||||
|
only through HTTP reasoning, leaving voice sessions without it.
|
||||||
|
- Inject ApiPilot independently of `S2S_INJECT_PROFILE`, including sessions with
|
||||||
|
profile injection disabled.
|
||||||
|
- Cache discovery using the authenticated identity or an anonymous session key.
|
||||||
|
Save the tool definitions and wrapped context in the session context and reuse
|
||||||
|
them when patching later `session.update` messages.
|
||||||
|
- Wrap the catalog with `APIPILOT_TOOLS_LIST_BEGIN/END` markers and include it
|
||||||
|
after the other context blocks. Record catalog readiness, count, timing,
|
||||||
|
authentication state, and injection length; full prompt dumps remain controlled
|
||||||
|
by `S2S_DEBUG_PROMPT`.
|
||||||
|
|
||||||
|
### Fixed — Two-pass execution, authentication, and marker handling
|
||||||
|
|
||||||
|
- Strengthen the first-pass prompt so a tool selection emits exactly one CALL
|
||||||
|
marker line, without a spoken preamble or trailing explanation.
|
||||||
|
- Replace exposed CALL markers in chat with a Spanish waiting message.
|
||||||
|
- Detect and execute CALL markers before sanitizing them. Sanitizing first had
|
||||||
|
erased the signal required to start the second pass.
|
||||||
|
- Correct the monotonic-clock helper used by second-pass processing.
|
||||||
|
- Reject illustrative phone values from model output and fill missing values
|
||||||
|
from the actual session context.
|
||||||
|
- Accept empty arguments and normalize malformed argument representations using
|
||||||
|
JSON, single-quoted mappings, key/value pairs, missing braces, or empty input.
|
||||||
|
- Initialize default arguments on all discovery and exception paths.
|
||||||
|
- Normalize execute endpoints, supporting an explicit execute URL and avoiding
|
||||||
|
an incorrect `/tools/execute` route when the backend expects `/execute`.
|
||||||
|
- Send empty arguments for authentication-only tools whose data is derived from
|
||||||
|
the session token, avoiding conflicts with a separate phone argument.
|
||||||
|
- Reuse the discovery authorization and demo-session identity for execution.
|
||||||
|
- Add the configured API-key override for deployments using that authentication
|
||||||
|
path, ahead of the service-token and session-token paths. The historical
|
||||||
|
override sent `X-API-Key`, `Authorization: Bearer`, and `x-demo-session`
|
||||||
|
together for discovery and execution, and bypassed phone login entirely.
|
||||||
|
Documentation uses placeholders rather than retaining the credential from
|
||||||
|
the old history.
|
||||||
|
- Add diagnostics for discovery, execution, schema validation, login-token
|
||||||
|
shape, default arguments, and upstream HTTP errors.
|
||||||
|
|
||||||
|
The historical authentication priority was: configured API key, configured
|
||||||
|
service token, discovery authorization, forced phone login when required, and
|
||||||
|
anonymous access to tools that do not require authentication. This records the change rather than
|
||||||
|
recommending a credential or deployment configuration.
|
||||||
|
|
||||||
|
### Fixed — Prevent internal markers from reaching speech
|
||||||
|
|
||||||
|
- Sanitize CALL and RESULT markers in the relay, LLM output processor, and TTS
|
||||||
|
handler, including truncated, inverted, and partially wrapped forms.
|
||||||
|
- Use the Spanish spoken placeholder
|
||||||
|
`Voy a consultar tus datos, un momento por favor.` for pending calls.
|
||||||
|
- Add phone-context fallback and prompt rules preventing internal marker syntax
|
||||||
|
from being spoken. Suppress the first-pass audio while preserving the second
|
||||||
|
pass that contains the actual tool result.
|
||||||
|
- Remove leaked Unix, Windows, and TRAE workspace paths. Preserve ordinary text
|
||||||
|
containing phone numbers when no internal marker context is present.
|
||||||
|
- The original offline sanitizer check passed 7 of 11 cases, with four
|
||||||
|
artificial edge cases still failing and end-to-end voice verification pending.
|
||||||
|
This is historical verification, not a claim about later changes.
|
||||||
|
|
||||||
|
## 2026-09-24
|
||||||
|
|
||||||
|
### Added — ApiPilot reasoning in two passes
|
||||||
|
|
||||||
|
- Discover tools dynamically and present their names and descriptions to the
|
||||||
|
reasoning model. A matching request produces
|
||||||
|
`<<<APIPILOT_CALL:NAME=tool_name,ARGUMENTS={...}>>>`.
|
||||||
|
- Parse the marker, execute the selected tool, and return its text to the model
|
||||||
|
in a second reasoning request. The model then produces the spoken answer;
|
||||||
|
an empty second-pass answer falls back to the raw tool-result text.
|
||||||
|
- Support `S2S_APIPILOT_TOOLS_EXECUTE_URL` as an explicit execute-endpoint
|
||||||
|
override. Normalize the configured endpoint instead of hardcoding a backend
|
||||||
|
route.
|
||||||
|
- Supply known caller-context values to argument construction and use bounded
|
||||||
|
catalog/context text rather than exposing authentication material to the model.
|
||||||
|
|
||||||
|
### Added — Browser fetch and WebSocket tracing
|
||||||
|
|
||||||
|
- Trace fetch calls with semantic endpoint tags, method, URL, timing, response
|
||||||
|
status, and bounded body/response previews. Distinguish configuration,
|
||||||
|
reasoning, profile, reports, search, ApiPilot, queue, and session operations.
|
||||||
|
- Trace WebSocket connection, handshake, close, session, and tool events while
|
||||||
|
limiting noisy audio-frame logs and masking secret URL parameters.
|
||||||
|
- Prefetch ApiPilot tools at every realtime connection in parallel with audio
|
||||||
|
and WebSocket setup. Connection setup awaits these operations while isolating
|
||||||
|
discovery failures. Cache per user, invalidate on identity changes, and
|
||||||
|
refresh after successful phone login.
|
||||||
|
- Provide the browser demo-discovery path and server-side discovery logs so
|
||||||
|
operators can follow both halves of a proxied request.
|
||||||
|
|
||||||
|
### Fixed — Discovery refresh after login
|
||||||
|
|
||||||
|
- Refresh the tool catalog after successful phone authentication, rather than
|
||||||
|
retaining the catalog fetched for the anonymous identity.
|
||||||
|
- Store client discovery state per identity and expose cache invalidation for
|
||||||
|
identity changes and explicit refresh requests.
|
||||||
|
|
||||||
|
### Added — Dynamic ApiPilot proxy
|
||||||
|
|
||||||
|
- Add `GET /api/apipilot/tools` and `POST /api/apipilot/execute` with a consistent
|
||||||
|
`{success, data, error}` envelope.
|
||||||
|
- Configure discovery URL, service-token override, cache TTL, and timeout through
|
||||||
|
environment variables. Cache by authorization/demo-session fingerprint and
|
||||||
|
allow a logged stale-cache fallback when discovery is temporarily unavailable.
|
||||||
|
- Re-discover missing tools once before returning a not-found response, enforce
|
||||||
|
`requires_auth`, validate required arguments, and strip unknown arguments when
|
||||||
|
the schema is closed. Tools with an open schema retain flexible arguments.
|
||||||
|
- Present only the minimal tool definition to the model. Return user-facing
|
||||||
|
errors in Spanish and keep error codes/status/raw details available for
|
||||||
|
diagnostics.
|
||||||
|
- Expose ApiPilot availability through `/api/config` and record configuration
|
||||||
|
availability in the startup banner without printing token values.
|
||||||
|
|
||||||
|
## 2026-09-23
|
||||||
|
|
||||||
|
### Added — Conversation downloads
|
||||||
|
|
||||||
|
- Add the `Descargar conversación` action to the chat header.
|
||||||
|
- Export JSON and text entirely in the browser, including user/assistant
|
||||||
|
messages, tool calls/results, timestamps, display names, and snapshot metadata.
|
||||||
|
- Preserve initials, interruption flags, and snapshot image `dataUrl` values.
|
||||||
|
Export persistent conversation history, excluding ephemeral orb bubbles.
|
||||||
|
- Include a schema version, export timestamp, message count, and conversation
|
||||||
|
entries in the JSON export. Build the text export from the same conversation
|
||||||
|
entries so both formats agree.
|
||||||
|
- Download through temporary Blob URLs, revoke those URLs afterward, and show
|
||||||
|
success or empty-conversation feedback. No server export endpoint is required.
|
||||||
|
|
||||||
|
### Added — Static latest-summary override
|
||||||
|
|
||||||
|
- Add `S2S_LATEST_SUMMARY_TEXT`, with escaped newline handling, ahead of the
|
||||||
|
`S2S_LATEST_SUMMARY` HTTP source.
|
||||||
|
- Accept the static summary without a phone number and apply it in both HTTP
|
||||||
|
reasoning and realtime session context. Preserve the summary wrapper and cap.
|
||||||
|
- Include override availability in startup/context diagnostics and update the
|
||||||
|
injection guide.
|
||||||
|
|
||||||
|
## 2026-09-07
|
||||||
|
|
||||||
|
### Added — Podcast episode context
|
||||||
|
|
||||||
|
- Read `episode_id` from the page query and pass it to realtime and HTTP
|
||||||
|
reasoning requests.
|
||||||
|
- Fetch the selected podcast episode independently of phone context, normalize
|
||||||
|
dictionary/list/string responses, and wrap the serialized episode with
|
||||||
|
`PODCAST_EPISODE_BEGIN/END` markers.
|
||||||
|
- Cap episode context at 24,000 characters and prioritize it before caller
|
||||||
|
profile, scheduled reports, and latest-summary context.
|
||||||
|
- Store episode context at WebSocket connection and patch subsequent client
|
||||||
|
`session.update` messages. Skip unavailable context without aborting the call.
|
||||||
|
|
||||||
|
### Added — Session-start modes
|
||||||
|
|
||||||
|
- Keep manual orb activation as the default.
|
||||||
|
- Add `S2S_START_SESSION_MODE=TALK`, which waits for the microphone/audio unlock
|
||||||
|
gesture and greets after session configuration.
|
||||||
|
- Add `TALKFIRST`, which attempts automatic activation and falls back to the
|
||||||
|
unlock overlay when browser permissions require a gesture.
|
||||||
|
- Delay automatic activation by two seconds. Use the existing `hola vocero`
|
||||||
|
greeting path rather than a separate greeting protocol.
|
||||||
|
|
||||||
|
### Changed — Debug-only reports controls
|
||||||
|
|
||||||
|
- Gate the reports-sidebar toggle and suggestions behind `?debug=true`.
|
||||||
|
- Apply the visibility gate in HTML, CSS, and client lifecycle handling so
|
||||||
|
controls do not flash into view before configuration loads.
|
||||||
|
|
||||||
|
## 2026-09-04
|
||||||
|
|
||||||
|
### Changed — Inline conversation and visual design
|
||||||
|
|
||||||
|
- Replace the overlay chat panel with a permanently visible inline conversation
|
||||||
|
below the voice stage and above the sticky footer; remove the chat toggle.
|
||||||
|
- Introduce the historical aurora background and glass surfaces using shared
|
||||||
|
violet, pink, and cyan brand tokens, including the animated voice orb.
|
||||||
|
- Make the chat surface continuous with the page, distinguish message roles,
|
||||||
|
and maintain safe spacing from viewport edges and the footer.
|
||||||
|
- Put reports in a 320-pixel desktop sidebar and an off-canvas mobile panel.
|
||||||
|
- Use safe viewport units and responsive layouts to improve mobile stability.
|
||||||
|
- Keep the text-entry handler connected while allowing the input row to be
|
||||||
|
hidden in voice-only layouts.
|
||||||
|
|
||||||
|
### Fixed — Chat scrolling and optional controls
|
||||||
|
|
||||||
|
- Schedule repeated scrolling during token streaming and after response/tool
|
||||||
|
completion to account for delayed line wrapping and layout changes.
|
||||||
|
- Disable scroll anchoring/smooth scrolling and reserve bottom space so the last
|
||||||
|
assistant line remains visible above the footer.
|
||||||
|
- Make removed chat-toggle, badge, backdrop, and close-button references optional
|
||||||
|
while keeping required chat/history elements strict.
|
||||||
|
- Synchronize stylesheet/script/import version query parameters and add static
|
||||||
|
no-cache handling for development updates.
|
||||||
|
|
||||||
|
### Added — Caller avatar and profile dialog
|
||||||
|
|
||||||
|
- Replace the header name with a circular initials avatar.
|
||||||
|
- Open a profile dialog showing contact details, account identifiers, role,
|
||||||
|
organization, locale, and available creation/login timestamps.
|
||||||
|
- Accept nested profile payloads, hide empty fields, support `mailto:`/`tel:`
|
||||||
|
links, and close through Escape, backdrop interaction, or the close control.
|
||||||
|
|
||||||
|
Historical context limits were 24,000 characters for an episode, 1,600 each for
|
||||||
|
profile/summary, and 12,000 for a full reports list. Context behavior has evolved;
|
||||||
|
consult [context injection](../docs/client/context-injection.md) for the current
|
||||||
|
configuration and ordering.
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
# Build from the project root: docker build -f vocero-api/Dockerfile -t vocero-api .
|
||||||
|
# Serve backend APIs, the WebSocket proxy, and shared project documentation.
|
||||||
|
# Run the React/Vite frontend separately; this image contains no browser UI.
|
||||||
|
# HF Spaces (sdk: docker) routes traffic to $PORT, default 7860.
|
||||||
|
FROM python:3.12.14-slim
|
||||||
|
|
||||||
|
COPY --from=ghcr.io/astral-sh/uv:0.12.5 /uv /uvx /bin/
|
||||||
|
|
||||||
|
ENV UV_LINK_MODE=copy \
|
||||||
|
UV_PYTHON_DOWNLOADS=never \
|
||||||
|
PATH="/app/vocero-api/.venv/bin:$PATH"
|
||||||
|
|
||||||
|
WORKDIR /app/vocero-api
|
||||||
|
|
||||||
|
COPY vocero-api/pyproject.toml vocero-api/uv.lock ./
|
||||||
|
RUN uv sync --locked --no-dev --no-install-project
|
||||||
|
|
||||||
|
COPY vocero-api/README.md vocero-api/server.py ./
|
||||||
|
COPY vocero-api/vocero_api/ ./vocero_api/
|
||||||
|
COPY docs/ /app/docs/
|
||||||
|
RUN uv sync --locked --no-dev
|
||||||
|
|
||||||
|
EXPOSE 7860
|
||||||
|
|
||||||
|
CMD ["uvicorn", "vocero_api.main:app", "--host", "0.0.0.0", "--port", "7860"]
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
UV ?= uv
|
||||||
|
UV_RUN = $(UV) run --locked
|
||||||
|
|
||||||
|
.PHONY: help install lock lock-check format format-check lint lint-fix type-check test check dev hooks-install hooks-run test-launcher test-image
|
||||||
|
|
||||||
|
help:
|
||||||
|
@printf '%s\n' \
|
||||||
|
'install Sync the uv environment, including development tools' \
|
||||||
|
'lock Resolve dependencies into uv.lock' \
|
||||||
|
'format Apply Ruff fixes and formatting' \
|
||||||
|
'check Check lock freshness, formatting, lint, types, and backend tests' \
|
||||||
|
'dev Start the existing React/FastAPI development launcher' \
|
||||||
|
'hooks-install Install optional local pre-commit checks' \
|
||||||
|
'test-launcher Exercise the Bash supervisor with local fixture processes' \
|
||||||
|
'test-image Build and smoke-test the backend Docker image'
|
||||||
|
|
||||||
|
install:
|
||||||
|
$(UV) sync --locked
|
||||||
|
|
||||||
|
lock:
|
||||||
|
$(UV) lock
|
||||||
|
|
||||||
|
lock-check:
|
||||||
|
$(UV) lock --check
|
||||||
|
|
||||||
|
format:
|
||||||
|
$(UV_RUN) ruff check --fix-only .
|
||||||
|
$(UV_RUN) ruff format .
|
||||||
|
|
||||||
|
format-check:
|
||||||
|
$(UV_RUN) ruff format --check .
|
||||||
|
|
||||||
|
lint:
|
||||||
|
$(UV_RUN) ruff check .
|
||||||
|
|
||||||
|
lint-fix:
|
||||||
|
$(UV_RUN) ruff check --fix .
|
||||||
|
|
||||||
|
type-check:
|
||||||
|
$(UV_RUN) mypy
|
||||||
|
|
||||||
|
test:
|
||||||
|
$(UV_RUN) pytest
|
||||||
|
|
||||||
|
check: lock-check format-check lint type-check test
|
||||||
|
|
||||||
|
dev:
|
||||||
|
../start_ui.sh
|
||||||
|
|
||||||
|
hooks-install:
|
||||||
|
$(UV_RUN) pre-commit install --install-hooks --config vocero-api/.pre-commit-config.yaml
|
||||||
|
|
||||||
|
hooks-run:
|
||||||
|
$(UV_RUN) pre-commit run --all-files --config vocero-api/.pre-commit-config.yaml
|
||||||
|
|
||||||
|
test-launcher:
|
||||||
|
npm --prefix ../vocero-frontend run test:launcher
|
||||||
|
|
||||||
|
test-image:
|
||||||
|
bash scripts/test_docker_image.sh
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
---
|
||||||
|
title: HF Realtime Voice
|
||||||
|
emoji: 🎙️
|
||||||
|
colorFrom: indigo
|
||||||
|
colorTo: purple
|
||||||
|
sdk: docker
|
||||||
|
app_port: 7860
|
||||||
|
pinned: false
|
||||||
|
short_description: Voice chat over WebSocket against a HF speech-to-speech
|
||||||
|
---
|
||||||
|
|
||||||
|
# Vocero API
|
||||||
|
|
||||||
|
FastAPI APIs and WebSocket relay for the [React voice frontend](../vocero-frontend).
|
||||||
|
This service handles configuration, caller context, upstream authentication,
|
||||||
|
search, reasoning, authenticated AquaPortal MCP tools, and optional load-balancer
|
||||||
|
allocation.
|
||||||
|
|
||||||
|
The root `start_ui.sh` now serves the [React frontend](../vocero-frontend) with
|
||||||
|
Vite on public port `8420`, proxying backend requests here on loopback port
|
||||||
|
`8421`. The browser UI lives entirely in `vocero-frontend`; running FastAPI directly
|
||||||
|
or through Docker exposes only backend routes and shared documentation.
|
||||||
|
|
||||||
|
The complete [usage guide](../docs/client/usage.md) covers local startup,
|
||||||
|
connection modes, Docker, settings, and queue behavior. See the
|
||||||
|
[documentation index](../docs/README.md), [architecture](../docs/client/architecture.md),
|
||||||
|
and [change history](CHANGELOG.md) for implementation details and history.
|
||||||
|
|
||||||
|
Build from the repository root so the container includes the documentation:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker build -f vocero-api/Dockerfile -t vocero-api .
|
||||||
|
```
|
||||||
|
|
||||||
|
The Python project is contained in this directory: `pyproject.toml`, `uv.lock`,
|
||||||
|
`.python-version`, `.venv`, quality tools, and tests. Use uv 0.12.5 and Python
|
||||||
|
3.12 (uv can provision Python). From the repository root:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make -C vocero-api install
|
||||||
|
make -C vocero-api check
|
||||||
|
```
|
||||||
|
|
||||||
|
Tests isolate environment configuration, use HTTP fixtures for allocation and
|
||||||
|
caller authentication, and start a loopback WebSocket service to verify audio
|
||||||
|
frames, keepalive, and teardown. No deployed services or database are needed.
|
||||||
|
|
||||||
|
See [backend development and compatibility](../docs/client/development.md) for
|
||||||
|
formatting, typing, schema policy, optional Git hooks, and deployment checks.
|
||||||
|
`uvicorn server:app --app-dir vocero-api` remains a supported entry point;
|
||||||
|
the implementation lives in the `vocero_api/` package. Its Python distribution
|
||||||
|
name is `vocero-api`.
|
||||||
|
|
||||||
|
FastAPI's API explorer remains at `/docs`. Project documents are served as files below
|
||||||
|
`/docs/client/`, `/docs/speech/`, `/docs/adr/`, `/docs/examples/`, and
|
||||||
|
`/docs/proposals/`. The historical `/project/` and `/public/docs.html` URLs
|
||||||
|
redirect to the retained architecture proposal. Markdown remains source text;
|
||||||
|
documentation directories are not browsable rendered pages. Start from the
|
||||||
|
repository documentation index or request `/docs/README.md` explicitly.
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=68", "wheel"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
|
[project]
|
||||||
|
name = "vocero-api"
|
||||||
|
version = "0.1.0"
|
||||||
|
description = "FastAPI integration and realtime proxy for the Vocero frontend"
|
||||||
|
readme = "README.md"
|
||||||
|
requires-python = ">=3.12,<3.13"
|
||||||
|
dependencies = [
|
||||||
|
"fastapi==0.115.6",
|
||||||
|
"uvicorn[standard]==0.34.0",
|
||||||
|
"httpx==0.28.1",
|
||||||
|
"python-dotenv==1.2.2",
|
||||||
|
"pydantic==2.13.5",
|
||||||
|
"websockets==17.2",
|
||||||
|
"mcp==1.29.0",
|
||||||
|
"jsonschema==4.26.0",
|
||||||
|
]
|
||||||
|
|
||||||
|
[dependency-groups]
|
||||||
|
dev = [
|
||||||
|
"mypy>=1.16,<2",
|
||||||
|
"pre-commit>=4.4,<5",
|
||||||
|
"pytest>=9,<10",
|
||||||
|
"ruff>=0.12,<1",
|
||||||
|
"types-jsonschema>=4.26.0.20261006",
|
||||||
|
]
|
||||||
|
|
||||||
|
[tool.uv]
|
||||||
|
required-version = "==0.12.5"
|
||||||
|
|
||||||
|
[tool.setuptools.packages.find]
|
||||||
|
include = ["vocero_api*"]
|
||||||
|
|
||||||
|
[tool.ruff]
|
||||||
|
target-version = "py312"
|
||||||
|
line-length = 120
|
||||||
|
include = ["pyproject.toml", "vocero_api/**/*.py", "server.py", "tests/**/*.py", "scripts/**/*.py"]
|
||||||
|
|
||||||
|
[tool.ruff.lint]
|
||||||
|
select = ["E4", "E7", "E9", "F", "I", "B", "UP"]
|
||||||
|
ignore = ["UP040", "UP046"]
|
||||||
|
|
||||||
|
[tool.ruff.format]
|
||||||
|
quote-style = "double"
|
||||||
|
indent-style = "space"
|
||||||
|
line-ending = "auto"
|
||||||
|
|
||||||
|
[tool.mypy]
|
||||||
|
python_version = "3.12"
|
||||||
|
plugins = ["pydantic.mypy"]
|
||||||
|
files = ["vocero_api", "server.py"]
|
||||||
|
check_untyped_defs = true
|
||||||
|
disallow_untyped_defs = true
|
||||||
|
no_implicit_optional = true
|
||||||
|
strict_equality = true
|
||||||
|
warn_redundant_casts = true
|
||||||
|
warn_return_any = true
|
||||||
|
warn_unused_configs = true
|
||||||
|
warn_unused_ignores = true
|
||||||
|
show_error_codes = true
|
||||||
|
pretty = true
|
||||||
|
|
||||||
|
[tool.pydantic-mypy]
|
||||||
|
init_typed = true
|
||||||
|
warn_untyped_fields = true
|
||||||
|
|
||||||
|
[tool.pytest.ini_options]
|
||||||
|
testpaths = ["tests"]
|
||||||
|
pythonpath = ["."]
|
||||||
|
# Starlette 0.41.3 uses an alias deprecated by AnyIO 4.15. Keep the runtime
|
||||||
|
# versions used for compatibility captures; remove this test-only exception
|
||||||
|
# when upgrading Starlette. Upstream: https://github.com/Kludex/starlette/issues/3497
|
||||||
|
filterwarnings = [
|
||||||
|
'ignore:^The anyio\.abc\.BlockingPortal alias is deprecated, use anyio\.from_thread\.BlockingPortal instead\.$:DeprecationWarning:^starlette\.testclient$',
|
||||||
|
]
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Build the actual runtime and check HTTP behavior without external services.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
IMAGE="vocero-api-smoke:$(date +%s)-$$"
|
||||||
|
container_id=""
|
||||||
|
cleanup() {
|
||||||
|
if [ -n "$container_id" ]; then
|
||||||
|
docker rm --force "$container_id" >/dev/null
|
||||||
|
fi
|
||||||
|
docker image rm "$IMAGE" >/dev/null 2>&1 || true
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
docker build --file "$ROOT_DIR/vocero-api/Dockerfile" --tag "$IMAGE" "$ROOT_DIR"
|
||||||
|
container_id="$(docker run --detach --publish 127.0.0.1::7860 "$IMAGE")"
|
||||||
|
port="$(docker port "$container_id" 7860/tcp)"
|
||||||
|
ready=false
|
||||||
|
for ((attempt = 0; attempt < 60; attempt++)); do
|
||||||
|
if curl --noproxy '*' --silent --fail --max-time 1 "http://$port/api/config" >/dev/null; then
|
||||||
|
ready=true
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 0.5
|
||||||
|
done
|
||||||
|
if [ "$ready" != true ]; then
|
||||||
|
docker logs "$container_id"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
docker exec -i "$container_id" python - <<'PY'
|
||||||
|
import json
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
base = "http://127.0.0.1:7860"
|
||||||
|
with urllib.request.urlopen(base + "/api/config") as response:
|
||||||
|
assert json.load(response) == {
|
||||||
|
"search": False, "lb": False, "allowDirect": True, "s2sUrl": "",
|
||||||
|
"primaryPrompt": "", "reasoningType": "PROXY", "injectProfile": False,
|
||||||
|
"debugPrompt": False, "startSessionMode": "", "theme": "",
|
||||||
|
}
|
||||||
|
for path in ["/docs", "/docs/README.md", "/openapi.json"]:
|
||||||
|
with urllib.request.urlopen(base + path) as response:
|
||||||
|
assert response.status == 200
|
||||||
|
try:
|
||||||
|
urllib.request.urlopen(base + "/")
|
||||||
|
except urllib.error.HTTPError as error:
|
||||||
|
assert error.code == 404
|
||||||
|
else:
|
||||||
|
raise AssertionError("The backend root must remain 404")
|
||||||
|
print("Docker runtime smoke checks passed")
|
||||||
|
PY
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
"""Compatibility entry point for existing ``uvicorn server:app`` deployments."""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Also support --app-dir /path/to/vocero-api when launched outside the checkout.
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||||
|
|
||||||
|
from vocero_api.main import app as app # noqa: E402
|
||||||
+1011
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,489 @@
|
|||||||
|
"""Golden wire behavior captured before the tooling/structure migration.
|
||||||
|
|
||||||
|
Fixtures include permissive payloads and existing failures deliberately. Changing
|
||||||
|
them requires a separate, explicitly approved API behavior change.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
with patch.dict(os.environ, {}, clear=True), patch("dotenv.load_dotenv", return_value=False):
|
||||||
|
from vocero_api import main as server
|
||||||
|
from vocero_api.core import config as settings
|
||||||
|
from vocero_api.realtime import proxy
|
||||||
|
from vocero_api.services import context
|
||||||
|
|
||||||
|
ASYNC_CLIENT = httpx.AsyncClient
|
||||||
|
FIXTURE = Path(__file__).parent / "fixtures" / "legacy-contracts.json"
|
||||||
|
BASE_CONFIG = {
|
||||||
|
"LOAD_BALANCER_URL": "",
|
||||||
|
"SPEECH_TO_SPEECH_URL": "",
|
||||||
|
"S2S_PROXY_TARGET": "",
|
||||||
|
"S2S_PRIMARY_PROMPT": "",
|
||||||
|
"SERPER_KEY": "",
|
||||||
|
"S2S_REASONING_TYPE": "PROXY",
|
||||||
|
"S2S_INJECT_PROFILE": False,
|
||||||
|
"S2S_INJECT_REPORTS": False,
|
||||||
|
"S2S_DEBUG_PROMPT": False,
|
||||||
|
"S2S_START_SESSION_MODE": "",
|
||||||
|
"S2S_THEME": "",
|
||||||
|
"VOCERO_USER_PHONE_AUTH_URL": "",
|
||||||
|
"S2S_REPORT_LIST": "",
|
||||||
|
"S2S_LOGIN_BY_PHONE": "",
|
||||||
|
"S2S_LATEST_SUMMARY": "",
|
||||||
|
"S2S_LATEST_SUMMARY_TEXT": "",
|
||||||
|
"OPENAI_API_KEY": "",
|
||||||
|
"S2S_REPORTS_QUICK_HINT_MAX": 5,
|
||||||
|
"S2S_REPORTS_INJECT_MODE": "minimal",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def cases():
|
||||||
|
"""Each case controls all upstream IO; no deployment services are contacted."""
|
||||||
|
yield "config-default", "GET", "/api/config", {}, {}, []
|
||||||
|
yield (
|
||||||
|
"config-proxy",
|
||||||
|
"GET",
|
||||||
|
"/api/config",
|
||||||
|
{
|
||||||
|
"headers": {"x-forwarded-proto": "https", "x-forwarded-host": "voice.test, internal.test"},
|
||||||
|
},
|
||||||
|
{"S2S_PROXY_TARGET": "ws://private.test/voice"},
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
yield (
|
||||||
|
"config-direct",
|
||||||
|
"GET",
|
||||||
|
"/api/config",
|
||||||
|
{},
|
||||||
|
{"SPEECH_TO_SPEECH_URL": "wss://voice.test/realtime?token=fixture"},
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
yield "config-lb", "GET", "/api/config", {}, {"LOAD_BALANCER_URL": "http://lb.test"}, []
|
||||||
|
for path in ("/api/me", "/api/session/heartbeat", "/api/session/end", "/", "/server.py"):
|
||||||
|
yield f"removed-{path}", "GET", path, {}, {}, []
|
||||||
|
for path in ("/project", "/project/", "/project/index.html", "/public/docs.html"):
|
||||||
|
yield f"redirect-{path}", "GET", path, {}, {}, []
|
||||||
|
for path in ("/api/session", "/api/queue/end"):
|
||||||
|
yield f"disabled-{path}", "POST", path, {}, {}, []
|
||||||
|
for method in ("GET", "DELETE"):
|
||||||
|
yield f"disabled-queue-{method}", method, "/api/queue/ticket", {}, {}, []
|
||||||
|
|
||||||
|
for name, payload in (
|
||||||
|
("missing", {}),
|
||||||
|
("blank", {"query": " "}),
|
||||||
|
("wrong-type", {"query": 17}),
|
||||||
|
("null", {"query": None}),
|
||||||
|
("unconfigured", {"query": "hello"}),
|
||||||
|
):
|
||||||
|
yield f"search-{name}", "POST", "/api/search", {"json": payload}, {}, []
|
||||||
|
search_config = {"SERPER_KEY": "server-fixture-key"}
|
||||||
|
yield (
|
||||||
|
"search-success-extra-fields",
|
||||||
|
"POST",
|
||||||
|
"/api/search",
|
||||||
|
{
|
||||||
|
"json": {
|
||||||
|
"query": " hello ",
|
||||||
|
"key": "caller-fixture-key",
|
||||||
|
"unknown_request_field": [1, 2],
|
||||||
|
}
|
||||||
|
},
|
||||||
|
search_config,
|
||||||
|
[
|
||||||
|
(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"answerBox": {"answer": "Answer"},
|
||||||
|
"organic": [
|
||||||
|
{"title": "Title", "snippet": "Text", "link": "https://example.test", "unknown": 1},
|
||||||
|
],
|
||||||
|
"future_upstream": {"nested": True},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
yield (
|
||||||
|
"search-loose-values",
|
||||||
|
"POST",
|
||||||
|
"/api/search",
|
||||||
|
{"json": {"query": "hello"}},
|
||||||
|
search_config,
|
||||||
|
[
|
||||||
|
(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"answerBox": {"answer": {"future": [1, None]}},
|
||||||
|
"organic": [{"title": None, "snippet": 3, "link": {"href": "unusual"}}, {}],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
yield (
|
||||||
|
"search-upstream-error",
|
||||||
|
"POST",
|
||||||
|
"/api/search",
|
||||||
|
{"json": {"query": "hello"}},
|
||||||
|
search_config,
|
||||||
|
[(429, {"message": "Fixture credits exhausted"})],
|
||||||
|
)
|
||||||
|
yield (
|
||||||
|
"search-invalid-upstream-json",
|
||||||
|
"POST",
|
||||||
|
"/api/search",
|
||||||
|
{"json": {"query": "hello"}},
|
||||||
|
search_config,
|
||||||
|
[(200, b"not-json")],
|
||||||
|
)
|
||||||
|
|
||||||
|
yield "profile-missing-phone", "GET", "/api/user/phone", {}, {}, []
|
||||||
|
yield "profile-unconfigured", "GET", "/api/user/phone?phoneNumber=123", {}, {}, []
|
||||||
|
profile_config = {"VOCERO_USER_PHONE_AUTH_URL": "http://profile.test/user"}
|
||||||
|
yield "profile-blank-phone", "GET", "/api/user/phone?phoneNumber=%20", {}, profile_config, []
|
||||||
|
for name, status, payload in (
|
||||||
|
("object", 200, {"result": {"name": "Caller", "unknown": {"nested": [1, None]}}}),
|
||||||
|
("array", 200, [1, {"future": "field"}]),
|
||||||
|
("null", 200, None),
|
||||||
|
("string", 200, "opaque profile"),
|
||||||
|
("error", 403, {"future_error": True}),
|
||||||
|
("invalid-json", 200, b"not-json"),
|
||||||
|
):
|
||||||
|
yield (
|
||||||
|
f"profile-{name}",
|
||||||
|
"GET",
|
||||||
|
"/api/user/phone?phoneNumber=%2B56000000000",
|
||||||
|
{},
|
||||||
|
profile_config,
|
||||||
|
[(status, payload)],
|
||||||
|
)
|
||||||
|
yield (
|
||||||
|
"profile-login-token",
|
||||||
|
"GET",
|
||||||
|
"/api/user/phone?phoneNumber=123",
|
||||||
|
{},
|
||||||
|
{
|
||||||
|
**profile_config,
|
||||||
|
"S2S_LOGIN_BY_PHONE": "http://profile.test/login",
|
||||||
|
},
|
||||||
|
[(200, {"result": {"data": {"accessToken": "fixture-token", "future": True}}}), (200, {"name": "Caller"})],
|
||||||
|
)
|
||||||
|
|
||||||
|
yield "reports-disabled", "GET", "/api/user/reports", {}, {}, []
|
||||||
|
yield "reports-unconfigured", "GET", "/api/user/reports", {}, {"S2S_INJECT_REPORTS": True}, []
|
||||||
|
reports_config = {"S2S_INJECT_REPORTS": True, "S2S_REPORT_LIST": "http://reports.test/list"}
|
||||||
|
yield "reports-empty", "GET", "/api/user/reports", {}, reports_config, [(200, [])]
|
||||||
|
yield "reports-upstream-failure", "GET", "/api/user/reports", {}, reports_config, [(503, {"error": "fixture"})]
|
||||||
|
yield (
|
||||||
|
"reports-normalization",
|
||||||
|
"GET",
|
||||||
|
"/api/user/reports?phoneNumber=123",
|
||||||
|
{},
|
||||||
|
reports_config,
|
||||||
|
[
|
||||||
|
(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"result": [
|
||||||
|
{
|
||||||
|
"sourceScheduleLabel": "Daily",
|
||||||
|
"date": "2026-01-02T12:00:00Z",
|
||||||
|
"podcastScript": "private script",
|
||||||
|
"future": True,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"sourceScheduleLabel": "Daily",
|
||||||
|
"date": "2026-01-01T12:00:00Z",
|
||||||
|
"podcastScript": "older script",
|
||||||
|
},
|
||||||
|
{"sourceSchedule": {"name": "Weekly", "date": "2026-01-03T12:00:00Z"}, "text": "weekly script"},
|
||||||
|
],
|
||||||
|
"future": True,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
lb_config = {"LOAD_BALANCER_URL": "http://lb.test"}
|
||||||
|
for name, status, payload in (
|
||||||
|
(
|
||||||
|
"grant",
|
||||||
|
200,
|
||||||
|
{"session_id": "session", "connect_url": "ws://voice.test?token=fixture", "future": {"nested": True}},
|
||||||
|
),
|
||||||
|
("camelcase", 200, {"sessionId": "session", "connectUrl": "ws://voice.test", "unknown": None}),
|
||||||
|
("queued", 200, {"state": "queued", "queue_id": "ticket", "position": 3, "future": [1]}),
|
||||||
|
("empty", 200, {}),
|
||||||
|
("capacity", 503, {"state": "at_capacity", "future": True}),
|
||||||
|
("other-error", 429, {"error": "fixture"}),
|
||||||
|
("invalid-json", 200, b"not-json"),
|
||||||
|
):
|
||||||
|
yield (
|
||||||
|
f"session-{name}",
|
||||||
|
"POST",
|
||||||
|
"/api/session",
|
||||||
|
{"content": b"arbitrary ignored request body"},
|
||||||
|
lb_config,
|
||||||
|
[(status, payload)],
|
||||||
|
)
|
||||||
|
for name, status, payload in (
|
||||||
|
("waiting", 200, {"state": "queued", "queue_id": "ticket", "position": None, "future": True}),
|
||||||
|
("grant", 200, {"session_id": "session", "connect_url": "ws://voice.test"}),
|
||||||
|
("expired", 404, {"unknown": True}),
|
||||||
|
("error", 500, {}),
|
||||||
|
("invalid-json", 200, b"bad"),
|
||||||
|
):
|
||||||
|
yield f"queue-{name}", "GET", "/api/queue/ticket", {}, lb_config, [(status, payload)]
|
||||||
|
yield "queue-delete", "DELETE", "/api/queue/ticket", {}, lb_config, [(500, {})]
|
||||||
|
for name, kwargs, upstream in (
|
||||||
|
("valid", {"json": {"queueId": "ticket", "future": True}}, [(204, b"")]),
|
||||||
|
("empty", {"content": b""}, []),
|
||||||
|
("invalid-json", {"content": b"bad"}, []),
|
||||||
|
("array", {"json": ["ticket"]}, []),
|
||||||
|
("null-id", {"json": {"queueId": None}}, []),
|
||||||
|
("numeric-id", {"json": {"queueId": 42}}, []),
|
||||||
|
):
|
||||||
|
yield f"queue-end-{name}", "POST", "/api/queue/end", kwargs, lb_config, upstream
|
||||||
|
|
||||||
|
for name, payload in (("missing", {}), ("proxy", {"transcript": "hello"}), ("proxy-blank", {"transcript": " "})):
|
||||||
|
yield f"reasoning-{name}", "POST", "/api/reasoning", {"json": payload}, {}, []
|
||||||
|
yield (
|
||||||
|
"reasoning-openai-profile-disabled",
|
||||||
|
"POST",
|
||||||
|
"/api/reasoning",
|
||||||
|
{"json": {"transcript": "hello"}},
|
||||||
|
{"S2S_REASONING_TYPE": "OPENAI"},
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
yield (
|
||||||
|
"reasoning-legacy-unbound-profile",
|
||||||
|
"POST",
|
||||||
|
"/api/reasoning",
|
||||||
|
{"json": {"transcript": "hello"}},
|
||||||
|
{
|
||||||
|
"S2S_REASONING_TYPE": "OPENAI",
|
||||||
|
"S2S_INJECT_PROFILE": True,
|
||||||
|
"S2S_LATEST_SUMMARY_TEXT": "fixture summary",
|
||||||
|
},
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
reasoning_config = {
|
||||||
|
**profile_config,
|
||||||
|
"S2S_INJECT_PROFILE": True,
|
||||||
|
"S2S_REASONING_TYPE": "OPENAI",
|
||||||
|
}
|
||||||
|
reasoning_request = {
|
||||||
|
"json": {
|
||||||
|
"transcript": " hello ",
|
||||||
|
"phoneNumber": "123",
|
||||||
|
"instructions": "Fixture instructions",
|
||||||
|
"language": "es",
|
||||||
|
"future": True,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
yield (
|
||||||
|
"reasoning-openai-answer",
|
||||||
|
"POST",
|
||||||
|
"/api/reasoning",
|
||||||
|
reasoning_request,
|
||||||
|
{
|
||||||
|
**reasoning_config,
|
||||||
|
"OPENAI_API_KEY": "fixture-key",
|
||||||
|
},
|
||||||
|
[
|
||||||
|
(200, {"name": "Caller"}),
|
||||||
|
(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"choices": [{"message": {"content": " answer ", "future": True}}],
|
||||||
|
"usage": {"total_tokens": 12},
|
||||||
|
"future": True,
|
||||||
|
},
|
||||||
|
),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def capture_http():
|
||||||
|
captures = {}
|
||||||
|
for name, method, path, kwargs, config, responses in cases():
|
||||||
|
remaining = iter(responses)
|
||||||
|
requests = []
|
||||||
|
|
||||||
|
def upstream(request, *, requests=requests, remaining=remaining):
|
||||||
|
requests.append(
|
||||||
|
{
|
||||||
|
"method": request.method,
|
||||||
|
"url": str(request.url),
|
||||||
|
"body": request.content.decode(),
|
||||||
|
"headers": {
|
||||||
|
key: request.headers[key]
|
||||||
|
for key in (
|
||||||
|
"authorization",
|
||||||
|
"x-demo-session",
|
||||||
|
"x-api-key",
|
||||||
|
"content-type",
|
||||||
|
)
|
||||||
|
if key in request.headers
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
status, payload = next(remaining)
|
||||||
|
if isinstance(payload, bytes):
|
||||||
|
return httpx.Response(status, content=payload)
|
||||||
|
return httpx.Response(
|
||||||
|
status, content=json.dumps(payload).encode(), headers={"content-type": "application/json"}
|
||||||
|
)
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.multiple(settings, **{**BASE_CONFIG, **config}),
|
||||||
|
patch.object(context, "_build_today_fixed_context_block", return_value="Fixture current date: 2026-01-04"),
|
||||||
|
patch.object(
|
||||||
|
httpx,
|
||||||
|
"AsyncClient",
|
||||||
|
side_effect=lambda **opts: ASYNC_CLIENT(transport=httpx.MockTransport(upstream), **opts),
|
||||||
|
),
|
||||||
|
TestClient(server.app, raise_server_exceptions=False) as client,
|
||||||
|
):
|
||||||
|
response = client.request(method, path, follow_redirects=False, **kwargs)
|
||||||
|
captures[name] = {
|
||||||
|
"status": response.status_code,
|
||||||
|
"body": response.text,
|
||||||
|
"headers": {
|
||||||
|
key: response.headers[key]
|
||||||
|
for key in ("content-type", "location", "set-cookie")
|
||||||
|
if key in response.headers
|
||||||
|
},
|
||||||
|
"upstream": requests,
|
||||||
|
}
|
||||||
|
if list(remaining):
|
||||||
|
raise AssertionError(f"Unused upstream fixtures in {name}")
|
||||||
|
return captures
|
||||||
|
|
||||||
|
|
||||||
|
def capture_openapi_requests():
|
||||||
|
schema = server.app.openapi()
|
||||||
|
return {
|
||||||
|
"operations": {
|
||||||
|
f"{method.upper()} {path}": {
|
||||||
|
key: operation[key] for key in ("operationId", "parameters", "requestBody") if key in operation
|
||||||
|
}
|
||||||
|
for path, methods in schema["paths"].items()
|
||||||
|
for method, operation in methods.items()
|
||||||
|
},
|
||||||
|
"models": {
|
||||||
|
name: schema["components"]["schemas"][name]
|
||||||
|
for name in (
|
||||||
|
"SearchRequest",
|
||||||
|
"ReasoningRequest",
|
||||||
|
"HTTPValidationError",
|
||||||
|
"ValidationError",
|
||||||
|
)
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
async def capture_realtime():
|
||||||
|
frames = []
|
||||||
|
|
||||||
|
class Socket:
|
||||||
|
async def send_text(self, message):
|
||||||
|
frames.append(message)
|
||||||
|
|
||||||
|
async def send_bytes(self, message):
|
||||||
|
frames.append({"binary": message.hex()})
|
||||||
|
|
||||||
|
async def upstream():
|
||||||
|
yield b"\x00\x01\x02\x03"
|
||||||
|
yield json.dumps({"type": "future.event", "future": {"nested": [None, 1]}})
|
||||||
|
yield json.dumps({"type": "response.text.done", "text": "Hello [rightspeak]world[endspeak]", "future": True})
|
||||||
|
yield json.dumps({"type": "error", "error": {"message": "Unknown or invalid event: session.update"}})
|
||||||
|
yield json.dumps({"type": "future.after.suppressed.error"})
|
||||||
|
|
||||||
|
await proxy._upstream_to_client(Socket(), upstream(), "fixture", {})
|
||||||
|
return frames
|
||||||
|
|
||||||
|
|
||||||
|
def capture():
|
||||||
|
return {
|
||||||
|
"http": capture_http(),
|
||||||
|
"openapi_requests": capture_openapi_requests(),
|
||||||
|
"realtime": asyncio.run(capture_realtime()),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class LegacyCompatibilityTests(unittest.TestCase):
|
||||||
|
def test_native_transcripts_are_relayed_without_external_http_calls(self):
|
||||||
|
events = [
|
||||||
|
{"type": "conversation.item.input_audio_transcription.completed", "transcript": "Hola", "item_id": "u1"},
|
||||||
|
{"type": "response.audio_transcript.done", "transcript": "Respuesta", "response_id": "r1"},
|
||||||
|
{"type": "response.output_audio_transcript.done", "transcript": "Respuesta", "response_id": "r1"},
|
||||||
|
{"type": "response.content_part.done", "part": {"type": "text", "text": "Respuesta"}},
|
||||||
|
*[
|
||||||
|
{
|
||||||
|
"type": event_type,
|
||||||
|
"item": {"id": "a1", "role": "assistant", "content": [{"type": "text", "text": "Respuesta"}]},
|
||||||
|
}
|
||||||
|
for event_type in ("response.output_item.done", "conversation.item.created")
|
||||||
|
],
|
||||||
|
{"type": "response.text.done", "text": "Respuesta", "response_id": "r1"},
|
||||||
|
]
|
||||||
|
messages = [json.dumps(event) for event in events]
|
||||||
|
forwarded = []
|
||||||
|
|
||||||
|
class Socket:
|
||||||
|
async def send_text(self, message):
|
||||||
|
forwarded.append(message)
|
||||||
|
|
||||||
|
async def upstream():
|
||||||
|
for message in messages:
|
||||||
|
yield message
|
||||||
|
|
||||||
|
async def relay():
|
||||||
|
await proxy._upstream_to_client(Socket(), upstream(), "fixture", {})
|
||||||
|
# Allow any accidentally scheduled background HTTP work to run.
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
|
||||||
|
for mode in ("PROXY", "OPENAI"):
|
||||||
|
forwarded.clear()
|
||||||
|
with (
|
||||||
|
self.subTest(mode=mode),
|
||||||
|
patch.multiple(settings, **{**BASE_CONFIG, "S2S_REASONING_TYPE": mode}),
|
||||||
|
patch.object(httpx, "AsyncClient", side_effect=AssertionError("Unexpected HTTP call")) as http,
|
||||||
|
):
|
||||||
|
asyncio.run(relay())
|
||||||
|
self.assertEqual(forwarded, messages)
|
||||||
|
http.assert_not_called()
|
||||||
|
|
||||||
|
def test_wire_contracts_match_pre_migration_capture(self):
|
||||||
|
expected = json.loads(FIXTURE.read_text())
|
||||||
|
actual = capture()
|
||||||
|
for name, response in actual["http"].items():
|
||||||
|
with self.subTest(case=name):
|
||||||
|
self.assertEqual(response, expected["http"][name])
|
||||||
|
# Additive conversation/MCP routes have their own contract tests. Every
|
||||||
|
# pre-existing operation and request schema still matches the capture.
|
||||||
|
operations = actual["openapi_requests"]["operations"]
|
||||||
|
added = set(operations) - set(expected["openapi_requests"]["operations"])
|
||||||
|
self.assertEqual(
|
||||||
|
added,
|
||||||
|
{
|
||||||
|
"POST /api/conversations",
|
||||||
|
"DELETE /api/conversations/current",
|
||||||
|
"POST /api/conversations/current/interrupt",
|
||||||
|
"GET /api/mcp/tools",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
for key in added:
|
||||||
|
del operations[key]
|
||||||
|
self.assertEqual(actual["openapi_requests"], expected["openapi_requests"])
|
||||||
|
self.assertEqual(actual["realtime"], expected["realtime"])
|
||||||
|
self.assertEqual(set(actual["http"]), set(expected["http"]))
|
||||||
@@ -0,0 +1,421 @@
|
|||||||
|
"""Linked content, explicit conversation selection, and authenticated greeting context."""
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
from unittest.mock import AsyncMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from websockets.sync.server import serve
|
||||||
|
|
||||||
|
with patch.dict(os.environ, {}, clear=True), patch("dotenv.load_dotenv", return_value=False):
|
||||||
|
from vocero_api.core import config as settings
|
||||||
|
from vocero_api.main import app
|
||||||
|
from vocero_api.models.mcp import BackendLogin, IntegrationError, McpToolDefinition
|
||||||
|
from vocero_api.providers import backend_login, mcp
|
||||||
|
from vocero_api.realtime.mcp_proxy import McpRelay
|
||||||
|
from vocero_api.services import content_context, conversations, mcp_reasoning, mcp_tools
|
||||||
|
|
||||||
|
REPORT_ID = "report-11111111-2222-4333-8444-555555555555"
|
||||||
|
TITLE = "Alimentación de La Leona"
|
||||||
|
OPENING_INSTRUCTION = "For the opening greeting only"
|
||||||
|
|
||||||
|
|
||||||
|
def report_result(report_id=REPORT_ID, title=TITLE, has_podcast=True):
|
||||||
|
return {
|
||||||
|
"data": {
|
||||||
|
"result": {
|
||||||
|
"status": "found",
|
||||||
|
"detail": {
|
||||||
|
"report": {"id": report_id, "title": title, "date": "2026-10-08", "has_podcast": has_podcast},
|
||||||
|
"summary": "Full details must not be copied to the startup prompt.",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def podcast_result(podcast_id="daily:episode-1", title="Resumen diario"):
|
||||||
|
return {
|
||||||
|
"data": {"podcast": {"id": podcast_id, "kind": podcast_id.split(":")[0], "title": title, "date": "2026-10-08"}}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def marker(name, arguments):
|
||||||
|
return f"<<<VOCERO_MCP_CALL:NAME={name},ARGUMENTS={json.dumps(arguments)}>>>"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def enabled(monkeypatch):
|
||||||
|
monkeypatch.setattr(settings, "S2S_MCP_URL", "http://mcp.test/mcp/")
|
||||||
|
monkeypatch.setattr(settings, "S2S_REASONING_TYPE", "OPENAI")
|
||||||
|
monkeypatch.setattr(settings, "S2S_START_SESSION_MODE", "TALKFIRST")
|
||||||
|
monkeypatch.setattr(settings, "S2S_PRIMARY_PROMPT", "")
|
||||||
|
monkeypatch.setattr(settings, "OPENAI_API_KEY", "fixture-key")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
backend_login,
|
||||||
|
"login",
|
||||||
|
AsyncMock(return_value=BackendLogin(sessionToken="private-alice", user={"id": "alice", "name": "Alice"})),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mcp,
|
||||||
|
"discover",
|
||||||
|
AsyncMock(return_value=[McpToolDefinition(name=name, parameters={}) for name in sorted(mcp.ALLOWED_TOOLS)]),
|
||||||
|
)
|
||||||
|
execute = AsyncMock(return_value=report_result())
|
||||||
|
monkeypatch.setattr(mcp, "execute", execute)
|
||||||
|
conversations.store.clear()
|
||||||
|
yield execute
|
||||||
|
conversations.store.clear()
|
||||||
|
|
||||||
|
|
||||||
|
def bootstrap(client, **selectors):
|
||||||
|
response = client.post("/api/conversations", json={"phoneNumber": "+56000000001", **selectors})
|
||||||
|
assert response.status_code == 200, response.text
|
||||||
|
item = conversations.store.get(client.cookies.get(conversations.COOKIE_NAME), response.json()["conversationId"])
|
||||||
|
return item
|
||||||
|
|
||||||
|
|
||||||
|
def test_report_link_is_resolved_once_with_the_caller_token_and_only_metadata_is_retained(enabled):
|
||||||
|
with TestClient(app) as client:
|
||||||
|
item = bootstrap(client, report_id=REPORT_ID)
|
||||||
|
enabled.assert_awaited_once_with(
|
||||||
|
"private-alice", "get_report", {"request": {"report_id": REPORT_ID, "limit": 1}}
|
||||||
|
)
|
||||||
|
assert item.linked_content.report.title == TITLE
|
||||||
|
assert item.current_content.podcast.id == f"report:{REPORT_ID}"
|
||||||
|
assert item.linked_content == item.current_content
|
||||||
|
prompt = mcp_tools.instructions("Base instructions", item)
|
||||||
|
assert TITLE in prompt and REPORT_ID in prompt and OPENING_INSTRUCTION in prompt
|
||||||
|
assert "Full details" not in prompt and "private-alice" not in prompt
|
||||||
|
catalog = client.get("/api/mcp/tools", headers={conversations.CONVERSATION_HEADER: item.id}).json()["tools"]
|
||||||
|
assert {tool["name"] for tool in catalog} == mcp.ALLOWED_TOOLS | {"select_content"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_link_needs_no_lookup_and_existing_clients_keep_working(enabled):
|
||||||
|
with TestClient(app) as client:
|
||||||
|
item = bootstrap(client)
|
||||||
|
enabled.assert_not_called()
|
||||||
|
assert item.current_content.report is None and item.current_content.podcast is None
|
||||||
|
assert not item.current_content.unavailable
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"result",
|
||||||
|
[
|
||||||
|
{"error": "Denied: never expose a private title"},
|
||||||
|
{"data": {"result": {"status": "ambiguous", "choices": []}}},
|
||||||
|
report_result(report_id="some-other-report"),
|
||||||
|
{"data": {"result": {"status": "found", "detail": {"report": {"id": REPORT_ID}}}}},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_unavailable_or_invalid_links_allow_a_generic_conversation_without_leaking_details(enabled, result):
|
||||||
|
enabled.return_value = result
|
||||||
|
with TestClient(app) as client:
|
||||||
|
item = bootstrap(client, report_id=REPORT_ID)
|
||||||
|
assert item.current_content.unavailable
|
||||||
|
assert item.current_content.report is None and item.current_content.podcast is None
|
||||||
|
prompt = mcp_tools.instructions("", item)
|
||||||
|
assert TITLE not in prompt and "private title" not in prompt
|
||||||
|
assert OPENING_INSTRUCTION in prompt
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("status", [401, 403, 502])
|
||||||
|
def test_link_transport_failures_preserve_authentication_boundary(enabled, status):
|
||||||
|
enabled.side_effect = IntegrationError(status, "Upstream detail must stay private")
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.post("/api/conversations", json={"phoneNumber": "+56000000001", "report_id": REPORT_ID})
|
||||||
|
if status == 401:
|
||||||
|
assert response.status_code == 401
|
||||||
|
assert not conversations.store._items
|
||||||
|
else:
|
||||||
|
assert response.status_code == 200
|
||||||
|
item = conversations.store.get(
|
||||||
|
client.cookies.get(conversations.COOKIE_NAME), response.json()["conversationId"]
|
||||||
|
)
|
||||||
|
assert item.current_content.unavailable
|
||||||
|
assert "Upstream detail" not in mcp_tools.instructions("", item)
|
||||||
|
|
||||||
|
|
||||||
|
def test_link_lookup_timeout_does_not_block_the_whole_conversation(enabled, monkeypatch):
|
||||||
|
cancelled = []
|
||||||
|
|
||||||
|
async def slow_lookup(*args):
|
||||||
|
try:
|
||||||
|
await asyncio.sleep(10)
|
||||||
|
finally:
|
||||||
|
cancelled.append(True)
|
||||||
|
|
||||||
|
enabled.side_effect = slow_lookup
|
||||||
|
monkeypatch.setattr(content_context, "LINK_LOOKUP_TIMEOUT_SEC", 0.01)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
assert bootstrap(client, report_id=REPORT_ID).current_content.unavailable
|
||||||
|
assert cancelled == [True]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("episode_available", [True, False])
|
||||||
|
def test_explicit_daily_episode_wins_without_falling_back_to_the_report_podcast(enabled, episode_available):
|
||||||
|
enabled.side_effect = [report_result(), podcast_result() if episode_available else {"error": "Unavailable"}]
|
||||||
|
with TestClient(app) as client:
|
||||||
|
item = bootstrap(client, report_id=REPORT_ID, episode_id="episode-1")
|
||||||
|
assert item.linked_content.report.id == REPORT_ID
|
||||||
|
assert (item.linked_content.podcast.id if item.linked_content.podcast else None) == (
|
||||||
|
"daily:episode-1" if episode_available else None
|
||||||
|
)
|
||||||
|
assert item.linked_content.unavailable is not episode_available
|
||||||
|
assert enabled.await_args_list[1].args == (
|
||||||
|
"private-alice",
|
||||||
|
"get_podcast",
|
||||||
|
{"request": {"podcast_id": "daily:episode-1"}},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_linked_content_is_isolated_between_tabs_and_cleared_when_closed(enabled):
|
||||||
|
enabled.side_effect = [report_result(), report_result("report-b", "Mortalidad", False)]
|
||||||
|
with TestClient(app) as client:
|
||||||
|
first = bootstrap(client, report_id=REPORT_ID)
|
||||||
|
second = bootstrap(client, report_id="report-b")
|
||||||
|
assert second.current_content.podcast is None
|
||||||
|
client.delete("/api/conversations/current", headers={conversations.CONVERSATION_HEADER: first.id})
|
||||||
|
assert first.linked_content.report is None and first.current_content.podcast is None
|
||||||
|
assert second.current_content.report.id == "report-b"
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_and_comparison_leave_selection_unchanged_but_explicit_selection_replaces_it(enabled):
|
||||||
|
async def scenario():
|
||||||
|
linked = await content_context.from_link("private-alice", REPORT_ID, None)
|
||||||
|
item = conversations.store.create("browser", await backend_login.login("+56000000001"), linked)
|
||||||
|
item.catalog = await content_context.catalog(item.backend_token)
|
||||||
|
turn = mcp_tools.ToolTurn(item, item.generation)
|
||||||
|
enabled.return_value = report_result("report-b", "Mortalidad", False)
|
||||||
|
await turn.call("list_reports", {"request": {"start_date": "2026-10-01"}})
|
||||||
|
assert enabled.await_args.args == ("private-alice", "list_reports", {"request": {"start_date": "2026-10-01"}})
|
||||||
|
await turn.call("get_report", {"request": {"report_id": "report-b"}})
|
||||||
|
assert item.current_content == linked
|
||||||
|
await turn.call("select_content", {"request": {"kind": "report", "id": "report-b"}})
|
||||||
|
assert item.current_content.report.id == "report-b" and item.current_content.podcast is None
|
||||||
|
assert item.linked_content == linked
|
||||||
|
enabled.return_value = {"error": "Denied"}
|
||||||
|
assert "error" in await turn.call("select_content", {"request": {"kind": "report", "id": "private-report"}})
|
||||||
|
assert item.current_content.report.id == "report-b"
|
||||||
|
await turn.call("get_podcast", {})
|
||||||
|
assert enabled.await_args.kwargs == {"podcast_id": None}
|
||||||
|
|
||||||
|
asyncio.run(scenario())
|
||||||
|
|
||||||
|
|
||||||
|
def test_explicit_podcast_arguments_are_preserved_and_reading_does_not_change_the_default(enabled):
|
||||||
|
async def scenario():
|
||||||
|
linked = await content_context.from_link("private-alice", REPORT_ID, None)
|
||||||
|
item = conversations.store.create("browser", await backend_login.login("+56000000001"), linked)
|
||||||
|
item.catalog = await content_context.catalog(item.backend_token)
|
||||||
|
enabled.return_value = podcast_result()
|
||||||
|
turn = mcp_tools.ToolTurn(item, item.generation)
|
||||||
|
await turn.call("get_podcast", {"request": {"podcast_id": "daily:episode-1"}})
|
||||||
|
assert enabled.await_args.args[2] == {"request": {"podcast_id": "daily:episode-1"}}
|
||||||
|
assert enabled.await_args.kwargs == {"podcast_id": f"report:{REPORT_ID}"}
|
||||||
|
assert item.current_content == linked
|
||||||
|
await turn.call("select_content", {"request": {"kind": "podcast", "id": "daily:episode-1"}})
|
||||||
|
assert item.current_content.podcast.id == "daily:episode-1" and item.current_content.report is None
|
||||||
|
await turn.call("read_podcast_transcript", {"request": {"query": "oxígeno"}})
|
||||||
|
assert enabled.await_args.kwargs == {"podcast_id": "daily:episode-1"}
|
||||||
|
|
||||||
|
asyncio.run(scenario())
|
||||||
|
|
||||||
|
|
||||||
|
def test_cancelled_selection_cannot_replace_the_current_reference(enabled):
|
||||||
|
async def scenario():
|
||||||
|
linked = await content_context.from_link("private-alice", REPORT_ID, None)
|
||||||
|
item = conversations.store.create("browser", await backend_login.login("+56000000001"), linked)
|
||||||
|
item.catalog = await content_context.catalog(item.backend_token)
|
||||||
|
started, release = asyncio.Event(), asyncio.Event()
|
||||||
|
|
||||||
|
async def slow_result(*args):
|
||||||
|
started.set()
|
||||||
|
try:
|
||||||
|
await release.wait()
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
await release.wait()
|
||||||
|
return report_result("report-b", "Mortalidad")
|
||||||
|
|
||||||
|
enabled.side_effect = slow_result
|
||||||
|
turn = mcp_tools.ToolTurn(item, item.generation)
|
||||||
|
item.pending = asyncio.create_task(
|
||||||
|
turn.call("select_content", {"request": {"kind": "report", "id": "report-b"}})
|
||||||
|
)
|
||||||
|
task = item.pending
|
||||||
|
await started.wait()
|
||||||
|
item.interrupt()
|
||||||
|
release.set()
|
||||||
|
with pytest.raises(asyncio.CancelledError):
|
||||||
|
await task
|
||||||
|
assert item.current_content == linked
|
||||||
|
|
||||||
|
asyncio.run(scenario())
|
||||||
|
|
||||||
|
|
||||||
|
def test_implicit_podcast_reads_use_the_new_selection_within_the_same_turn(enabled):
|
||||||
|
async def scenario():
|
||||||
|
linked = await content_context.from_link("private-alice", REPORT_ID, None)
|
||||||
|
item = conversations.store.create("browser", await backend_login.login("+56000000001"), linked)
|
||||||
|
item.catalog = await content_context.catalog(item.backend_token)
|
||||||
|
turn = mcp_tools.ToolTurn(item, item.generation)
|
||||||
|
await turn.call("get_podcast", {})
|
||||||
|
assert enabled.await_args.kwargs == {"podcast_id": f"report:{REPORT_ID}"}
|
||||||
|
enabled.return_value = podcast_result()
|
||||||
|
await turn.call("select_content", {"request": {"kind": "podcast", "id": "daily:episode-1"}})
|
||||||
|
await turn.call("get_podcast", {})
|
||||||
|
assert enabled.await_args.kwargs == {"podcast_id": "daily:episode-1"}
|
||||||
|
with pytest.raises(IntegrationError):
|
||||||
|
await turn.call("get_podcast", {})
|
||||||
|
enabled.reset_mock()
|
||||||
|
fresh = mcp_tools.ToolTurn(item, item.generation)
|
||||||
|
assert "error" in await fresh.call("select_content", {"request": {"kind": "report", "id": " "}})
|
||||||
|
enabled.assert_not_called()
|
||||||
|
assert item.current_content.podcast.id == "daily:episode-1"
|
||||||
|
|
||||||
|
asyncio.run(scenario())
|
||||||
|
|
||||||
|
|
||||||
|
def test_revoked_authentication_stops_remaining_link_lookups(enabled):
|
||||||
|
enabled.side_effect = IntegrationError(401, "La sesión expiró.")
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.post(
|
||||||
|
"/api/conversations",
|
||||||
|
json={"phoneNumber": "+56000000001", "report_id": REPORT_ID, "episode_id": "episode-1"},
|
||||||
|
)
|
||||||
|
assert response.status_code == 401
|
||||||
|
assert enabled.await_count == 1
|
||||||
|
assert not conversations.store._items
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("mode", ["TALKFIRST", "TALK", ""])
|
||||||
|
def test_http_greeting_and_followup_use_server_context_and_refresh_after_selection(enabled, monkeypatch, mode):
|
||||||
|
monkeypatch.setattr(settings, "S2S_START_SESSION_MODE", mode)
|
||||||
|
model = AsyncMock(
|
||||||
|
side_effect=[
|
||||||
|
"Hola, Alice.",
|
||||||
|
marker("select_content", {"request": {"kind": "report", "id": "report-b"}}),
|
||||||
|
"Revisemos mortalidad.",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(mcp_reasoning, "response_text", model)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
item = bootstrap(client, report_id=REPORT_ID)
|
||||||
|
headers = {conversations.CONVERSATION_HEADER: item.id, "X-Vocero-Turn": "1"}
|
||||||
|
assert client.post("/api/reasoning", headers=headers, json={"transcript": "hola vocero"}).status_code == 200
|
||||||
|
opening = model.await_args_list[0].args[0][0]["content"]
|
||||||
|
assert TITLE in opening
|
||||||
|
assert (OPENING_INSTRUCTION in opening) is (mode == "TALKFIRST")
|
||||||
|
assert not item.greeting_pending
|
||||||
|
enabled.return_value = report_result("report-b", "Mortalidad")
|
||||||
|
headers["X-Vocero-Turn"] = "2"
|
||||||
|
assert (
|
||||||
|
client.post(
|
||||||
|
"/api/reasoning",
|
||||||
|
headers=headers,
|
||||||
|
json={"transcript": "Ahora revisemos el otro reporte", "report_id": "browser-spoof"},
|
||||||
|
).status_code
|
||||||
|
== 200
|
||||||
|
)
|
||||||
|
current_prompt = model.await_args.args[0][0]["content"]
|
||||||
|
assert '"id": "report-b"' in current_prompt and TITLE in current_prompt
|
||||||
|
assert "browser-spoof" not in current_prompt and OPENING_INSTRUCTION not in current_prompt
|
||||||
|
assert item.linked_content.report.id == REPORT_ID
|
||||||
|
|
||||||
|
|
||||||
|
def test_voice_selection_refreshes_instructions_before_the_next_response_without_new_speech_protocol(enabled):
|
||||||
|
async def scenario():
|
||||||
|
linked = await content_context.from_link("private-alice", REPORT_ID, None)
|
||||||
|
item = conversations.store.create("browser", await backend_login.login("+56000000001"), linked)
|
||||||
|
item.catalog = await content_context.catalog(item.backend_token)
|
||||||
|
item.greeting_pending = False
|
||||||
|
browser, speech = AsyncMock(), AsyncMock()
|
||||||
|
relay = McpRelay(browser, speech, item)
|
||||||
|
relay.base_instructions = "Existing voice instructions"
|
||||||
|
enabled.return_value = report_result("report-b", "Mortalidad")
|
||||||
|
await relay.tool_call("select_content", {"request": {"kind": "report", "id": "report-b"}}, relay.turn)
|
||||||
|
events = [json.loads(call.args[0]) for call in speech.send.await_args_list]
|
||||||
|
assert [event["type"] for event in events] == ["session.update", "conversation.item.create", "response.create"]
|
||||||
|
assert "Existing voice instructions" in events[0]["session"]["instructions"]
|
||||||
|
assert '"id": "report-b"' in events[0]["session"]["instructions"]
|
||||||
|
assert TITLE in events[0]["session"]["instructions"]
|
||||||
|
assert "private-alice" not in json.dumps(events)
|
||||||
|
|
||||||
|
asyncio.run(scenario())
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("available", [True, False])
|
||||||
|
def test_voice_talkfirst_has_verified_link_context_before_greeting_without_a_session_ack(
|
||||||
|
enabled, monkeypatch, available
|
||||||
|
):
|
||||||
|
monkeypatch.setattr(settings, "S2S_REASONING_TYPE", "PROXY")
|
||||||
|
enabled.return_value = report_result() if available else {"error": "Inaccessible title must not be disclosed"}
|
||||||
|
received, errors = [], []
|
||||||
|
refreshed = threading.Event()
|
||||||
|
|
||||||
|
def voice(socket):
|
||||||
|
try:
|
||||||
|
socket.send(json.dumps({"type": "session.created"}))
|
||||||
|
for raw in socket:
|
||||||
|
event = json.loads(raw)
|
||||||
|
received.append(event)
|
||||||
|
if event["type"] == "session.update" and OPENING_INSTRUCTION not in event["session"]["instructions"]:
|
||||||
|
refreshed.set()
|
||||||
|
if event["type"] == "response.create":
|
||||||
|
socket.send(json.dumps({"type": "response.created", "response": {"id": "opening"}}))
|
||||||
|
socket.send(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"type": "response.done",
|
||||||
|
"response": {
|
||||||
|
"id": "opening",
|
||||||
|
"status": "completed",
|
||||||
|
"output": [{"content": [{"text": "Hola, Alice."}]}],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
errors.append(exc)
|
||||||
|
|
||||||
|
with serve(voice, "127.0.0.1", 0) as upstream:
|
||||||
|
thread = threading.Thread(target=upstream.serve_forever, daemon=True)
|
||||||
|
thread.start()
|
||||||
|
monkeypatch.setattr(
|
||||||
|
settings, "S2S_PROXY_TARGET", f"ws://127.0.0.1:{upstream.socket.getsockname()[1]}/v1/realtime"
|
||||||
|
)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
item = bootstrap(client, report_id=REPORT_ID)
|
||||||
|
with client.websocket_connect(f"/v1/realtime?conversation_id={item.id}") as socket:
|
||||||
|
assert socket.receive_json()["type"] == "session.created"
|
||||||
|
socket.send_json({"type": "session.update", "session": {"instructions": "Existing instructions"}})
|
||||||
|
socket.send_json(
|
||||||
|
{
|
||||||
|
"type": "conversation.item.create",
|
||||||
|
"item": {
|
||||||
|
"role": "user",
|
||||||
|
"type": "message",
|
||||||
|
"content": [{"type": "input_text", "text": "hola vocero"}],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
socket.send_json({"type": "response.create"})
|
||||||
|
assert socket.receive_json()["type"] == "response.created"
|
||||||
|
assert socket.receive_json()["type"] == "response.done"
|
||||||
|
assert refreshed.wait(2)
|
||||||
|
assert not item.greeting_pending
|
||||||
|
thread.join(3)
|
||||||
|
assert not errors
|
||||||
|
assert [event["type"] for event in received[:3]] == [
|
||||||
|
"session.update",
|
||||||
|
"conversation.item.create",
|
||||||
|
"response.create",
|
||||||
|
]
|
||||||
|
initial = received[0]["session"]["instructions"]
|
||||||
|
assert OPENING_INSTRUCTION in initial and "Existing instructions" in initial
|
||||||
|
assert (TITLE in initial) is available
|
||||||
|
assert "Inaccessible title" not in initial
|
||||||
|
assert "private-alice" not in json.dumps(received)
|
||||||
@@ -0,0 +1,366 @@
|
|||||||
|
"""Caller isolation, bounded tool loops, and the unchanged speech protocol."""
|
||||||
|
|
||||||
|
import ast
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import threading
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import AsyncMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from websockets.sync.server import serve
|
||||||
|
|
||||||
|
with patch.dict(os.environ, {}, clear=True), patch("dotenv.load_dotenv", return_value=False):
|
||||||
|
from vocero_api.core import config as settings
|
||||||
|
from vocero_api.main import app
|
||||||
|
from vocero_api.models.mcp import BackendLogin, IntegrationError, McpToolDefinition
|
||||||
|
from vocero_api.providers import backend_login, caller, mcp
|
||||||
|
from vocero_api.services import conversations, mcp_reasoning, mcp_tools
|
||||||
|
|
||||||
|
|
||||||
|
def catalog():
|
||||||
|
return [
|
||||||
|
McpToolDefinition(
|
||||||
|
name=name,
|
||||||
|
description=f"Published {name} content",
|
||||||
|
parameters={
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"request": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"report_id": {"type": "string"}, "offset": {"type": "integer"}},
|
||||||
|
"additionalProperties": False,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
for name in ("list_reports", "get_report")
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def marker(name, arguments):
|
||||||
|
return f"<<<VOCERO_MCP_CALL:NAME={name},ARGUMENTS={json.dumps(arguments)}>>>"
|
||||||
|
|
||||||
|
|
||||||
|
def test_mcp_marker_survives_the_existing_speech_sanitizer_without_model_dependencies():
|
||||||
|
# Execute only the pure sanitizer and its constants, not the GPU pipeline imports.
|
||||||
|
path = Path(__file__).resolve().parents[2] / "speech_to_speech/LLM/lm_output_processor.py"
|
||||||
|
tree = ast.parse(path.read_text())
|
||||||
|
selected = [
|
||||||
|
node
|
||||||
|
for node in tree.body
|
||||||
|
if (
|
||||||
|
isinstance(node, ast.Assign)
|
||||||
|
and isinstance(node.targets[0], ast.Name)
|
||||||
|
and node.targets[0].id.startswith("_APIPILOT_")
|
||||||
|
)
|
||||||
|
or (isinstance(node, ast.FunctionDef) and node.name == "_sanitize_apipilot_before_client_and_tts")
|
||||||
|
]
|
||||||
|
namespace = {"re": re}
|
||||||
|
exec(compile(ast.Module(body=selected, type_ignores=[]), str(path), "exec"), namespace)
|
||||||
|
sanitize = namespace["_sanitize_apipilot_before_client_and_tts"]
|
||||||
|
selection = marker("list_reports", {})
|
||||||
|
assert sanitize(selection) == (selection, False, "")
|
||||||
|
assert sanitize(selection.replace("VOCERO_MCP", "APIPILOT"))[1] is True
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"text",
|
||||||
|
["<<<VOC", "<<<VOCERO_MCP_CALL:NAME=get_report,ARGUMENTS=not-json>>>", "Before " + marker("list_reports", {})],
|
||||||
|
)
|
||||||
|
def test_malformed_markers_are_not_treated_as_user_facing_answers(text):
|
||||||
|
with pytest.raises(IntegrationError):
|
||||||
|
mcp_tools.selection(text)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def enabled(monkeypatch):
|
||||||
|
monkeypatch.setattr(settings, "S2S_MCP_URL", "http://mcp.test/mcp")
|
||||||
|
monkeypatch.setattr(settings, "S2S_REASONING_TYPE", "OPENAI")
|
||||||
|
monkeypatch.setattr(settings, "OPENAI_API_KEY", "local-model-key")
|
||||||
|
monkeypatch.setattr(settings, "S2S_PRIMARY_PROMPT", "")
|
||||||
|
login = AsyncMock(
|
||||||
|
return_value=BackendLogin.model_validate(
|
||||||
|
{"sessionToken": "demo-private-alice", "user": {"id": "alice", "name": "Alice"}}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(backend_login, "login", login)
|
||||||
|
monkeypatch.setattr(mcp, "discover", AsyncMock(return_value=catalog()))
|
||||||
|
monkeypatch.setattr(caller, "_login_by_phone", AsyncMock(side_effect=AssertionError("Legacy login")))
|
||||||
|
monkeypatch.setattr(caller, "_fetch_podcast_episode", AsyncMock(side_effect=AssertionError("Unscoped episode")))
|
||||||
|
conversations.store.clear()
|
||||||
|
yield login
|
||||||
|
conversations.store.clear()
|
||||||
|
|
||||||
|
|
||||||
|
def bootstrap(client):
|
||||||
|
response = client.post("/api/conversations", json={"phoneNumber": "+56000000001"})
|
||||||
|
assert response.status_code == 200
|
||||||
|
return response.json()["conversationId"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_http_chains_tools_retains_history_and_ignores_browser_identity(enabled, monkeypatch):
|
||||||
|
model = AsyncMock(
|
||||||
|
side_effect=[
|
||||||
|
marker("list_reports", {}),
|
||||||
|
marker("get_report", {"request": {"report_id": "r1"}}),
|
||||||
|
"Resumen r1",
|
||||||
|
"Seguimiento r1",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
execute = AsyncMock(return_value={"data": {"result": {"id": "r1", "summary": "Accessible finding"}}})
|
||||||
|
monkeypatch.setattr(mcp_reasoning, "response_text", model)
|
||||||
|
monkeypatch.setattr(mcp, "execute", execute)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
identity = bootstrap(client)
|
||||||
|
headers = {conversations.CONVERSATION_HEADER: identity, "x-vocero-turn": "1"}
|
||||||
|
body = {
|
||||||
|
"transcript": "Resume el último reporte",
|
||||||
|
"phoneNumber": "another-user",
|
||||||
|
"episode_id": "private-episode",
|
||||||
|
}
|
||||||
|
assert client.post("/api/reasoning", headers=headers, json=body).json() == {"answer": "Resumen r1"}
|
||||||
|
assert execute.await_args_list[0].args == ("demo-private-alice", "list_reports", {})
|
||||||
|
assert execute.await_args_list[1].args == ("demo-private-alice", "get_report", {"request": {"report_id": "r1"}})
|
||||||
|
assert client.post("/api/reasoning", headers=headers, json=body).status_code == 409
|
||||||
|
headers["x-vocero-turn"] = "2"
|
||||||
|
assert (
|
||||||
|
client.post("/api/reasoning", headers=headers, json={"transcript": "¿Y la recomendación?"}).status_code
|
||||||
|
== 200
|
||||||
|
)
|
||||||
|
messages = model.await_args_list[-1].args[0]
|
||||||
|
assert any(message["content"] == "Resumen r1" for message in messages)
|
||||||
|
prompt = messages[0]["content"]
|
||||||
|
assert '"additionalProperties": false' in prompt
|
||||||
|
assert '"request"' in prompt
|
||||||
|
assert "Alice" in prompt
|
||||||
|
assert "demo-private" not in json.dumps(messages)
|
||||||
|
assert "private-episode" not in json.dumps(messages)
|
||||||
|
assert client.post("/api/reasoning", json=body).status_code == 401
|
||||||
|
assert client.get("/api/user/phone?phoneNumber=123").status_code == 409
|
||||||
|
assert client.get("/api/user/reports").status_code == 409
|
||||||
|
assert client.get("/api/apipilot/tools").status_code == 404
|
||||||
|
assert client.post("/api/apipilot/execute", json={"name": "get_report"}).status_code == 404
|
||||||
|
enabled.assert_awaited_once_with("+56000000001")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("status", [401, 403, 502])
|
||||||
|
def test_http_errors_fail_closed_and_revocation_removes_the_conversation(enabled, monkeypatch, status):
|
||||||
|
monkeypatch.setattr(mcp, "discover", AsyncMock(side_effect=IntegrationError(status, "Consulta no disponible.")))
|
||||||
|
model = AsyncMock()
|
||||||
|
monkeypatch.setattr(mcp_reasoning, "response_text", model)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
identity = bootstrap(client)
|
||||||
|
response = client.post(
|
||||||
|
"/api/reasoning",
|
||||||
|
headers={conversations.CONVERSATION_HEADER: identity, "x-vocero-turn": "1"},
|
||||||
|
json={"transcript": "Hola"},
|
||||||
|
)
|
||||||
|
assert response.status_code == status
|
||||||
|
model.assert_not_called()
|
||||||
|
if status == 401:
|
||||||
|
with pytest.raises(IntegrationError):
|
||||||
|
conversations.store.get(client.cookies.get(conversations.COOKIE_NAME), identity)
|
||||||
|
|
||||||
|
|
||||||
|
def test_tool_arguments_duplicates_and_budgets_are_checked_before_execution(enabled, monkeypatch):
|
||||||
|
execute = AsyncMock(return_value={"data": {"items": [], "next_offset": None}})
|
||||||
|
monkeypatch.setattr(mcp, "execute", execute)
|
||||||
|
monkeypatch.setattr(settings, "S2S_MCP_MAX_CALLS_PER_TURN", 3)
|
||||||
|
|
||||||
|
async def scenario():
|
||||||
|
login = await backend_login.login("+56000000001")
|
||||||
|
item = conversations.store.create("browser", login)
|
||||||
|
item.catalog = catalog()
|
||||||
|
turn = mcp_tools.ToolTurn(item, item.generation)
|
||||||
|
assert "error" in await turn.call("delete_user", {})
|
||||||
|
assert "error" in await turn.call("get_report", {"request": {"user_id": "bob"}})
|
||||||
|
execute.assert_not_called()
|
||||||
|
await turn.call("list_reports", {"request": {"offset": 0}})
|
||||||
|
with pytest.raises(IntegrationError):
|
||||||
|
await turn.call("list_reports", {"request": {"offset": 10}})
|
||||||
|
fresh = mcp_tools.ToolTurn(item, item.interrupt())
|
||||||
|
await fresh.call("list_reports", {})
|
||||||
|
with pytest.raises(IntegrationError):
|
||||||
|
await fresh.call("list_reports", {})
|
||||||
|
assert execute.await_count == 2
|
||||||
|
|
||||||
|
asyncio.run(scenario())
|
||||||
|
|
||||||
|
|
||||||
|
def test_cancelled_tool_result_cannot_reach_a_new_turn_even_if_transport_returns_late(enabled, monkeypatch):
|
||||||
|
async def scenario():
|
||||||
|
started = asyncio.Event()
|
||||||
|
cancelled = asyncio.Event()
|
||||||
|
release = asyncio.Event()
|
||||||
|
|
||||||
|
async def execute(*args):
|
||||||
|
started.set()
|
||||||
|
try:
|
||||||
|
await release.wait()
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
cancelled.set()
|
||||||
|
await release.wait()
|
||||||
|
return {"data": "stale result"}
|
||||||
|
|
||||||
|
monkeypatch.setattr(mcp, "execute", execute)
|
||||||
|
item = conversations.store.create("browser", await backend_login.login("+56000000001"))
|
||||||
|
item.catalog = catalog()
|
||||||
|
turn = mcp_tools.ToolTurn(item, item.generation)
|
||||||
|
item.pending = asyncio.create_task(turn.call("get_report", {}))
|
||||||
|
pending = item.pending
|
||||||
|
await started.wait()
|
||||||
|
item.interrupt()
|
||||||
|
await cancelled.wait()
|
||||||
|
release.set()
|
||||||
|
with pytest.raises(asyncio.CancelledError):
|
||||||
|
await pending
|
||||||
|
assert item.history == []
|
||||||
|
|
||||||
|
asyncio.run(scenario())
|
||||||
|
|
||||||
|
|
||||||
|
def test_interrupt_for_an_old_http_turn_cannot_cancel_a_new_turn(enabled):
|
||||||
|
with TestClient(app) as client:
|
||||||
|
identity = bootstrap(client)
|
||||||
|
item = conversations.store.get(client.cookies.get(conversations.COOKIE_NAME), identity)
|
||||||
|
item.client_turn = 2
|
||||||
|
headers = {conversations.CONVERSATION_HEADER: identity, "x-vocero-turn": "1"}
|
||||||
|
assert client.post("/api/conversations/current/interrupt", headers=headers).status_code == 204
|
||||||
|
assert item.generation == 0
|
||||||
|
headers["x-vocero-turn"] = "2"
|
||||||
|
assert client.post("/api/conversations/current/interrupt", headers=headers).status_code == 204
|
||||||
|
assert item.generation == 1
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("speech_order", [False, True])
|
||||||
|
def test_voice_relay_chains_tools_across_turns_hides_markers_and_preserves_audio(enabled, monkeypatch, speech_order):
|
||||||
|
monkeypatch.setattr(settings, "S2S_REASONING_TYPE", "PROXY")
|
||||||
|
monkeypatch.setattr(settings, "S2S_PRIMARY_PROMPT", "Deployment prompt")
|
||||||
|
execute = AsyncMock(return_value={"data": {"id": "r1", "summary": "Accessible finding"}})
|
||||||
|
monkeypatch.setattr(mcp, "execute", execute)
|
||||||
|
received = []
|
||||||
|
paths = []
|
||||||
|
errors = []
|
||||||
|
|
||||||
|
def voice(socket):
|
||||||
|
try:
|
||||||
|
paths.append(socket.request.path)
|
||||||
|
socket.send(json.dumps({"type": "session.created"}))
|
||||||
|
count = 0
|
||||||
|
for raw in socket:
|
||||||
|
if isinstance(raw, bytes):
|
||||||
|
socket.send(raw)
|
||||||
|
continue
|
||||||
|
event = json.loads(raw)
|
||||||
|
received.append(event)
|
||||||
|
if event["type"] == "session.update":
|
||||||
|
socket.send(raw)
|
||||||
|
if event["type"] != "response.create":
|
||||||
|
continue
|
||||||
|
step = count % 3
|
||||||
|
name = "list_reports" if step == 0 else "get_report"
|
||||||
|
text = (
|
||||||
|
marker(name, {} if step == 0 else {"request": {"report_id": "r1"}})
|
||||||
|
if step < 2
|
||||||
|
else "Resultado publicado"
|
||||||
|
)
|
||||||
|
response_id = f"response-{count}"
|
||||||
|
count += 1
|
||||||
|
if speech_order:
|
||||||
|
socket.send(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"type": "response.output_audio_transcript.done",
|
||||||
|
"response_id": response_id,
|
||||||
|
"transcript": text,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
socket.send(json.dumps({"type": "response.created", "response": {"id": response_id, "output": None}}))
|
||||||
|
# Audio before text must wait until a normal answer is identified.
|
||||||
|
socket.send(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"type": "response.audio.delta",
|
||||||
|
"response_id": response_id,
|
||||||
|
"delta": "hidden-tool-audio" if step < 2 else "answer-audio",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if not speech_order:
|
||||||
|
for part in (text[:3], text[3:]):
|
||||||
|
socket.send(
|
||||||
|
json.dumps({"type": "response.text.delta", "response_id": response_id, "delta": part})
|
||||||
|
)
|
||||||
|
done = {
|
||||||
|
"type": "response.done",
|
||||||
|
"response": {
|
||||||
|
"id": response_id,
|
||||||
|
"status": "completed",
|
||||||
|
"output": None if speech_order else [{"content": [{"text": text}]}],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
socket.send(json.dumps(done))
|
||||||
|
socket.send(json.dumps(done))
|
||||||
|
except Exception as exc:
|
||||||
|
errors.append(exc)
|
||||||
|
|
||||||
|
with serve(voice, "127.0.0.1", 0) as upstream:
|
||||||
|
thread = threading.Thread(target=upstream.serve_forever, daemon=True)
|
||||||
|
thread.start()
|
||||||
|
monkeypatch.setattr(
|
||||||
|
settings, "S2S_PROXY_TARGET", f"ws://127.0.0.1:{upstream.socket.getsockname()[1]}/v1/realtime"
|
||||||
|
)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
identity = bootstrap(client)
|
||||||
|
with client.websocket_connect(f"/v1/realtime?conversation_id={identity}") as socket:
|
||||||
|
assert socket.receive_json() == {"type": "session.created"}
|
||||||
|
socket.send_json(
|
||||||
|
{"type": "session.update", "session": {"instructions": "Deployment prompt\nSession RAG notes"}}
|
||||||
|
)
|
||||||
|
prompt = socket.receive_json()["session"]["instructions"]
|
||||||
|
assert '"request"' in prompt and '"offset"' in prompt
|
||||||
|
assert "Alice" in prompt and "demo-private" not in prompt
|
||||||
|
assert "Session RAG notes" in prompt
|
||||||
|
socket.send_bytes(b"\x00\x01")
|
||||||
|
assert socket.receive_bytes() == b"\x00\x01"
|
||||||
|
for _ in range(5):
|
||||||
|
socket.send_json(
|
||||||
|
{
|
||||||
|
"type": "conversation.item.create",
|
||||||
|
"item": {
|
||||||
|
"type": "message",
|
||||||
|
"role": "user",
|
||||||
|
"content": [{"type": "input_text", "text": "Resume el reporte"}],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
socket.send_json({"type": "response.create"})
|
||||||
|
events = []
|
||||||
|
while True:
|
||||||
|
event = socket.receive_json()
|
||||||
|
events.append(event)
|
||||||
|
assert "hidden-tool-audio" not in json.dumps(event)
|
||||||
|
assert "VOCERO_MCP" not in json.dumps(event)
|
||||||
|
if event["type"] == "response.done" and any(e.get("delta") == "answer-audio" for e in events):
|
||||||
|
break
|
||||||
|
assert sum(event["type"] == "vocero.tool.started" for event in events) == 2
|
||||||
|
assert any(event.get("delta") == "answer-audio" for event in events)
|
||||||
|
assert execute.await_count == 10
|
||||||
|
assert all(call.args[0] == "demo-private-alice" for call in execute.await_args_list)
|
||||||
|
assert identity not in paths[0]
|
||||||
|
assert "demo-private" not in json.dumps(received)
|
||||||
|
thread.join(3)
|
||||||
|
assert not errors
|
||||||
|
|
||||||
|
|
||||||
|
def test_proxy_configuration_rejects_a_direct_speech_bypass(enabled, monkeypatch):
|
||||||
|
monkeypatch.setattr(settings, "S2S_REASONING_TYPE", "PROXY")
|
||||||
|
monkeypatch.setattr(settings, "SPEECH_TO_SPEECH_URL", "ws://speech.test/realtime")
|
||||||
|
with TestClient(app) as client:
|
||||||
|
assert client.get("/api/config").status_code == 503
|
||||||
@@ -0,0 +1,191 @@
|
|||||||
|
"""Conversation ownership and real SDK transport against isolated HTTP fixtures."""
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from unittest.mock import AsyncMock, patch
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
with patch.dict(os.environ, {}, clear=True), patch("dotenv.load_dotenv", return_value=False):
|
||||||
|
from vocero_api.core import config as settings
|
||||||
|
from vocero_api.main import app
|
||||||
|
from vocero_api.models.mcp import BackendLogin, IntegrationError
|
||||||
|
from vocero_api.providers import backend_login, mcp
|
||||||
|
from vocero_api.services import conversations
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def enabled(monkeypatch):
|
||||||
|
monkeypatch.setattr(settings, "S2S_MCP_URL", "http://mcp.test/mcp")
|
||||||
|
monkeypatch.setattr(settings, "S2S_LOGIN_BY_PHONE", "http://backend.test/phone-login")
|
||||||
|
conversations.store.clear()
|
||||||
|
yield
|
||||||
|
conversations.store.clear()
|
||||||
|
|
||||||
|
|
||||||
|
def login_payload(user="alice"):
|
||||||
|
return {"sessionToken": f"demo-secret-{user}", "user": {"id": user, "name": user.title()}}
|
||||||
|
|
||||||
|
|
||||||
|
def test_cookie_bound_conversations_keep_tokens_private_and_identity_per_tab(enabled, monkeypatch):
|
||||||
|
login = AsyncMock(side_effect=[BackendLogin.model_validate(login_payload(user)) for user in ("alice", "bob")])
|
||||||
|
monkeypatch.setattr(backend_login, "login", login)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
first = client.post("/api/conversations", json={"phoneNumber": "+56000000001"})
|
||||||
|
cookie = client.cookies.get(conversations.COOKIE_NAME)
|
||||||
|
assert "HttpOnly" in first.headers["set-cookie"]
|
||||||
|
assert "SameSite=strict" in first.headers["set-cookie"]
|
||||||
|
assert "demo-secret" not in first.text
|
||||||
|
alice = first.json()["conversationId"]
|
||||||
|
bob = client.post("/api/conversations", json={"phoneNumber": "+56000000002"}).json()["conversationId"]
|
||||||
|
assert conversations.store.get(cookie, alice).backend_token == "demo-secret-alice"
|
||||||
|
assert conversations.store.get(cookie, bob).backend_token == "demo-secret-bob"
|
||||||
|
client.cookies.clear()
|
||||||
|
assert client.get("/api/mcp/tools", headers={conversations.CONVERSATION_HEADER: alice}).status_code == 401
|
||||||
|
client.cookies.set(conversations.COOKIE_NAME, cookie)
|
||||||
|
assert (
|
||||||
|
client.delete("/api/conversations/current", headers={conversations.CONVERSATION_HEADER: alice}).status_code
|
||||||
|
== 204
|
||||||
|
)
|
||||||
|
with pytest.raises(IntegrationError):
|
||||||
|
conversations.store.get(cookie, alice)
|
||||||
|
assert conversations.store.get(cookie, bob).profile.id == "bob"
|
||||||
|
|
||||||
|
|
||||||
|
def test_auth_failure_and_cross_origin_requests_never_create_a_conversation(enabled, monkeypatch):
|
||||||
|
login = AsyncMock(side_effect=IntegrationError(401, "Sesión no válida."))
|
||||||
|
monkeypatch.setattr(backend_login, "login", login)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
assert (
|
||||||
|
client.post(
|
||||||
|
"/api/conversations", json={"phoneNumber": "+56000000001"}, headers={"Origin": "https://elsewhere.test"}
|
||||||
|
).status_code
|
||||||
|
== 403
|
||||||
|
)
|
||||||
|
login.assert_not_called()
|
||||||
|
response = client.post("/api/conversations", json={"phoneNumber": "+56000000001"})
|
||||||
|
assert response.status_code == 401
|
||||||
|
assert "set-cookie" not in response.headers
|
||||||
|
|
||||||
|
|
||||||
|
def test_store_expires_bounds_and_cancels_pending_work(enabled, monkeypatch):
|
||||||
|
async def scenario():
|
||||||
|
store = conversations.ConversationStore()
|
||||||
|
monkeypatch.setattr(settings, "S2S_MAX_CONVERSATIONS", 1)
|
||||||
|
item = store.create("browser", BackendLogin.model_validate(login_payload()))
|
||||||
|
with pytest.raises(IntegrationError, match="capacidad"):
|
||||||
|
store.create("other-browser", BackendLogin.model_validate(login_payload("bob")))
|
||||||
|
item.pending = asyncio.create_task(asyncio.sleep(100))
|
||||||
|
pending = item.pending
|
||||||
|
item.expires_at = 0
|
||||||
|
with pytest.raises(IntegrationError):
|
||||||
|
store.get("browser", item.id)
|
||||||
|
await asyncio.gather(pending, return_exceptions=True)
|
||||||
|
assert pending.cancelled()
|
||||||
|
assert item.backend_token == ""
|
||||||
|
assert item.closed.is_set()
|
||||||
|
store.create("browser", BackendLogin.model_validate(login_payload()))
|
||||||
|
|
||||||
|
asyncio.run(scenario())
|
||||||
|
|
||||||
|
|
||||||
|
def test_backend_login_validates_exact_contract_without_leaking_credentials(enabled, monkeypatch, caplog):
|
||||||
|
original = httpx.AsyncClient
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
def upstream(request):
|
||||||
|
calls.append(request)
|
||||||
|
return httpx.Response(200, json=login_payload())
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
backend_login.httpx, "AsyncClient", lambda **kwargs: original(transport=httpx.MockTransport(upstream), **kwargs)
|
||||||
|
)
|
||||||
|
result = asyncio.run(backend_login.login("+56000000001"))
|
||||||
|
assert result.sessionToken.get_secret_value() == "demo-secret-alice"
|
||||||
|
assert json.loads(calls[0].content) == {"phoneNumber": "+56000000001"}
|
||||||
|
assert "demo-secret" not in caplog.text
|
||||||
|
assert "demo-secret" not in repr(result)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("use_sse", [False, True])
|
||||||
|
def test_sdk_discovery_and_execution_preserve_nested_schema_results_and_per_user_headers(enabled, monkeypatch, use_sse):
|
||||||
|
calls = []
|
||||||
|
original = httpx.AsyncClient
|
||||||
|
|
||||||
|
def upstream(request):
|
||||||
|
calls.append(request)
|
||||||
|
if request.method == "GET":
|
||||||
|
return httpx.Response(405)
|
||||||
|
message = json.loads(request.content)
|
||||||
|
method = message["method"]
|
||||||
|
if "id" not in message:
|
||||||
|
return httpx.Response(202)
|
||||||
|
if method == "initialize":
|
||||||
|
result = {
|
||||||
|
"protocolVersion": "2025-11-25",
|
||||||
|
"capabilities": {"tools": {}},
|
||||||
|
"serverInfo": {"name": "test", "version": "1"},
|
||||||
|
}
|
||||||
|
elif method == "tools/list":
|
||||||
|
result = {
|
||||||
|
"tools": [
|
||||||
|
{
|
||||||
|
"name": "get_report",
|
||||||
|
"inputSchema": {"type": "object", "properties": {"request": {"type": "object"}}},
|
||||||
|
},
|
||||||
|
{"name": "delete_user", "inputSchema": {"type": "object"}},
|
||||||
|
]
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
assert method == "tools/call"
|
||||||
|
assert message["params"]["arguments"] == {"request": {"report_id": "r1"}}
|
||||||
|
user = request.headers["authorization"].removeprefix("Bearer ")
|
||||||
|
assert request.headers["x-vocero-podcast-id"] == f"report:{user}"
|
||||||
|
result = {
|
||||||
|
"content": [],
|
||||||
|
"structuredContent": {"result": {"status": "found", "caller": request.headers["authorization"]}},
|
||||||
|
}
|
||||||
|
body = {"jsonrpc": "2.0", "id": message["id"], "result": result}
|
||||||
|
if use_sse:
|
||||||
|
return httpx.Response(
|
||||||
|
200, text=f"event: message\ndata: {json.dumps(body)}\n\n", headers={"content-type": "text/event-stream"}
|
||||||
|
)
|
||||||
|
return httpx.Response(200, json=body)
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mcp.httpx, "AsyncClient", lambda **kwargs: original(transport=httpx.MockTransport(upstream), **kwargs)
|
||||||
|
)
|
||||||
|
|
||||||
|
async def scenario():
|
||||||
|
catalog = await mcp.discover("alice")
|
||||||
|
assert [tool.name for tool in catalog] == ["get_report"]
|
||||||
|
assert catalog[0].parameters["properties"]["request"] == {"type": "object"}
|
||||||
|
outputs = await asyncio.gather(
|
||||||
|
*(
|
||||||
|
mcp.execute(user, "get_report", {"request": {"report_id": "r1"}}, podcast_id=f"report:{user}")
|
||||||
|
for user in ("alice", "bob")
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert [output["data"]["result"]["caller"] for output in outputs] == ["Bearer alice", "Bearer bob"]
|
||||||
|
|
||||||
|
asyncio.run(scenario())
|
||||||
|
assert all(request.headers.get("authorization") in {"Bearer alice", "Bearer bob"} for request in calls)
|
||||||
|
assert all("x-demo-session" not in request.headers and "x-api-key" not in request.headers for request in calls)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("status", [401, 403, 500])
|
||||||
|
def test_mcp_transport_errors_are_safe_and_preserve_authentication_status(enabled, monkeypatch, status):
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mcp,
|
||||||
|
"http_client",
|
||||||
|
lambda token, podcast_id=None: httpx.AsyncClient(
|
||||||
|
transport=httpx.MockTransport(lambda req: httpx.Response(status, text="secret-upstream-body"))
|
||||||
|
),
|
||||||
|
)
|
||||||
|
with pytest.raises(IntegrationError) as error:
|
||||||
|
asyncio.run(mcp.discover("secret-token"))
|
||||||
|
assert error.value.status == (status if status in {401, 403} else 502)
|
||||||
|
assert "secret" not in str(error.value)
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
"""Validate owned fields without tightening or rewriting upstream payloads."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from pydantic import ValidationError
|
||||||
|
|
||||||
|
from vocero_api.api.responses import validated_json_response
|
||||||
|
from vocero_api.models.requests import QueueEndPayload, SearchRequest
|
||||||
|
from vocero_api.models.responses import (
|
||||||
|
AllocationResponse,
|
||||||
|
QueueCancellationResponse,
|
||||||
|
ReasoningResponse,
|
||||||
|
ReportsResponse,
|
||||||
|
SearchResponse,
|
||||||
|
UserProfileResponse,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("model", "payload"),
|
||||||
|
[
|
||||||
|
(AllocationResponse, {}),
|
||||||
|
(AllocationResponse, {"state": [1, None], "connectUrl": "wss://host", "future": {"x": 1.0}}),
|
||||||
|
(ReasoningResponse, {"answer": "á", "future": [1, None, True]}),
|
||||||
|
(SearchResponse, {"query": "q", "answer": {"x": 2}, "results": [{"title": [], "snippet": 7, "url": None}]}),
|
||||||
|
(
|
||||||
|
ReportsResponse,
|
||||||
|
{"count": 1, "wrappedLen": 2, "hints": [], "entries": [{"sourceScheduleLabel": 42, "date": None}]},
|
||||||
|
),
|
||||||
|
*[(UserProfileResponse, payload) for payload in [{}, [], 42, False, None, "profile"]],
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_upstream_values_and_missing_fields_keep_exact_wire_bytes(model, payload):
|
||||||
|
original = JSONResponse(payload, status_code=202)
|
||||||
|
validated = validated_json_response(payload, model, status_code=202)
|
||||||
|
assert validated.body == original.body
|
||||||
|
assert validated.raw_headers == original.raw_headers
|
||||||
|
assert validated.status_code == original.status_code
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("payload", [{}, {"ok": "true"}, {"ok": 1}, {"ok": None}])
|
||||||
|
def test_owned_response_fields_are_actually_validated(payload):
|
||||||
|
with pytest.raises(ValidationError):
|
||||||
|
validated_json_response(payload, QueueCancellationResponse)
|
||||||
|
|
||||||
|
|
||||||
|
def test_request_models_keep_legacy_extra_field_and_beacon_behavior():
|
||||||
|
assert SearchRequest.model_validate({"query": "q", "future": True}).model_dump() == {"query": "q", "key": None}
|
||||||
|
for value in [None, "id", 42, False, [], {}, {"future": True}]:
|
||||||
|
assert QueueEndPayload.model_validate({"queueId": value, "future": 1}).queueId == value
|
||||||
|
assert QueueEndPayload.model_validate({}).queueId == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_http_routes_publish_response_schemas():
|
||||||
|
# Import the app only after the compatibility suite's isolated configuration.
|
||||||
|
from vocero_api.main import app
|
||||||
|
|
||||||
|
spec = app.openapi()
|
||||||
|
for path, operations in spec["paths"].items():
|
||||||
|
if path.startswith("/api/"):
|
||||||
|
for operation in operations.values():
|
||||||
|
if "204" in operation["responses"]:
|
||||||
|
assert path in {"/api/conversations/current", "/api/conversations/current/interrupt"}
|
||||||
|
assert "content" not in operation["responses"]["204"]
|
||||||
|
continue
|
||||||
|
schema = operation["responses"]["200"]["content"]["application/json"]["schema"]
|
||||||
|
assert "$ref" in schema, path
|
||||||
|
allocation = spec["components"]["schemas"]["AllocationResponse"]
|
||||||
|
assert not allocation.get("required")
|
||||||
|
assert allocation["additionalProperties"] is True
|
||||||
|
config = spec["components"]["schemas"]["ConfigResponse"]
|
||||||
|
assert config["properties"]["reasoningType"]["enum"] == ["PROXY", "OPENAI"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_legacy_entry_point_exports_the_renamed_application():
|
||||||
|
import server
|
||||||
|
from vocero_api.main import app
|
||||||
|
|
||||||
|
assert server.app is app
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("value", "expected"),
|
||||||
|
[
|
||||||
|
("OPENAI", "OPENAI"),
|
||||||
|
(" 'openai' ", "OPENAI"),
|
||||||
|
('"OPENAI"', "OPENAI"),
|
||||||
|
("PROXY", "PROXY"),
|
||||||
|
("", "PROXY"),
|
||||||
|
("UNSUPPORTED", "PROXY"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_reasoning_configuration_preserves_normalization_and_native_fallback(value, expected):
|
||||||
|
from vocero_api.core.config import _normalize_reasoning_type
|
||||||
|
|
||||||
|
assert _normalize_reasoning_type(value) == expected
|
||||||
@@ -0,0 +1,290 @@
|
|||||||
|
"""Backend contract checks with local HTTP fixtures and a loopback voice server.
|
||||||
|
|
||||||
|
Run from vocero-api: uv run --locked pytest tests
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
import unittest
|
||||||
|
from unittest.mock import AsyncMock, Mock, patch
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from websockets.sync.server import serve
|
||||||
|
|
||||||
|
# Never consume deployment credentials or contact services from a developer's .env.
|
||||||
|
with patch.dict(os.environ, {}, clear=True), patch("dotenv.load_dotenv", return_value=False):
|
||||||
|
from vocero_api import main as server
|
||||||
|
from vocero_api.core import config as settings
|
||||||
|
from vocero_api.providers import caller
|
||||||
|
|
||||||
|
|
||||||
|
GRANT = {
|
||||||
|
"session_id": "local-session",
|
||||||
|
"connect_url": "ws://voice.test/v1/realtime?token=fixture",
|
||||||
|
"pending_timeout_s": 60,
|
||||||
|
}
|
||||||
|
TICKET = {"state": "queued", "queue_id": "local-ticket", "position": 2, "poll_interval_s": 1}
|
||||||
|
ASYNC_CLIENT = httpx.AsyncClient
|
||||||
|
|
||||||
|
|
||||||
|
class BackendContractTests(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.upstream = Mock(side_effect=AssertionError("Unexpected upstream HTTP request"))
|
||||||
|
self.enterContext(patch.object(settings, "LOAD_BALANCER_URL", "http://lb.test"))
|
||||||
|
self.enterContext(
|
||||||
|
patch.object(
|
||||||
|
httpx,
|
||||||
|
"AsyncClient",
|
||||||
|
side_effect=lambda **kwargs: ASYNC_CLIENT(
|
||||||
|
transport=httpx.MockTransport(self.upstream),
|
||||||
|
**kwargs,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.client = self.enterContext(TestClient(server.app))
|
||||||
|
|
||||||
|
def respond(self, *responses):
|
||||||
|
self.upstream.side_effect = iter(responses)
|
||||||
|
|
||||||
|
def test_bootstrap_and_removed_account_contract(self):
|
||||||
|
response = self.client.get("/api/config")
|
||||||
|
self.assertEqual(response.status_code, 200)
|
||||||
|
self.assertTrue(response.json()["lb"])
|
||||||
|
self.assertFalse(response.json()["allowDirect"])
|
||||||
|
self.assertNotIn("auth", response.json())
|
||||||
|
paths = self.client.get("/openapi.json").json()["paths"]
|
||||||
|
for method, path in (
|
||||||
|
("GET", "/api/me"),
|
||||||
|
("POST", "/api/session/heartbeat"),
|
||||||
|
("POST", "/api/session/end"),
|
||||||
|
):
|
||||||
|
self.assertNotIn(path, paths)
|
||||||
|
self.assertEqual(self.client.request(method, path).status_code, 404)
|
||||||
|
self.assertFalse(any(getattr(route, "path", "").startswith("/oauth") for route in server.app.routes))
|
||||||
|
self.upstream.assert_not_called()
|
||||||
|
|
||||||
|
def test_only_documentation_is_served_as_static_content(self):
|
||||||
|
for path in ("/", "/index.html", "/main.js", "/style.css", "/server.py", "/requirements.txt"):
|
||||||
|
self.assertEqual(self.client.get(path).status_code, 404)
|
||||||
|
self.assertEqual(self.client.get("/docs").status_code, 200)
|
||||||
|
self.assertEqual(self.client.get("/docs/README.md").status_code, 200)
|
||||||
|
for path in ("/project/", "/public/docs.html"):
|
||||||
|
response = self.client.get(path, follow_redirects=False)
|
||||||
|
self.assertEqual(response.status_code, 307)
|
||||||
|
self.assertEqual(response.headers["location"], "/docs/proposals/vocero-architecture.html")
|
||||||
|
self.upstream.assert_not_called()
|
||||||
|
|
||||||
|
def test_session_grant_passes_through_without_extra_fields_or_cookies(self):
|
||||||
|
self.respond(httpx.Response(200, json=GRANT))
|
||||||
|
response = self.client.post("/api/session", json={})
|
||||||
|
self.assertEqual(response.status_code, 200)
|
||||||
|
self.assertEqual(response.json(), GRANT)
|
||||||
|
self.assertNotIn("set-cookie", response.headers)
|
||||||
|
request = self.upstream.call_args.args[0]
|
||||||
|
self.assertEqual((request.method, str(request.url)), ("POST", "http://lb.test/session"))
|
||||||
|
self.assertEqual(json.loads(request.content), {})
|
||||||
|
|
||||||
|
def test_queue_polling_returns_positions_and_then_grant(self):
|
||||||
|
self.respond(
|
||||||
|
httpx.Response(200, json=TICKET),
|
||||||
|
httpx.Response(200, json={**TICKET, "position": 1}),
|
||||||
|
httpx.Response(200, json=GRANT),
|
||||||
|
)
|
||||||
|
self.assertEqual(self.client.post("/api/session").json(), TICKET)
|
||||||
|
self.assertEqual(self.client.get("/api/queue/local-ticket").json(), {**TICKET, "position": 1})
|
||||||
|
response = self.client.get("/api/queue/local-ticket")
|
||||||
|
self.assertEqual(response.json(), GRANT)
|
||||||
|
self.assertNotIn("set-cookie", response.headers)
|
||||||
|
self.assertEqual(
|
||||||
|
[(call.args[0].method, call.args[0].url.path) for call in self.upstream.call_args_list],
|
||||||
|
[("POST", "/session"), ("GET", "/queue/local-ticket"), ("GET", "/queue/local-ticket")],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_capacity_and_expired_queue_responses_remain(self):
|
||||||
|
self.respond(httpx.Response(503, json={"state": "at_capacity"}), httpx.Response(404))
|
||||||
|
response = self.client.post("/api/session")
|
||||||
|
self.assertEqual((response.status_code, response.json()), (503, {"state": "at_capacity"}))
|
||||||
|
response = self.client.get("/api/queue/local-ticket")
|
||||||
|
self.assertEqual((response.status_code, response.json()), (404, {"state": "expired"}))
|
||||||
|
|
||||||
|
def test_explicit_and_beacon_queue_cancellation(self):
|
||||||
|
self.respond(httpx.Response(204), httpx.Response(204))
|
||||||
|
self.assertEqual(self.client.delete("/api/queue/local-ticket").json(), {"ok": True})
|
||||||
|
# sendBeacon posts a JSON blob, not a form or a session identifier.
|
||||||
|
response = self.client.post(
|
||||||
|
"/api/queue/end",
|
||||||
|
content=json.dumps({"queueId": "local-ticket"}),
|
||||||
|
headers={"Content-Type": "application/json"},
|
||||||
|
)
|
||||||
|
self.assertEqual(response.json(), {"ok": True})
|
||||||
|
for call in self.upstream.call_args_list:
|
||||||
|
self.assertEqual((call.args[0].method, call.args[0].url.path), ("DELETE", "/queue/local-ticket"))
|
||||||
|
self.assertEqual(self.upstream.call_count, 2)
|
||||||
|
|
||||||
|
def test_empty_beacon_is_harmless_and_cancellation_is_best_effort(self):
|
||||||
|
self.assertEqual(self.client.post("/api/queue/end", content="").json(), {"ok": True})
|
||||||
|
self.upstream.assert_not_called()
|
||||||
|
self.upstream.side_effect = httpx.ConnectError("Local fixture unavailable")
|
||||||
|
self.assertEqual(self.client.delete("/api/queue/local-ticket").json(), {"ok": True})
|
||||||
|
|
||||||
|
def test_upstream_failures_are_still_reported(self):
|
||||||
|
self.upstream.side_effect = httpx.ConnectError("Local fixture unavailable")
|
||||||
|
self.assertEqual(self.client.post("/api/session").status_code, 502)
|
||||||
|
self.assertEqual(self.client.get("/api/queue/local-ticket").status_code, 502)
|
||||||
|
self.respond(httpx.Response(429), httpx.Response(500))
|
||||||
|
self.assertEqual(self.client.post("/api/session").status_code, 502)
|
||||||
|
self.assertEqual(self.client.get("/api/queue/local-ticket").status_code, 502)
|
||||||
|
|
||||||
|
def test_direct_mode_disables_lb_and_preserves_pinned_voice_url(self):
|
||||||
|
with (
|
||||||
|
patch.object(settings, "LOAD_BALANCER_URL", ""),
|
||||||
|
patch.object(
|
||||||
|
settings,
|
||||||
|
"SPEECH_TO_SPEECH_URL",
|
||||||
|
GRANT["connect_url"],
|
||||||
|
),
|
||||||
|
):
|
||||||
|
config = self.client.get("/api/config").json()
|
||||||
|
self.assertEqual(config["s2sUrl"], GRANT["connect_url"])
|
||||||
|
self.assertFalse(config["lb"])
|
||||||
|
self.assertTrue(config["allowDirect"])
|
||||||
|
for method, path in (
|
||||||
|
("POST", "/api/session"),
|
||||||
|
("GET", "/api/queue/local-ticket"),
|
||||||
|
("DELETE", "/api/queue/local-ticket"),
|
||||||
|
("POST", "/api/queue/end"),
|
||||||
|
):
|
||||||
|
self.assertEqual(self.client.request(method, path).status_code, 404)
|
||||||
|
self.upstream.assert_not_called()
|
||||||
|
|
||||||
|
def test_caller_profile_still_uses_upstream_login_token(self):
|
||||||
|
profile = {"result": {"name": "Local Caller"}}
|
||||||
|
self.respond(
|
||||||
|
httpx.Response(200, json={"token": "fixture-token"}),
|
||||||
|
httpx.Response(200, json=profile),
|
||||||
|
)
|
||||||
|
with (
|
||||||
|
patch.object(settings, "S2S_LOGIN_BY_PHONE", "http://profile.test/login"),
|
||||||
|
patch.object(
|
||||||
|
settings,
|
||||||
|
"VOCERO_USER_PHONE_AUTH_URL",
|
||||||
|
"http://profile.test/user",
|
||||||
|
),
|
||||||
|
):
|
||||||
|
response = self.client.get("/api/user/phone", params={"phoneNumber": "+56000000000"})
|
||||||
|
self.assertEqual(response.json(), profile)
|
||||||
|
login, lookup = [call.args[0] for call in self.upstream.call_args_list]
|
||||||
|
self.assertEqual(json.loads(login.content), {"phoneNumber": "+56000000000"})
|
||||||
|
self.assertEqual(lookup.headers["authorization"], "Bearer fixture-token")
|
||||||
|
self.assertEqual(lookup.url.params["phoneNumber"], "+56000000000")
|
||||||
|
|
||||||
|
def test_apipilot_routes_and_contracts_are_retired(self):
|
||||||
|
self.assertNotIn("apipilot", self.client.get("/api/config").json())
|
||||||
|
spec = self.client.get("/openapi.json").json()
|
||||||
|
for method, path in (("GET", "/api/apipilot/tools"), ("POST", "/api/apipilot/execute")):
|
||||||
|
self.assertNotIn(path, spec["paths"])
|
||||||
|
response = self.client.request(method, path, json={"name": "get_report"})
|
||||||
|
self.assertEqual(response.status_code, 404)
|
||||||
|
schemas = spec["components"]["schemas"]
|
||||||
|
for name in ("ApipilotExecuteRequest", "ToolCatalogResponse", "ToolExecutionResponse", "ToolInvocation"):
|
||||||
|
self.assertNotIn(name, schemas)
|
||||||
|
self.assertNotIn("apipilot", schemas["ConfigResponse"]["properties"])
|
||||||
|
self.assertEqual(set(schemas["ReasoningResponse"]["properties"]), {"answer"})
|
||||||
|
self.upstream.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
class VoiceProxyTests(unittest.TestCase):
|
||||||
|
def test_caller_and_episode_context_reach_the_voice_session(self):
|
||||||
|
closed = threading.Event()
|
||||||
|
login = AsyncMock(return_value=("fixture-token", {}))
|
||||||
|
profile = AsyncMock(return_value={"user": {"name": "Local Caller", "email": "caller@example.test"}})
|
||||||
|
episode = AsyncMock(return_value={"title": "Local episode", "text": "Local episode text"})
|
||||||
|
|
||||||
|
def voice(socket):
|
||||||
|
try:
|
||||||
|
socket.send(json.dumps({"type": "session.created"}))
|
||||||
|
for frame in socket:
|
||||||
|
socket.send(frame)
|
||||||
|
finally:
|
||||||
|
closed.set()
|
||||||
|
|
||||||
|
with serve(voice, "127.0.0.1", 0) as upstream:
|
||||||
|
thread = threading.Thread(target=upstream.serve_forever, daemon=True)
|
||||||
|
thread.start()
|
||||||
|
self.addCleanup(thread.join, 5)
|
||||||
|
target = f"ws://127.0.0.1:{upstream.socket.getsockname()[1]}/v1/realtime"
|
||||||
|
with (
|
||||||
|
patch.object(settings, "S2S_PROXY_TARGET", target),
|
||||||
|
patch.object(settings, "S2S_INJECT_PROFILE", True),
|
||||||
|
patch.object(settings, "S2S_LOGIN_BY_PHONE", "http://profile.test/login"),
|
||||||
|
patch.object(settings, "VOCERO_USER_PHONE_AUTH_URL", "http://profile.test/user"),
|
||||||
|
patch.object(caller, "_login_by_phone", login),
|
||||||
|
patch.object(caller, "_fetch_user_profile_for_phone", profile),
|
||||||
|
patch.object(caller, "_fetch_podcast_episode", episode),
|
||||||
|
patch.object(httpx, "AsyncClient", side_effect=AssertionError("Unexpected HTTP request")),
|
||||||
|
TestClient(server.app) as client,
|
||||||
|
):
|
||||||
|
with client.websocket_connect(
|
||||||
|
"/v1/realtime?phoneNumber=%2B56000000000&episode_id=fixture-episode",
|
||||||
|
) as socket:
|
||||||
|
self.assertEqual(socket.receive_json(), {"type": "session.created"})
|
||||||
|
socket.send_json({"type": "session.update", "session": {"instructions": "Local fixture"}})
|
||||||
|
update = socket.receive_json()
|
||||||
|
self.assertEqual(update["type"], "session.update")
|
||||||
|
instructions = update["session"]["instructions"]
|
||||||
|
for expected in (
|
||||||
|
"Local fixture",
|
||||||
|
"Local Caller",
|
||||||
|
"Local episode text",
|
||||||
|
"caller@example.test",
|
||||||
|
):
|
||||||
|
self.assertIn(expected, instructions)
|
||||||
|
# Configuration injection must preserve ordinary audio forwarding.
|
||||||
|
audio = {"type": "input_audio_buffer.append", "audio": "AAA="}
|
||||||
|
socket.send_json(audio)
|
||||||
|
self.assertEqual(socket.receive_json(), audio)
|
||||||
|
self.assertTrue(closed.wait(3), "Client disconnect must close the upstream WebSocket")
|
||||||
|
login.assert_awaited_once_with("+56000000000")
|
||||||
|
profile.assert_awaited_once_with("+56000000000", auth_token="fixture-token")
|
||||||
|
episode.assert_awaited_once_with("fixture-episode")
|
||||||
|
|
||||||
|
def test_proxy_voice_frames_keepalive_and_client_disconnect(self):
|
||||||
|
closed = threading.Event()
|
||||||
|
received = []
|
||||||
|
keepalive = threading.Event()
|
||||||
|
|
||||||
|
def voice(socket):
|
||||||
|
try:
|
||||||
|
socket.send(json.dumps({"type": "session.created"}))
|
||||||
|
if socket.ping().wait(3):
|
||||||
|
keepalive.set()
|
||||||
|
for frame in socket:
|
||||||
|
received.append(frame)
|
||||||
|
socket.send(frame)
|
||||||
|
finally:
|
||||||
|
closed.set()
|
||||||
|
|
||||||
|
with serve(voice, "127.0.0.1", 0) as upstream:
|
||||||
|
thread = threading.Thread(target=upstream.serve_forever, daemon=True)
|
||||||
|
thread.start()
|
||||||
|
self.addCleanup(thread.join, 5)
|
||||||
|
target = f"ws://127.0.0.1:{upstream.socket.getsockname()[1]}/v1/realtime"
|
||||||
|
with patch.object(settings, "S2S_PROXY_TARGET", target), TestClient(server.app) as client:
|
||||||
|
config = client.get("/api/config", headers={"host": "localhost:19420"}).json()
|
||||||
|
self.assertEqual(config["s2sUrl"], "ws://localhost:19420/v1/realtime")
|
||||||
|
with client.websocket_connect("/v1/realtime") as socket:
|
||||||
|
self.assertEqual(socket.receive_json(), {"type": "session.created"})
|
||||||
|
update = {"type": "session.update", "session": {"instructions": "Local fixture"}}
|
||||||
|
socket.send_json(update)
|
||||||
|
self.assertEqual(socket.receive_json(), update)
|
||||||
|
socket.send_bytes(b"\x00\x01\x02\x03")
|
||||||
|
self.assertEqual(socket.receive_bytes(), b"\x00\x01\x02\x03")
|
||||||
|
self.assertTrue(keepalive.wait(3))
|
||||||
|
self.assertTrue(closed.wait(3), "Client disconnect must close the upstream WebSocket")
|
||||||
|
self.assertEqual(len(received), 2)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
Generated
+879
@@ -0,0 +1,879 @@
|
|||||||
|
version = 1
|
||||||
|
revision = 3
|
||||||
|
requires-python = "==3.12.*"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "annotated-types"
|
||||||
|
version = "0.8.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/5f/56/a8120250d128bed162cd73c76d45f6ef9991f3e068f62a8ee060afa3104a/annotated_types-0.8.0.tar.gz", hash = "sha256:13b2beaad985e05e2d6407ee4c4f35590b11f8d693a258a561055cac8f64cab7", size = 15893, upload-time = "2026-07-23T20:16:13.995Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/99/91/8acff4f5e50511b911bbccb72b8628a49c68ce14148cd9f6431094859a90/annotated_types-0.8.0-py3-none-any.whl", hash = "sha256:f072f4d804ea359e4eaf198b1af7a8b0943881a87f31bb764f8bf219bb9419e0", size = 13427, upload-time = "2026-07-23T20:16:12.938Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "anyio"
|
||||||
|
version = "4.15.1"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "idna" },
|
||||||
|
{ name = "typing-extensions" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/a9/d2/f4d173e22df740bc37b1db102b386ba719b66e95b0f0d751f556b387e6d2/anyio-4.15.1.tar.gz", hash = "sha256:9f28306018cbd6d329e64a36d58256edff76dd996fe423bc957326e578b82a94", size = 276966, upload-time = "2026-09-05T10:42:39.44Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/12/b8/4bd346e22b28902df4d651910f5242c28d84e4a5c2435ca5c3f797ed7e2e/anyio-4.15.1-py3-none-any.whl", hash = "sha256:6152fdbbf9a77fdec97731721bebf7c4c44f7c29b424b0065826173efc7ed101", size = 132079, upload-time = "2026-09-05T10:42:37.923Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "attrs"
|
||||||
|
version = "26.1.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/9a/8e/82a0fe20a541c03148528be8cac2408564a6c9a0cc7e9171802bc1d26985/attrs-26.1.0.tar.gz", hash = "sha256:d03ceb89cb322a8fd706d4fb91940737b6642aa36998fe130a9bc96c985eff32", size = 952055, upload-time = "2026-03-19T14:22:25.026Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/64/b4/17d4b0b2a2dc85a6df63d1157e028ed19f90d4cd97c36717afef2bc2f395/attrs-26.1.0-py3-none-any.whl", hash = "sha256:c647aa4a12dfbad9333ca4e71fe62ddc36f4e63b2d260a37a8b83d2f043ac309", size = 67548, upload-time = "2026-03-19T14:22:23.645Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "certifi"
|
||||||
|
version = "2026.7.22"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/a3/c2/24167ea9858356b47a87a50d39908bfdb72ceeefe0041586e704e5376b3a/certifi-2026.7.22.tar.gz", hash = "sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55", size = 138112, upload-time = "2026-07-22T03:35:12.644Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/0b/a7/71ac2cff56fec219ed242bb11b8efb69fcc4bec75db06fb7bfe35de520e6/certifi-2026.7.22-py3-none-any.whl", hash = "sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775", size = 136983, upload-time = "2026-07-22T03:35:11.276Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "cffi"
|
||||||
|
version = "2.1.1"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "pycparser", marker = "implementation_name != 'PyPy'" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/9e/ef/008a1939e372c06329a3fce4279c02f328488f3526744906eeec3da7ad5f/cffi-2.1.1.tar.gz", hash = "sha256:dd31f52ea1086513bb9df30f8fcee9b8918323ae067a3d5b78bc826a000712be", size = 530807, upload-time = "2026-08-03T21:21:18.939Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/10/69/43965eccfdead3b9220015fd1320e117be8c6ed01a62ffab76eeb752f5d5/cffi-2.1.1-cp312-cp312-macosx_10_15_x86_64.whl", hash = "sha256:c8c69575568085ba0b1b10c0249d779a214aea6f6522e949a0fc9fb0fcb449d0", size = 184821, upload-time = "2026-08-03T21:19:44.887Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/54/7d/16e5a096677b5e313ca80cd5e5170efa3ea44624a82bb111925522da64b1/cffi-2.1.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f81b3b8f3d4e343550fa4baa0e479bba9f2d29ce9c2e9b51d1ce1718d7442fcf", size = 184719, upload-time = "2026-08-03T21:19:46.129Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/56/e6/8941622732edec876dd17d0453dce07317ae96db34f2ec1436c9d3785986/cffi-2.1.1-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:811bd1e21d32de12efca32393a0ab3f5133b54fce9bd44b8bd77ab07da14bf6a", size = 214799, upload-time = "2026-08-03T21:19:47.218Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/44/de/f98430906df1545ffde0d543dd124a7a439bc2cd32b36b9c53f805df7333/cffi-2.1.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:68e62fe11f30d5ca8289242866f0a5291402d8529ca2178ab8afc5c9694ae890", size = 222389, upload-time = "2026-08-03T21:19:48.331Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/6a/5b/717f1526b9957b34456313c31645c5b82b8fb5c3fe9e4752999be7128bfc/cffi-2.1.1-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:4a7c934f7360e8cd64fe9efadcbd10c7c6364f531e432b9a4bf5ccbc9e0e8b50", size = 210249, upload-time = "2026-08-03T21:19:49.543Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/64/b3/f8aa4f3e34986c7e4ec45072d1b1b9dd295b6b18007b45518d79726dd725/cffi-2.1.1-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:3143d81e29e1e20a9ce10901ec369012947876596f75a222235965f2b7ae832e", size = 208775, upload-time = "2026-08-03T21:19:50.918Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b1/db/dceb9dd5b231e1da801793f8acc9f3c52a7e1afe40bb1aae37e02b0faad5/cffi-2.1.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c1453022f490d2459a11819d83ad1d586e9ff65a12ac3e705ffebd46d3685dcf", size = 221822, upload-time = "2026-08-03T21:19:52.054Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a0/d2/6cd24ae3be000a634109c247d1475d62e5616d0dc78c82770942ec384248/cffi-2.1.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:208f941bb9d18e768138677f0a6d2ce01f590df56043dda1df1535ac57c88517", size = 225232, upload-time = "2026-08-03T21:19:53.109Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/cb/52/3fa190537004dd7f0ab860a6dc7c0175b8667f68d1e618a46f5498d30250/cffi-2.1.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:210019b6c7cf07f081b4c54635c8cf744377001350e29cc0f81c4377b4797735", size = 223597, upload-time = "2026-08-03T21:19:54.515Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/80/fb/0bb75b7039588c074b37ae99f40d9bfddf990ecb2fbc346ebccd2e56b9be/cffi-2.1.1-cp312-cp312-win32.whl", hash = "sha256:046bfc24911b37851ee1b51aab8bffe713d89c68c6a057b09484ce9fd5f69b4e", size = 175292, upload-time = "2026-08-03T21:19:55.566Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d9/79/615cc094e2fb508cade7de88d3b4f6c4ec2bab695c97bce9153dc65aadf5/cffi-2.1.1-cp312-cp312-win_amd64.whl", hash = "sha256:f53e442b08449d42821fa4a4fba000095af9f62742a500f978a9f557ec44339a", size = 185919, upload-time = "2026-08-03T21:19:56.89Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/70/c6/d0ea84713fe46b243a436a18fcd47d639732747e21635c8a27191b06dc30/cffi-2.1.1-cp312-cp312-win_arm64.whl", hash = "sha256:7bde5e4cc5c10140859842b9d383af292b22639a4dffb725314baf45968cef80", size = 180093, upload-time = "2026-08-03T21:19:58.155Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "cfgv"
|
||||||
|
version = "3.5.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/4e/b5/721b8799b04bf9afe054a3899c6cf4e880fcf8563cc71c15610242490a0c/cfgv-3.5.0.tar.gz", hash = "sha256:d5b1034354820651caa73ede66a6294d6e95c1b00acc5e9b098e917404669132", size = 7334, upload-time = "2025-11-19T20:55:51.612Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/db/3c/33bac158f8ab7f89b2e59426d5fe2e4f63f7ed25df84c036890172b412b5/cfgv-3.5.0-py2.py3-none-any.whl", hash = "sha256:a8dc6b26ad22ff227d2634a65cb388215ce6cc96bbcc5cfde7641ae87e8dacc0", size = 7445, upload-time = "2025-11-19T20:55:50.744Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "click"
|
||||||
|
version = "8.5.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/c7/0e/7fa0ef50764b67090eca4114772a2abf8b6148198475e54c660b97caeee6/click-8.5.0.tar.gz", hash = "sha256:ba0d2089de75ea0310e2dde03160e6ca10009947fb95a182f9b54021bb272e34", size = 382235, upload-time = "2026-08-26T13:33:14.56Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/58/50/6c0d534c5f134586a8e1ba4e330569e32f057e33372ae556463212fb4cd3/click-8.5.0-py3-none-any.whl", hash = "sha256:255bc9599cf7748b4b1a446ccc735421bd08a2ae529a8b88597d3de5664ee360", size = 125251, upload-time = "2026-08-26T13:33:12.928Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "colorama"
|
||||||
|
version = "0.4.6"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "cryptography"
|
||||||
|
version = "50.0.2"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "cffi", marker = "platform_python_implementation != 'PyPy'" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/9d/af/182eb91b0df3fe75c4d9f26fe70684569566745f6ba7e5c9c73a862c5252/cryptography-50.0.2.tar.gz", hash = "sha256:7b46165bb56eb4704e2eaaf86f3c940d19154535d9b0ca7d6d590b04060e00d5", size = 880623, upload-time = "2026-09-30T15:30:04.884Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e5/56/d194340cc4a57535e82e1bee9e89667ac4b7c13b5d3f59686deae3094dd5/cryptography-50.0.2-cp311-abi3-macosx_11_0_arm64.whl", hash = "sha256:fa8f5efb344d6908a1ce62f4a24e2e5780f825d6f53f5f50ec5ffacac72936cb", size = 3914904, upload-time = "2026-09-30T14:43:44.339Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d9/69/c9bd862c3bf43d6399c433caf002df16e2dffd4be49bdf515cda38038711/cryptography-50.0.2-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:79def8d059362e7831389ed3be0ecdf58a89386e1271e35dd9f5af84e81bffd0", size = 4731146, upload-time = "2026-09-30T14:43:47.113Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/21/69/64cef1f702bf6657e0cc186ed1a2891d50d29fb41586b254e1c07adea261/cryptography-50.0.2-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:630ebfea3bf689d075f82316324ff7433dc447fe6bc1bfc76524b74b4a9567d2", size = 4719841, upload-time = "2026-09-30T14:43:49.01Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/38/6b/61a3f8d8c5e1e49a6cddccafc4015cc1c0021360ab0acb4080e7a423644a/cryptography-50.0.2-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:f9f6143a8c75945eb960d9eb98905a441394abfa24afaae239d514ffb2586480", size = 4738340, upload-time = "2026-09-30T14:43:50.932Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7b/2e/7212ca32fd43dc91f2f41db20160b268098874b4c9a0e7be94d6835f5b2e/cryptography-50.0.2-cp311-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:a582ab2ae1d34f67112cadc86702774c9ea4374df6bca6afe672817203c99134", size = 5367029, upload-time = "2026-09-30T14:43:52.911Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1a/f1/b474e930c4d910328780e3940da76f5aa5cbc48ce1fc14e44d239d9ea9db/cryptography-50.0.2-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:4061c0079120205fb760c58acab6443e217307dcf05e3702cf970e0689972856", size = 4753050, upload-time = "2026-09-30T14:43:55.272Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7c/52/9af10e80ac16b0fcc2123f9cbd5e7afbd0fd5075bb7a607c592258a39cda/cryptography-50.0.2-cp311-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:ac9ed99d81760c62fe89d5f0815cdfa1ba9a35141cf30f1c2d044f04b4803d2e", size = 4376724, upload-time = "2026-09-30T14:43:57.24Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/71/37/6202e488cc1eb625ea110c292c6bda92823176e023f427d8d5660ce8d632/cryptography-50.0.2-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:87e9ce85beb6b328ba370cc6e6aea483c92617b4c95b1d33a49297eb662bfb04", size = 4737859, upload-time = "2026-09-30T14:43:59.541Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8f/30/e86d7d518489b0ae2497091a35287abcb1a2ce4037837a34afbe9b1d6964/cryptography-50.0.2-cp311-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:f265528741e048bce55c3463ed721fb0aa45a5888d8add8cfeccb3035451bbdc", size = 5324103, upload-time = "2026-09-30T14:44:01.901Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d3/69/2c833a049475e0a3444e94c7d0aca0aa51d166374a449b09e92ac98138de/cryptography-50.0.2-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:9dab55f57c74c3cad24c323bacbbd04be4705ba6eb0d92e920b1fc4837ed5079", size = 4752576, upload-time = "2026-09-30T14:44:04.545Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/6c/5d/906970b83bbfc1f5bbfb677a143c181f2801f23b6a7204a3b47c42c97e65/cryptography-50.0.2-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:25784ce8b9621c90c643efb9e1e2162ab3b0224cae446ad5e70e7fcb1ce18b51", size = 4870819, upload-time = "2026-09-30T14:44:06.884Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/68/e3/f2298d3bb55e0c4a91841ec4d01b3f020ba8c5fbf15ccdcc6dcf03f97025/cryptography-50.0.2-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:85d0d9a31b9098e98534226d5686b47264b95e62ce459dc2e62fdfc809f9fe93", size = 5030152, upload-time = "2026-09-30T14:44:09.443Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/9a/4f/adfc442765721292fff86d314ce385d3249d22db42295c0dd057727b60f3/cryptography-50.0.2-cp311-abi3-win_amd64.whl", hash = "sha256:7afa5a6602a9f29af1f3a2965f831bae7c9d5d597b7cbb716d41ab3b7d89879c", size = 3824692, upload-time = "2026-09-30T14:44:11.671Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2d/49/93f6a6e7a87c9aa68d44d3e1cdb5fe8f60c90d5d2f46acae9a56892816b8/cryptography-50.0.2-cp315-abi3.abi3t-macosx_11_0_arm64.whl", hash = "sha256:edc3342adf8f697fc5f59c887a304356f147b397809440ed64e2fa6af2f50f37", size = 4133708, upload-time = "2026-09-30T14:44:41.807Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8c/75/32ac2a56243d778805c16ca6a32b8f74fb757df7e28d7ecb560afafb59cf/cryptography-50.0.2-cp315-abi3.abi3t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d370b8d1dfcdf7130178137f6fbee6140774a1acc6cacefc4b42643ec11d0a3a", size = 4956267, upload-time = "2026-09-30T14:44:43.693Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/aa/a4/2c8d734e43d97f0842ee9f1b7b4bfb3d0cf5e19edebf43c2afe6675c2320/cryptography-50.0.2-cp315-abi3.abi3t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f2f9bd7f90c64fe89253f0a2c05e3c4856072660429ce8831b4235bf29403a67", size = 4966465, upload-time = "2026-09-30T14:44:45.769Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c2/58/ee288c829a6f41f6235ae9dd33d82fd19b45442b65b4c8a3da36963d9f7a/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_28_aarch64.whl", hash = "sha256:e275096ea1e60cc595cda2836fd4a6c725d1125108b868be17f53684d164e2cc", size = 4959356, upload-time = "2026-09-30T14:44:48.211Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/92/20/9ded6d51ddd9897f6b6e81fb9ebea7951d7cc5d6c890b0ed8abf77a51a80/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_28_ppc64le.whl", hash = "sha256:b13478603dcd0a2479ff8e87e2c19a7d525734686fe3c49542472293a204212d", size = 5548822, upload-time = "2026-09-30T14:44:50.86Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/02/a8/8df951850d6b31d2a00218f19e2b3f999523437ed7a819df7fa427942fca/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_28_x86_64.whl", hash = "sha256:58a0c478eeca76fe5e07993c5a0703def34a6dc6a0cda4f5564639b33112ffe7", size = 5001199, upload-time = "2026-09-30T14:44:53.379Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8b/f9/36b3022218ce75b7cdf068fb95f809f9bd0d820e4955ef43b90c255cc7ac/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_31_armv7l.whl", hash = "sha256:d38cdff612d06fa6a32840d5e1b1f7a27cee4a349aa9085d94a67789d6bfd408", size = 4629333, upload-time = "2026-09-30T14:44:55.635Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8c/72/20f99a219f6af47cdd1cbd978c243b92d71496e168a746138af44ded4f29/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_34_aarch64.whl", hash = "sha256:fdd28f912fccfec1846a94e2e1e8f9b0012f557f0c46fe4f3eb0d7a87afcf90b", size = 4958822, upload-time = "2026-09-30T14:44:59.639Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f2/20/196f112617fb08eb4d608a2a6c422373d46f9cc2857f38fc0667033c0899/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_34_ppc64le.whl", hash = "sha256:cbc8738fd8526d80f35cb3a40d41f41a2e7030bb3b18b09a6778ef63d291c2fd", size = 5506351, upload-time = "2026-09-30T14:45:02.267Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/24/95/83378121ef3eaaaf71d4b781577ff794acb39b9e1b87a3f156898c8497ed/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_34_x86_64.whl", hash = "sha256:e105ab60406787da31fccc883fc0f733af1efd78f0136a4599692c4083a73d0c", size = 5000859, upload-time = "2026-09-30T14:45:05.009Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/22/f7/70fd7ae4d1dbfa7ba29b02e1b9068771519a86027756510b700ce81086a8/cryptography-50.0.2-cp315-abi3.abi3t-musllinux_1_2_aarch64.whl", hash = "sha256:6f8700550aa1474a91e5dc07049c46f98b423b5b1ddd0483e0b51362eeeaf5be", size = 5092151, upload-time = "2026-09-30T15:29:15.932Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d4/be/688367b74de86984bd58d8efacfc7c9e68b89a6a22ced0fb4f38db50254a/cryptography-50.0.2-cp315-abi3.abi3t-musllinux_1_2_x86_64.whl", hash = "sha256:c71be1cbfa5cd9a41ee452acf1eccd82b2c05950358b106ec8ceb83411d1a020", size = 5286120, upload-time = "2026-09-30T15:29:18.309Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/39/d1/55f8a3f2ef5d1529e16835ef10cf0fe3d559ce237b46dddc440c0bba3649/cryptography-50.0.2-cp315-abi3.abi3t-win_amd64.whl", hash = "sha256:c423ab384a46c4dff7217b2ea5ba2e11cffdeab6441acd04cf65a369caf0366c", size = 4111557, upload-time = "2026-09-30T15:29:20.155Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/23/ad/ac987755d00e1e64273760228d2635ae38dae2be83e3c6e0d3289d91dec3/cryptography-50.0.2-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:0ec5f09541743261e66e291b4a0cbf0fb2997aeaab6d9e9c740b9dba1b58d1c2", size = 3943588, upload-time = "2026-09-30T15:29:22.265Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d5/8d/6d585339bedf85d45044c85d8412dac53f2bb6f918e8b7777efba1787844/cryptography-50.0.2-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:c5e67125c7dca78d199ec4e116aa93dbb83494808ecbb8211a2cb09b1bf41dbd", size = 4756166, upload-time = "2026-09-30T15:29:24.58Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/bf/f1/1c1f6874e8550cfddd4b688ceb38cefb6ed15ceed224d56f133f3d88c214/cryptography-50.0.2-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:ee247f5c245c9a2fe7c8e2214e295918838e44e00a45a6718451e4004219e767", size = 4749145, upload-time = "2026-09-30T15:29:26.807Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c1/63/61b15dc1a8de03fe0adbe3fd7608b3ad5c73bf50993bbcb1faaa930afe33/cryptography-50.0.2-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:dfe9763530994147d9af1def057a5b9658b00e8f8fe8743d144d1e0911c2e454", size = 4763638, upload-time = "2026-09-30T15:29:28.588Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/fc/35/b345bdfa40c9126df1a9d33236aa98418367931b8725f84fc3ae2b98dc59/cryptography-50.0.2-cp39-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:58ddb5a8e3179d12f19e4ea34d2d32e9d63a4baa142c875c1eb59f41b7243acd", size = 5382217, upload-time = "2026-09-30T15:29:30.589Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/4f/87/ef344a9e616871f2519c22d6afcda79ddd5d35e9592d95eb6e677608d055/cryptography-50.0.2-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:f21e8a22c8605750c7af886bab299a363721264061b4ac0a30efb73cfd58efc5", size = 4781387, upload-time = "2026-09-30T15:29:32.605Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/90/5b/f2fdb13cd0b96f6f932c8627bb292a45f11c64d21620a8e120aee9a3b848/cryptography-50.0.2-cp39-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:9c8402a82ea0dc4ceeab793db05f0fafa8ca139ca34fcde5df0f596103c74107", size = 4403790, upload-time = "2026-09-30T15:29:34.374Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/bc/ce/7e4f662b1e3c393513569e402cfc85ac7da0bd3d5435e122a3140219eb2d/cryptography-50.0.2-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:0ddc924c04591c2811ca024d62ecad4f7f6f08af8939c211438f48a16bd23602", size = 4764319, upload-time = "2026-09-30T15:29:36.149Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3c/3f/86ff33ce34cc0de6847fb96e035a1a760d81652e38643f617c02ad32ef7a/cryptography-50.0.2-cp39-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:a6557e5f38e065ca9fbdaf7cfc7435ecb1d113aa81a022d1b51921ee7432e227", size = 5338560, upload-time = "2026-09-30T15:29:39.053Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/40/cf/6b5c8e2fd9202d98988ab7cb5cc5c991704c4ad55f492ff408e4969f83f1/cryptography-50.0.2-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:1981f1db4630889b9ef7803fadef12b056f428cb6b85c27ba57b774793b6093c", size = 4780973, upload-time = "2026-09-30T15:29:41.251Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/10/bf/8d6ebc7dded797bd0f0160d52188021211f011a2b164ef0ae1dac4587465/cryptography-50.0.2-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:7a8701d6b584d76e909e3d305b7d126b41439876a5aaf76cddc67fc230eafa2e", size = 4897738, upload-time = "2026-09-30T15:29:43.106Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d4/aa/f3f6e0de7e6253b8baa8b2d8fb9d50924fa75cee3d4624bd4bc1208ee923/cryptography-50.0.2-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:ce47f66801c20ec6c6632453bb5960fe38939e9306970b48b3a5a26de7745d94", size = 5058280, upload-time = "2026-09-30T15:29:44.827Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f6/b6/a1faf3a27ae9405fb34b1713cc73b2d8a26b04d5c561578fa2e6ef3e5bb9/cryptography-50.0.2-cp39-abi3-win_amd64.whl", hash = "sha256:4e81d95e5bafc2d6e34e4bed780e53e4d5b9a2f928573428aa4d35fbec1eb0de", size = 3854095, upload-time = "2026-09-30T15:29:46.782Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "distlib"
|
||||||
|
version = "0.4.3"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/c9/02/bd72be9134d25ed783ecbbc38a539ffaefbf90c78418c7fb7229600dbac7/distlib-0.4.3.tar.gz", hash = "sha256:f152097224a0ae24be5a0f6bae1b9359af82133bce63f98a95f86cae1aede9ed", size = 615141, upload-time = "2026-06-12T08:04:52.847Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/02/08/9c41fb51ab5b43eb21674aff13df270e8ba6c4b29c8624e328dc7a9482af/distlib-0.4.3-py2.py3-none-any.whl", hash = "sha256:4b0ce306c966eb73bc3a7b6abad017c556dadd92c44701562cd528ac7fde4d5b", size = 470628, upload-time = "2026-06-12T08:04:50.506Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "fastapi"
|
||||||
|
version = "0.115.6"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "pydantic" },
|
||||||
|
{ name = "starlette" },
|
||||||
|
{ name = "typing-extensions" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/93/72/d83b98cd106541e8f5e5bfab8ef2974ab45a62e8a6c5b5e6940f26d2ed4b/fastapi-0.115.6.tar.gz", hash = "sha256:9ec46f7addc14ea472958a96aae5b5de65f39721a46aaf5705c480d9a8b76654", size = 301336, upload-time = "2024-12-03T22:46:01.629Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/52/b3/7e4df40e585df024fac2f80d1a2d579c854ac37109675db2b0cc22c0bb9e/fastapi-0.115.6-py3-none-any.whl", hash = "sha256:e9240b29e36fa8f4bb7290316988e90c381e5092e0cbe84e7818cc3713bcf305", size = 94843, upload-time = "2024-12-03T22:45:59.368Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "filelock"
|
||||||
|
version = "4.0.12"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/53/e4/34efcb869715cf299e47d1ac7b2624d2bcb6f2d3dffc2f0abe8417f65ab2/filelock-4.0.12.tar.gz", hash = "sha256:cf42711a7ac791818b299fab0332a088c65aeeefa36290de98db92c434303b0c", size = 253880, upload-time = "2026-10-05T13:17:43.677Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/da/a1/b047e1c486a2c66283633b7dbb73872ca16ca1ce6a426f70d4d037fc8e1c/filelock-4.0.12-py3-none-any.whl", hash = "sha256:5f17ee83ecee8a6f3e389c75822fb70a1c2f0506b99438a6dffa1d56793588c8", size = 111732, upload-time = "2026-10-05T13:17:42.361Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "h11"
|
||||||
|
version = "0.16.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250, upload-time = "2025-04-24T03:35:25.427Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "httpcore"
|
||||||
|
version = "1.0.9"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "certifi" },
|
||||||
|
{ name = "h11" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/06/94/82699a10bca87a5556c9c59b5963f2d039dbd239f25bc2a63907a05a14cb/httpcore-1.0.9.tar.gz", hash = "sha256:6e34463af53fd2ab5d807f399a9b45ea31c3dfa2276f15a2c3f00afff6e176e8", size = 85484, upload-time = "2025-04-24T22:06:22.219Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "httptools"
|
||||||
|
version = "0.8.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/43/e5/d471fcb0e14523fe1c3f4ba58ca52480e7bd70ad7109a3846bc75892f7fb/httptools-0.8.0.tar.gz", hash = "sha256:6b2a32f18d97e16e90827d7a819ffa8dbd8cc245fc4e1fa9d1095b54ef4bd999", size = 271342, upload-time = "2026-05-25T22:17:48.841Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/14/88/1d21a36da8f5cb0fa49eafd4b169eba5608d57e75bbcf61845cbc6243216/httptools-0.8.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:880490234c10f70a9830743097e8958d6e4b9f5a0ffc24515023afeef984054d", size = 208247, upload-time = "2026-05-25T22:17:07.843Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a5/42/cc4feea2945cb3051038f090c9b36bd5b8a9d7f5a894a506a8983e33fd1c/httptools-0.8.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:5931891fb7b441b8a3853cf1b85c82c903defce084dd5f6771ca46e31bf862c5", size = 113064, upload-time = "2026-05-25T22:17:09.136Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e3/a6/febbb8b8db0f58b38e44ad6cb946e6a255ae49b55f2e8543408fb7501ccd/httptools-0.8.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b15fc622b0f869d19207c4089a501d9bcc63ca5e071ffdd2f03f922df882dcb2", size = 523851, upload-time = "2026-05-25T22:17:10.106Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b7/e4/f90a0df0b83beff265b7e3b65f2a4cefd95792d4be0ac3e16049f2acd3c2/httptools-0.8.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:425f83884fd6343828d8c565f046cb72b6d19063f6924093e11bcd8e1548cd09", size = 518842, upload-time = "2026-05-25T22:17:11.218Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/9e/2d/0c9ac76dd2c893841fbf6498d6acec4f2442e1b7067f6e3e316a80e494e8/httptools-0.8.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:ef7c3c97f4311c7be57e2986629df89d49cb434dbff78eafcd48c2bff986b15a", size = 501238, upload-time = "2026-05-25T22:17:12.728Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ca/42/906adc91ae3a5fa9c59c0a2f21c139725bd7e5b41ae6acd485cd14123ebf/httptools-0.8.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:a1afd7c9fbff0d9f5d489c4ce2768bd09c84a46ddefc7161e6aa82ae35c85745", size = 509567, upload-time = "2026-05-25T22:17:13.842Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/05/0b/4240efeb672751ee5b9b380cb0e3fdc050bc05f68adc7a8aefc4fcd9a69a/httptools-0.8.0-cp312-cp312-win_amd64.whl", hash = "sha256:cd96f29b4bab1d42fa6e3d008711c75e0f79e94e06827330160e3a304227f150", size = 90918, upload-time = "2026-05-25T22:17:15.155Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "httpx"
|
||||||
|
version = "0.28.1"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "anyio" },
|
||||||
|
{ name = "certifi" },
|
||||||
|
{ name = "httpcore" },
|
||||||
|
{ name = "idna" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/b1/df/48c586a5fe32a0f01324ee087459e112ebb7224f646c0b5023f5e79e9956/httpx-0.28.1.tar.gz", hash = "sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc", size = 141406, upload-time = "2024-12-06T15:37:23.222Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "httpx-sse"
|
||||||
|
version = "0.4.3"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/0f/4c/751061ffa58615a32c31b2d82e8482be8dd4a89154f003147acee90f2be9/httpx_sse-0.4.3.tar.gz", hash = "sha256:9b1ed0127459a66014aec3c56bebd93da3c1bc8bb6618c8082039a44889a755d", size = 15943, upload-time = "2025-10-10T21:48:22.271Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d2/fd/6668e5aec43ab844de6fc74927e155a3b37bf40d7c3790e49fc0406b6578/httpx_sse-0.4.3-py3-none-any.whl", hash = "sha256:0ac1c9fe3c0afad2e0ebb25a934a59f4c7823b60792691f779fad2c5568830fc", size = 8960, upload-time = "2025-10-10T21:48:21.158Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "identify"
|
||||||
|
version = "2.6.20"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/53/35/d70c0006c7cee65999ea94a6273e60b2094f600a3d8b71b04318253fc643/identify-2.6.20.tar.gz", hash = "sha256:ad729860a923858d26917c2f4fb0a1d83d27a75b1e090c06440c573f048f3285", size = 99681, upload-time = "2026-09-26T20:29:07.187Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/fa/70/fffc9613501877c0a10a1ef73a165e6ded3e53f6e7502227fef56d973933/identify-2.6.20-py2.py3-none-any.whl", hash = "sha256:6a16b69b93187244e0548cbfd25b3e4a6f9a7a2ad784625c3bec2b8d27b81aaa", size = 99506, upload-time = "2026-09-26T20:29:06.054Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "idna"
|
||||||
|
version = "3.20"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/f5/08/8eea9d4b8302028f3abb2c0813953f7aec26d33b7a8960ed760e65ff29fa/idna-3.20.tar.gz", hash = "sha256:a7db850025b95ded1eae8a46181a1a6c56c92c96f0e2b005d9ff8dc0210cab44", size = 216463, upload-time = "2026-09-17T14:11:04.752Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/58/a2/bb081bab032533a855d44de1d56f8e8426114ff1ba5d1f07a438a0a654f8/idna-3.20-py3-none-any.whl", hash = "sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c", size = 69583, upload-time = "2026-09-17T14:11:03.168Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "iniconfig"
|
||||||
|
version = "2.3.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "jsonschema"
|
||||||
|
version = "4.26.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "attrs" },
|
||||||
|
{ name = "jsonschema-specifications" },
|
||||||
|
{ name = "referencing" },
|
||||||
|
{ name = "rpds-py" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/b3/fc/e067678238fa451312d4c62bf6e6cf5ec56375422aee02f9cb5f909b3047/jsonschema-4.26.0.tar.gz", hash = "sha256:0c26707e2efad8aa1bfc5b7ce170f3fccc2e4918ff85989ba9ffa9facb2be326", size = 366583, upload-time = "2026-01-07T13:41:07.246Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/69/90/f63fb5873511e014207a475e2bb4e8b2e570d655b00ac19a9a0ca0a385ee/jsonschema-4.26.0-py3-none-any.whl", hash = "sha256:d489f15263b8d200f8387e64b4c3a75f06629559fb73deb8fdfb525f2dab50ce", size = 90630, upload-time = "2026-01-07T13:41:05.306Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "jsonschema-specifications"
|
||||||
|
version = "2025.9.1"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "referencing" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/19/74/a633ee74eb36c44aa6d1095e7cc5569bebf04342ee146178e2d36600708b/jsonschema_specifications-2025.9.1.tar.gz", hash = "sha256:b540987f239e745613c7a9176f3edb72b832a4ac465cf02712288397832b5e8d", size = 32855, upload-time = "2025-09-08T01:34:59.186Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/41/45/1a4ed80516f02155c51f51e8cedb3c1902296743db0bbc66608a0db2814f/jsonschema_specifications-2025.9.1-py3-none-any.whl", hash = "sha256:98802fee3a11ee76ecaca44429fda8a41bff98b00a0f2838151b113f210cc6fe", size = 18437, upload-time = "2025-09-08T01:34:57.871Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "librt"
|
||||||
|
version = "0.16.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/04/f5/9dc696772d241814bacac7880bac32f2930b5a6ebc1f85317b83161a011c/librt-0.16.0.tar.gz", hash = "sha256:ac38d6d8d66bf3d744148dbbc0b8e193e195a51e364ed55e224631f5721891fc", size = 219839, upload-time = "2026-09-29T00:55:32.891Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ad/76/bbdaeb87b7c47b5c7343e90222b9bfa4e4a8a83f647e02933ad0129225b1/librt-0.16.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:fe52bf4641069e7978a14253b036cb9002def1926317e710f2e249f8a8c47742", size = 151028, upload-time = "2026-09-29T00:52:41.508Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/fd/0c/ab8ed3dab0085931aec4a792c7eaac8dc6c5ff4691fda3a5360d9d8a9cd2/librt-0.16.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:5bcc2c4726ced915b00de0c9856a4eeabfb3fddb93e10e0b8f735b7709358b6d", size = 155283, upload-time = "2026-09-29T00:52:42.873Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/eb/36/494e79d460c80c1f030661e8287c9eca5e1ad652dc2b2180b6cd42abce0a/librt-0.16.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ff7baa55f8e7c69851419e50a666015d02a74198716fd45c0125a2112e0a389f", size = 503185, upload-time = "2026-09-29T00:52:44.638Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/9b/34/a8464038dd9db6e4381fa2b6eb73dc9a50888d77102c4c139304ac35cddc/librt-0.16.0-cp312-cp312-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:b95d5d92ab83d39e760a52091bb1baba664f3a2351e39b1e16801e5747c2f0e9", size = 496738, upload-time = "2026-09-29T00:52:46.441Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/6a/53/e0e5e334ef0c6ed27039d323819368b9ef6712be87d55ee2bf9799398afd/librt-0.16.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b6d085d70bce51d43c5c7c36d63490770180d8779e71c49305c87b4213918de7", size = 513818, upload-time = "2026-09-29T00:52:48.068Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ad/f7/7ce72cbf19d0addd05090b339152fd0548a02562c2866a603e6e3b3da2df/librt-0.16.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:36e53948e99bbe3ffea257124cfcae1cfb01831555c9a9c903c9f9a72db7fd07", size = 531952, upload-time = "2026-09-29T00:52:49.816Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/83/22/0b1bcb6a8e723c8b4fd60dfc8ae8ec6461c54073fbc8685efeb8d900d407/librt-0.16.0-cp312-cp312-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:54d11f726aae9df5a6ffbbf0a03a52449bbac84a53ef03669cb41cdfd4ae41bf", size = 524465, upload-time = "2026-09-29T00:52:51.354Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/64/2e/e9c23b8b9df1813da1be205deca9606beb7ddd033972246cd426d05374a0/librt-0.16.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:4323193ac0cd025f85af531df8ba91bf24d1973b401697347a6282e8fd3fcf5e", size = 543191, upload-time = "2026-09-29T00:52:53.284Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/bc/e5/6a8b21b342c03ed7e230fa3afbfd2edc58e6e87ef1f0d11fa2b9a748c252/librt-0.16.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:e42f8e098b9c5396fefa05fb1cc7e33b0e08fc51da106b5de4a45fd22aac6743", size = 546979, upload-time = "2026-09-29T00:52:54.932Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/84/9e/b5129023eced1be01e01c22757f53be551d463b1bb7264f787927404c1d6/librt-0.16.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:39ec1d5a14e37baf1450a6cabf03fe552340808bf1ad9d71824ab90117716459", size = 555317, upload-time = "2026-09-29T00:52:56.869Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/71/89/28bba5938c725fe91f06bf93f7fa6c6b150229df53a87b454b0d5c2a796e/librt-0.16.0-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:d1aabe3925cbb4a08d15b7b20ba4011b53019da0c4173a25155139b7b1baed65", size = 535846, upload-time = "2026-09-29T00:52:58.475Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/22/92/63773026614f888c5d4e370e395ce42ca604b89f70b3acdfedbf94851b80/librt-0.16.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:300c3ffdc459f4a779a8411ecb188e3ac0b1ff3a3a7b099642555dedae06c69b", size = 573700, upload-time = "2026-09-29T00:53:00.127Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/66/8f/347d4677eefb57cd9f8e01d95a1dcee9a91b4e44e664173c32ff6f3752e5/librt-0.16.0-cp312-cp312-win32.whl", hash = "sha256:c17194318e4c0c0348b36f36c2ec7534436fe0a4c15582403162a4f08c80797a", size = 106158, upload-time = "2026-09-29T00:53:02.031Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f0/2c/5193dc81127cd5ddfad031391b046bf32dda219b38463ab872407ca30646/librt-0.16.0-cp312-cp312-win_amd64.whl", hash = "sha256:25a58a19ea8d83b68209f04912df765e9260635ef77646542ed4b4abe6bc7940", size = 126176, upload-time = "2026-09-29T00:53:03.445Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ff/3d/9668a400c8dd81d162eba38b33fa49fa6205f1a64493570a13fbb815c3ee/librt-0.16.0-cp312-cp312-win_arm64.whl", hash = "sha256:f7be7cf555bc30ec12622e9447299cc4a9b8ff307548b634794353db0c2065dc", size = 116572, upload-time = "2026-09-29T00:53:04.815Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "mcp"
|
||||||
|
version = "1.29.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "anyio" },
|
||||||
|
{ name = "httpx" },
|
||||||
|
{ name = "httpx-sse" },
|
||||||
|
{ name = "jsonschema" },
|
||||||
|
{ name = "pydantic" },
|
||||||
|
{ name = "pydantic-settings" },
|
||||||
|
{ name = "pyjwt", extra = ["crypto"] },
|
||||||
|
{ name = "python-multipart" },
|
||||||
|
{ name = "pywin32", marker = "sys_platform == 'win32'" },
|
||||||
|
{ name = "sse-starlette" },
|
||||||
|
{ name = "starlette" },
|
||||||
|
{ name = "typing-extensions" },
|
||||||
|
{ name = "typing-inspection" },
|
||||||
|
{ name = "uvicorn", marker = "sys_platform != 'emscripten'" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/30/d3/f9acc21dfc886e4f78e2add1a47db46ce16884346afde53f8a064c02c891/mcp-1.29.0.tar.gz", hash = "sha256:52d01f334de1868cc3bb2d6604931126a67631f99a6c5d3b82ba47290315ec36", size = 643148, upload-time = "2026-07-28T13:41:41.939Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/01/c8/248b201f6d753d69fd5d6506011abbb35a946d9142b2ae311a948fd0be3d/mcp-1.29.0-py3-none-any.whl", hash = "sha256:f5a075bb611f23d6f4d080c6a1699fa62772eebc562ba9e66b306ddde1c755f7", size = 223436, upload-time = "2026-07-28T13:41:40.337Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "mypy"
|
||||||
|
version = "1.20.2"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "librt", marker = "platform_python_implementation != 'PyPy'" },
|
||||||
|
{ name = "mypy-extensions" },
|
||||||
|
{ name = "pathspec" },
|
||||||
|
{ name = "typing-extensions" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/04/af/e3d4b3e9ec91a0ff9aabfdb38692952acf49bbb899c2e4c29acb3a6da3ae/mypy-1.20.2.tar.gz", hash = "sha256:e8222c26daaafd9e8626dec58ae36029f82585890589576f769a650dd20fd665", size = 3817349, upload-time = "2026-04-21T17:12:28.473Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/71/4e/7560e4528db9e9b147e4c0f22660466bf30a0a1fe3d63d1b9d3b0fd354ee/mypy-1.20.2-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:4dbfcf869f6b0517f70cf0030ba6ea1d6645e132337a7d5204a18d8d5636c02b", size = 14539393, upload-time = "2026-04-21T17:07:12.52Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/32/d9/34a5efed8124f5a9234f55ac6a4ced4201e2c5b81e1109c49ad23190ec8c/mypy-1.20.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:4b6481b228d072315b053210b01ac320e1be243dc17f9e5887ef167f23f5fae4", size = 13361642, upload-time = "2026-04-21T17:06:53.742Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d1/14/eb377acf78c03c92d566a1510cda8137348215b5335085ef662ab82ecd3a/mypy-1.20.2-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:34397cdced6b90b836e38182076049fdb41424322e0b0728c946b0939ebdf9f6", size = 13740347, upload-time = "2026-04-21T17:12:04.73Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b9/94/7e4634a32b641aa1c112422eed1bbece61ee16205f674190e8b536f884de/mypy-1.20.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a5da6976f20cae27059ea8d0c86e7cef3de720e04c4bb9ee18e3690fdb792066", size = 14734042, upload-time = "2026-04-21T17:07:43.16Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7a/f3/f7e62395cb7f434541b4491a01149a4439e28ace4c0c632bbf5431e92d1f/mypy-1.20.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:56908d7e08318d39f85b1f0c6cfd47b0cac1a130da677630dac0de3e0623e102", size = 14964958, upload-time = "2026-04-21T17:11:00.665Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3e/0d/47e3c3a0ec2a876e35aeac365df3cac7776c36bbd4ed18cc521e1b9d255b/mypy-1.20.2-cp312-cp312-win_amd64.whl", hash = "sha256:d52ad8d78522da1d308789df651ee5379088e77c76cb1994858d40a426b343b9", size = 10911340, upload-time = "2026-04-21T17:10:49.179Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d6/b2/6c852d72e0ea8b01f49da817fb52539993cde327e7d010e0103dc12d0dac/mypy-1.20.2-cp312-cp312-win_arm64.whl", hash = "sha256:785b08db19c9f214dc37d65f7c165d19a30fcecb48abfa30f31b01b5acaabb58", size = 9833947, upload-time = "2026-04-21T17:09:05.267Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/28/9a/f23c163e25b11074188251b0b5a0342625fc1cdb6af604757174fa9acc9b/mypy-1.20.2-py3-none-any.whl", hash = "sha256:a94c5a76ab46c5e6257c7972b6c8cff0574201ca7dc05647e33e795d78680563", size = 2637314, upload-time = "2026-04-21T17:05:54.5Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "mypy-extensions"
|
||||||
|
version = "1.1.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/a2/6e/371856a3fb9d31ca8dac321cda606860fa4548858c0cc45d9d1d4ca2628b/mypy_extensions-1.1.0.tar.gz", hash = "sha256:52e68efc3284861e772bbcd66823fde5ae21fd2fdb51c62a211403730b916558", size = 6343, upload-time = "2025-04-22T14:54:24.164Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/79/7b/2c79738432f5c924bef5071f933bcc9efd0473bac3b4aa584a6f7c1c8df8/mypy_extensions-1.1.0-py3-none-any.whl", hash = "sha256:1be4cccdb0f2482337c4743e60421de3a356cd97508abadd57d47403e94f5505", size = 4963, upload-time = "2025-04-22T14:54:22.983Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "nodeenv"
|
||||||
|
version = "1.11.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/9a/8e/105de02c1322cfada6d9710d9146ef8026419d433c9d08359a2d35805811/nodeenv-1.11.0.tar.gz", hash = "sha256:3ce8fe5b71d16e8af7039ca65257354100bc772965d6bc549070649e53b1b146", size = 99301, upload-time = "2026-09-26T11:29:21.367Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/54/c8/12811c9b48fde162bb72b6f2e78fada9a247a09d7bb5be2050a5d099c77b/nodeenv-1.11.0-py2.py3-none-any.whl", hash = "sha256:edaa16e6c14d7cf395d75d4bbd5a26390f4dc06501a33b4e76282b02cc688a25", size = 34277, upload-time = "2026-09-26T11:29:19.933Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "packaging"
|
||||||
|
version = "26.3"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/7d/fa/3944b40b07da9ce895c0e6303a5ab7d53da063554f534556b134a54d6093/packaging-26.3.tar.gz", hash = "sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79", size = 313412, upload-time = "2026-08-04T18:15:28.737Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/63/34/ba1c580383c9eada3711951fef0795c80b829a078d72188184bcab9dd527/packaging-26.3-py3-none-any.whl", hash = "sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c", size = 129956, upload-time = "2026-08-04T18:15:27.159Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pathspec"
|
||||||
|
version = "1.1.1"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/5a/82/42f767fc1c1143d6fd36efb827202a2d997a375e160a71eb2888a925aac1/pathspec-1.1.1.tar.gz", hash = "sha256:17db5ecd524104a120e173814c90367a96a98d07c45b2e10c2f3919fff91bf5a", size = 135180, upload-time = "2026-04-27T01:46:08.907Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f1/d9/7fb5aa316bc299258e68c73ba3bddbc499654a07f151cba08f6153988714/pathspec-1.1.1-py3-none-any.whl", hash = "sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189", size = 57328, upload-time = "2026-04-27T01:46:07.06Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "platformdirs"
|
||||||
|
version = "4.12.3"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/42/23/4a86fc741c38c5b69792a4ef954b281afa69bea9f083f881de1b0d23bc07/platformdirs-4.12.3.tar.gz", hash = "sha256:427fc0bb321ae0c5b037fa03238ca74820437be162e78b4848c4d4055b9b766c", size = 59538, upload-time = "2026-10-03T23:40:46.595Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c5/9b/6ce1ead737fe496611bda600c263b9a11ae7bd8f41bb13f9bd7e0a2c37a4/platformdirs-4.12.3-py3-none-any.whl", hash = "sha256:080f3b39423b5abfca9a23d84c4e9795f54d395cd8459867a8ded44084fcd5f8", size = 32493, upload-time = "2026-10-03T23:40:45.104Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pluggy"
|
||||||
|
version = "1.6.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pre-commit"
|
||||||
|
version = "4.6.2"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "cfgv" },
|
||||||
|
{ name = "identify" },
|
||||||
|
{ name = "nodeenv" },
|
||||||
|
{ name = "pyyaml" },
|
||||||
|
{ name = "virtualenv" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/74/89/1f3e8e1fc3e97de0fa963495832f581f025f29471602a309e48808244292/pre_commit-4.6.2.tar.gz", hash = "sha256:8f5d7bfb021ecdbcd9d49d89847082dd24172ccde534390081a679ad046e2441", size = 198670, upload-time = "2026-08-10T22:07:18.421Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/45/e2/bbb7129c9e7999a6b8ee9cca3b66486c25c423ab5a75f34071798b74ce94/pre_commit-4.6.2-py2.py3-none-any.whl", hash = "sha256:e2dde9a75d3bce11bd3831c26d134df00a2803c1d818be6a0383c3dcda25dc4e", size = 226202, upload-time = "2026-08-10T22:07:16.942Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pycparser"
|
||||||
|
version = "3.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz", hash = "sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29", size = 103492, upload-time = "2026-01-21T14:26:51.89Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", size = 48172, upload-time = "2026-01-21T14:26:50.693Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pydantic"
|
||||||
|
version = "2.13.5"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "annotated-types" },
|
||||||
|
{ name = "pydantic-core" },
|
||||||
|
{ name = "typing-extensions" },
|
||||||
|
{ name = "typing-inspection" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/53/ef/fc4f868f4e2cee79f863883abffceff107875f569b848507319842d2a681/pydantic-2.13.5.tar.gz", hash = "sha256:51a9c5f7b2f8e636f04c6cada605d9b6a3bf1348fdf945a3d8869b19bba0ee08", size = 845750, upload-time = "2026-08-28T14:04:00.916Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/eb/47/c95ffc2009878c7aac0c5e08528022dcb885933252a88b5f170058014464/pydantic-2.13.5-py3-none-any.whl", hash = "sha256:346a034f080da3755d8e9cb5e00e8b07de1d39e4f6e2c87d8ab7cafa0b269a73", size = 472589, upload-time = "2026-08-28T14:03:59.136Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pydantic-core"
|
||||||
|
version = "2.46.5"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "typing-extensions" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/af/f9/8a06bea35ef8daf588f707784c973a7046e0034c8d8cfb08828eeffb8b75/pydantic_core-2.46.5.tar.gz", hash = "sha256:10416c15b8839ecc4ef4d0885da76da6fd0f67333a0eb8aff6d93c4b8f2910fc", size = 472262, upload-time = "2026-08-28T10:01:31.677Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/82/3f/76358795aa7a8c6d4f36e2cb828ad1c90ee118e1393a9281664f5aade9d4/pydantic_core-2.46.5-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:b9fe6fb92520e3fd61f2e49000b6911b188824f089b75973ea06d6267f0b476d", size = 2076516, upload-time = "2026-08-28T09:58:21.576Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/db/50/26b091836076ce4cb2fac264186936acc069e0595772cfd02a563bc4761a/pydantic_core-2.46.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:a39ac25a9a2fa4072efdb429833c4a4c8009a51ff9eea3eeae131713cd27991e", size = 1922874, upload-time = "2026-08-28T09:58:23.766Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/09/f0/2a8ce3849e299d44e2d2c196b6082643a3235565a735cb51db7a6261f614/pydantic_core-2.46.5-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4fdc8b93a41521988916eeaa271173fcca7fa0803d62f87675aac8dcec1c8e29", size = 1951772, upload-time = "2026-08-28T09:58:25.435Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/87/46/ac0dc8bdd9e6048183a14eb127764e7ad9240021c17513074a4711b0e31e/pydantic_core-2.46.5-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b98134087d9de723658d17a42c7d0da8d6e2ef08015dee7dc93889047315f5e4", size = 2031832, upload-time = "2026-08-28T09:58:27.102Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c4/c2/339de5bef7be36301a2231eaa52e62163742c2281f11b5f4892bc79785cd/pydantic_core-2.46.5-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e652ab17569c94bff5475520f907b7148b8c24036a8ebbe5cf7cf7493d28579a", size = 2208645, upload-time = "2026-08-28T09:58:28.948Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7b/a0/9ff22b797724262da14427abaed4dd1d864a139693fc5e7809114376a716/pydantic_core-2.46.5-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d925f3d9afd05a8c0fb3a1031463a8d59ebe5e2afad297e29c78be19e13b4e62", size = 2265935, upload-time = "2026-08-28T09:58:30.625Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c0/a4/eb9409ec0736e50aa70a412f16c204ed149516846912f7e6724d4c73ee53/pydantic_core-2.46.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0fc5be0abd4a407e200d844b404e33639a554e7bd0d448e7b9ae181be4789ac2", size = 2066284, upload-time = "2026-08-28T09:58:32.289Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c0/02/7f6156ffc926857f1c37c07d9a388682865a81830ab6a1b637082c25e399/pydantic_core-2.46.5-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:816ff0a6550ffc06c098ccd2e0698600f9aa7da192a79eaa6f9af504a35db869", size = 2105889, upload-time = "2026-08-28T09:58:33.986Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/92/b1/e781d357ebe09fc929f995700f1b3503e8897f1cece183ecb1300d4d67e9/pydantic_core-2.46.5-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c7ea57fc63aa7da93a1bd2d644e6577befae10c52c4e36377635eea1056a74f5", size = 2158006, upload-time = "2026-08-28T09:58:35.647Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/70/0a/644597d84ab400e50609c192120b85c9681c22d3a20461b9060a79be0a7a/pydantic_core-2.46.5-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:efd62a42486f1bda5d24cb4f63d15a3c7768375fe83d36f9417b4ad7a2fb20b3", size = 2158408, upload-time = "2026-08-28T09:58:37.38Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1e/ee/ca3b7b3a4b3769ffe9ce9432a7c9be755de9593a46d3b0d54d0409323e44/pydantic_core-2.46.5-cp312-cp312-musllinux_1_1_armv7l.whl", hash = "sha256:2bc9419666990c06d7397831f2126a1ecc3594aaa3ff7de5bf2d066802f4e07b", size = 2309609, upload-time = "2026-08-28T09:58:39.22Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ce/52/39fa1f451486019524ca685020390e7ca351832fd874530ba30c8628e6dc/pydantic_core-2.46.5-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:18a09e1e1011b462f2e32774f25859ef1223d5c2b0546a633cf56654710721e0", size = 2342618, upload-time = "2026-08-28T09:58:40.89Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/81/5e/468fc630568c61dcef3cd47ad32ffbeed9af643f49208d1ea86ab4f890c4/pydantic_core-2.46.5-cp312-cp312-win32.whl", hash = "sha256:5cb482e9e84c851f4e623fe4acc1ced89168cf1fe18f7089db4548c8f5bbb65b", size = 1939475, upload-time = "2026-08-28T09:58:42.591Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/cf/c9/4c19f41b84cf6b622a72fbeed7665b25d47a187d68d47d0d430c07f23268/pydantic_core-2.46.5-cp312-cp312-win_amd64.whl", hash = "sha256:5e81740c09e310f5aa5cbd3e434a01c154d4bef93241c7877b39f211d2b78ba8", size = 2043140, upload-time = "2026-08-28T09:58:44.272Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/af/dd/0c1a050299147c746e5256db16d645ab5efd4f78c59937d581a0524e74a2/pydantic_core-2.46.5-cp312-cp312-win_arm64.whl", hash = "sha256:f7b0ec93a2893de856652154d73b7ba622f26fa97726487dcac373de5f4c6084", size = 1997729, upload-time = "2026-08-28T09:58:46.13Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/df/dd/053c2e4303f791f3b8f8a14ab0b22008e8eb21d868c0c90b4f9be705b76a/pydantic_core-2.46.5-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:013d6f3483d81e02e7c328831808f336c8596ee33b4bd4026b9ffb1e960b8942", size = 2062540, upload-time = "2026-08-28T10:01:00.318Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d7/dd/a18df751a5e37dd51bfad7f68e766999125bebe68c9e1d10a493ad01bd63/pydantic_core-2.46.5-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:e9c134bb666dd54b778b9fc0d2b50cbb7f979b9e3716f26a88c9ab3b6fc1dd0f", size = 1902040, upload-time = "2026-08-28T10:01:02.529Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b7/13/01d40f9d07ce8a779fd6e0bd8ad4fba91309500dd67b869e2e219d261a6d/pydantic_core-2.46.5-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:347ec774390c87326a2e4929d58d3f7e8763a104d5d35f4cd595a4c952366433", size = 1967479, upload-time = "2026-08-28T10:01:05.004Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/fa/04/c81d4841331c2178b6fb09ae225425e110ed72d990c9fe556c4ec03d1013/pydantic_core-2.46.5-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8e24d8f05fa2d28513d94e877e9c75ad66175376209b3977f916e240e623193c", size = 2111034, upload-time = "2026-08-28T10:01:07.345Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pydantic-settings"
|
||||||
|
version = "2.15.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "pydantic" },
|
||||||
|
{ name = "python-dotenv" },
|
||||||
|
{ name = "typing-inspection" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/68/ca/31c57507b13119d7d3cfa1576dad2911a4861e3be07b579395f4e9d393f9/pydantic_settings-2.15.0.tar.gz", hash = "sha256:694b793e84f766ba76a90ebdefc01d0a9a045dab0382bee70393da93712ad117", size = 261253, upload-time = "2026-08-07T09:24:57.419Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/30/a4/2bffa9f8e804325a09867f0e9d30795c80ea9f8d62560bd1b6ad6220eb2f/pydantic_settings-2.15.0-py3-none-any.whl", hash = "sha256:0ba092c291c94baceb5eff768aa0d56400a457585bc0175925a5a5510303da42", size = 69413, upload-time = "2026-08-07T09:24:55.839Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pygments"
|
||||||
|
version = "2.21.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/49/2e/ced460408999b33da6b31b0021b0f37d329e202d4169aeb164493778f25b/pygments-2.21.0.tar.gz", hash = "sha256:610ca751c9bc2492b38eb9a38a7fbc93edbbb2d7182edaf34e66ae493dee5c8c", size = 5005329, upload-time = "2026-08-17T08:02:48.824Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/71/46/17f022dd3e953bf20a04a028a21ec746d942f8d2af30fa0f124fa0e6a684/pygments-2.21.0-py3-none-any.whl", hash = "sha256:2363c69b61c4a97c838da3b130dcd6468f4848992b21a82f2a63ec34377137d9", size = 1250147, upload-time = "2026-08-17T08:02:44.912Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pyjwt"
|
||||||
|
version = "2.15.1"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/43/ea/5194e52748b0da83d71e082d75496eaec6e58f419f5e184786ded517e6a9/pyjwt-2.15.1.tar.gz", hash = "sha256:4f259e80cdfb6b3fc18a7de51fd1ef9ec79652f25019bae68975ca2468a34df8", size = 121252, upload-time = "2026-09-28T18:40:42.598Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/50/ca/44de4e75f8aadc457f0634be3b542815078ded46dca30efb960edeecad6e/pyjwt-2.15.1-py3-none-any.whl", hash = "sha256:42d59d631f7768a1028a64c7ff581a9bf7519804daf91fc5b6c56e30eec5e193", size = 33860, upload-time = "2026-09-28T18:40:41.429Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[package.optional-dependencies]
|
||||||
|
crypto = [
|
||||||
|
{ name = "cryptography" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pytest"
|
||||||
|
version = "9.1.1"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "colorama", marker = "sys_platform == 'win32'" },
|
||||||
|
{ name = "iniconfig" },
|
||||||
|
{ name = "packaging" },
|
||||||
|
{ name = "pluggy" },
|
||||||
|
{ name = "pygments" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "python-discovery"
|
||||||
|
version = "1.6.1"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "filelock" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/0c/57/250bd238b966cece44328235eb85290045d059265fdaf7527a3a958123db/python_discovery-1.6.1.tar.gz", hash = "sha256:cf87d3627dfb4412437fdd5b13eae402607722998d21567993aedbc59b23c15e", size = 84338, upload-time = "2026-09-18T01:31:53.971Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/16/7d/e9ffbadfbf89c93848412d04594135c4ae8c1d37d9e053b9c3ed718fabc4/python_discovery-1.6.1-py3-none-any.whl", hash = "sha256:d43fcdef879fe795352bd13ccf8d185ba5a9f86f36cfcd00529f596e737442b3", size = 38664, upload-time = "2026-09-18T01:31:52.448Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "python-dotenv"
|
||||||
|
version = "1.2.2"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/82/ed/0301aeeac3e5353ef3d94b6ec08bbcabd04a72018415dcb29e588514bba8/python_dotenv-1.2.2.tar.gz", hash = "sha256:2c371a91fbd7ba082c2c1dc1f8bf89ca22564a087c2c287cd9b662adde799cf3", size = 50135, upload-time = "2026-03-01T16:00:26.196Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/0b/d7/1959b9648791274998a9c3526f6d0ec8fd2233e4d4acce81bbae76b44b2a/python_dotenv-1.2.2-py3-none-any.whl", hash = "sha256:1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a", size = 22101, upload-time = "2026-03-01T16:00:25.09Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "python-multipart"
|
||||||
|
version = "0.0.32"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/5b/42/55c32bb9b12693c092ad250a0e82edb5b31ddeda6eb772de5f308b3804ad/python_multipart-0.0.32.tar.gz", hash = "sha256:be54b7f3fa167bb83e4fcd936b887b708f4e57fe75911c02aebf53efaf8d938e", size = 46881, upload-time = "2026-06-04T16:18:58.647Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e1/04/e8135ebd1ad02c56ec633277529b2602ff99ff634be76cdba5744cf554fd/python_multipart-0.0.32-py3-none-any.whl", hash = "sha256:ff6d3f776f16878c894e52e107296ffc890e913c611b1a4ec6c44e2821fe2e23", size = 30042, upload-time = "2026-06-04T16:18:57.319Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pywin32"
|
||||||
|
version = "312"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/83/ff/32aa7d2ed0ab12b323aaa64f9b75e6ad4f8fd09f9ccfc28c79414d46838d/pywin32-312-cp312-cp312-win32.whl", hash = "sha256:dab4f65ac9c4e48400a2a0530c46c3c579cd5905ecd11b80692373915269208b", size = 6371877, upload-time = "2026-06-04T07:49:28.836Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/03/d9/77040d3b43df3f3be32ea289433d660d2727f5ba327bc73be835127d9d60/pywin32-312-cp312-cp312-win_amd64.whl", hash = "sha256:b457f6d628a47e8a7346ce22acb7e1a46a4a78b52e1d17e1af56871bd19a93bc", size = 6914841, upload-time = "2026-06-04T07:49:31.85Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e3/cc/7b1ec671775756020a0ee7f4feeaf3c568f0ab86bd3900088cf986937a92/pywin32-312-cp312-cp312-win_arm64.whl", hash = "sha256:6017c58e12f6809fbb0555b75df144c2922a9ffd18e4b9b5afa863b6c1a9d950", size = 6727901, upload-time = "2026-06-04T07:49:34.244Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pyyaml"
|
||||||
|
version = "6.0.3"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "referencing"
|
||||||
|
version = "0.37.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "attrs" },
|
||||||
|
{ name = "rpds-py" },
|
||||||
|
{ name = "typing-extensions" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/22/f5/df4e9027acead3ecc63e50fe1e36aca1523e1719559c499951bb4b53188f/referencing-0.37.0.tar.gz", hash = "sha256:44aefc3142c5b842538163acb373e24cce6632bd54bdb01b21ad5863489f50d8", size = 78036, upload-time = "2025-10-13T15:30:48.871Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2c/58/ca301544e1fa93ed4f80d724bf5b194f6e4b945841c5bfd555878eea9fcb/referencing-0.37.0-py3-none-any.whl", hash = "sha256:381329a9f99628c9069361716891d34ad94af76e461dcb0335825aecc7692231", size = 26766, upload-time = "2025-10-13T15:30:47.625Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "rpds-py"
|
||||||
|
version = "2026.9.1"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/42/68/3bd46b8a5e01d3c2ebdf9c5e9497912e3fe0cde02bac21a7130ca866e403/rpds_py-2026.9.1.tar.gz", hash = "sha256:4793ef7f78268b124b73fa933440f01d258bbae01de9fa53e9080c9ab0425a12", size = 63948, upload-time = "2026-10-04T16:32:36.469Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/5d/34/a828586ea3329fbb50895b50e9cf98ca3d924a9f41a0b26d443bff1b3794/rpds_py-2026.9.1-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:50906f5aea24b5a865cbd0a589698288631d9f3a54c3a937c83aefa95a0d14af", size = 346822, upload-time = "2026-10-04T16:29:16.258Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/90/81/ac6a0d064982251856ce009c9e1dd51a34110b3c055aa7d1aad18b4899a3/rpds_py-2026.9.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:e21c1429e205828ea886a2293a4a2c8e01f4c25d9893ca330e97a6cf73f52e7b", size = 341380, upload-time = "2026-10-04T16:29:17.706Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/42/ff/bf7d54f362748fd6a49277b9d6531c394074110fedf50ad791ec59133fbb/rpds_py-2026.9.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2693b2728bbcc48d09a981a356954b0c47c53ff25b545856f28a889ea619f69a", size = 373053, upload-time = "2026-10-04T16:29:19.063Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/60/de/74b0ccdbbd28687b9b5fdb34c1cabed88352182facaced9d6f5486b8b9ed/rpds_py-2026.9.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:8601470267d938bcb7f3ab1a336100af51a4fd5b6ed030ef52461bb3ef5e7e07", size = 377742, upload-time = "2026-10-04T16:29:20.543Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c2/6d/b979775a3057b2a5c26d75ecbff60a20a24ef081db6dd76836fca3f20247/rpds_py-2026.9.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3890a6aa36e6baa53d5258a2a25d3ef8b37ad165a6ab27a892d7c3e3a432cd69", size = 487289, upload-time = "2026-10-04T16:29:21.958Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1d/5d/7c34734ce3ece943d9d6ee0122a74a21bbc75b47acb5b7053c83f6efb1ed/rpds_py-2026.9.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6b5b393eda5ea42cca1c1a6665f2a4882b4fd5d1777e41ce0545a107fb008c9d", size = 390710, upload-time = "2026-10-04T16:29:23.378Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1d/6d/b26eb1e75395925b3a142ed351cbed2ec8212f5c9d937aee3df32c701baa/rpds_py-2026.9.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:addeda51556dac7c1a2f14cda62db8b621cd12afba3091d03a96c72932387eab", size = 371261, upload-time = "2026-10-04T16:29:25.1Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7c/98/b2fdfe10301a9e27af21e634337fbba7baac0bc17fe44619a17c26bba2cb/rpds_py-2026.9.1-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:d9edf30457d74eebfd76b045535e36f1cd89062566a128a0db2145ca042d787e", size = 376075, upload-time = "2026-10-04T16:29:26.748Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c5/b1/c4b8d954e49c3c69cf8063c0c9e0919c99f3d2cecc46311606a1cbe835dc/rpds_py-2026.9.1-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:815d26356930846a40c7bc1366e7b1b0320ab8a063e66c11298a208bed0fd237", size = 399358, upload-time = "2026-10-04T16:29:28.282Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b3/59/9559a7293c97dff0cbb293efffbd36644103883ecc27741af8ef90c84ca8/rpds_py-2026.9.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:3b5a6f40f0a1486b4b36c888123afc67acdbd9f33235927acf5ff295429a0ba3", size = 550267, upload-time = "2026-10-04T16:29:29.851Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/6c/9d/6dc60e49511de4c8b00e74f9c1d43aed4d27f712cdb1ade92f2c199cbf64/rpds_py-2026.9.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:b5b8b0753718d258fd454283fbd57e14545d3b40583fa672e27cb4f987626bcc", size = 614018, upload-time = "2026-10-04T16:29:31.54Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/fb/bc/92a4ecf27301b886232a413360f6b348a81d55ffef654f2ebda5970ebc3d/rpds_py-2026.9.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:46d80bc76b51a6c24f9944368c28d38b8bcbcea1da4f2f8d3ebc31a67e8c6ec6", size = 578509, upload-time = "2026-10-04T16:29:33.01Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/08/53/efb97f2e6589b7ab8394385a73870e0b2ab2a195281c117b21fd6d7d4855/rpds_py-2026.9.1-cp312-cp312-win32.whl", hash = "sha256:befc2d6a953e563f8a7bfd87a42c22ebf8a3e980dcb7b6a4d17b70b0e914e8a3", size = 205401, upload-time = "2026-10-04T16:29:34.451Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1c/84/1700cc748d0eaa747486e4e82882d2575224e6e9a3148df583058850c629/rpds_py-2026.9.1-cp312-cp312-win_amd64.whl", hash = "sha256:5ce8943f79c2210f7abcc28e86367b03b28d95027fd01c46d2472373ae70c86f", size = 222923, upload-time = "2026-10-04T16:29:35.709Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c3/89/0302215373c2f8b4408b0fdfee955368cc378a98ff59508d47a7e0dc2dbf/rpds_py-2026.9.1-cp312-cp312-win_arm64.whl", hash = "sha256:501909f2e4a1e2dee528ef766fe3c469060ebc17e54a8383d404ba07a81a6f02", size = 218415, upload-time = "2026-10-04T16:29:37.263Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a3/90/a9ba81408369d34c1aca73319c15adc91ed45c1d21e9e8ee4832ffb7fe0b/rpds_py-2026.9.1-pp312-pypy312_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6cdc537c8633d7fd92a82e2e0d2ab74320a3f63d5e59fb9cf08711e08fe151c4", size = 381109, upload-time = "2026-10-04T16:32:08.994Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b3/44/5192a0bed94cec86bd2a5e1cc5a1bb8f7eec3aec907eacd2deb13e87e326/rpds_py-2026.9.1-pp312-pypy312_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:10e208f2425d973938afcd56e28a7c4be32e27b6a60b5d381f49fb9d8acf9759", size = 388222, upload-time = "2026-10-04T16:32:11.321Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/92/cd/5356549711448f18b52f7a11ffbe90f1228774996fbf3c6cd1b18d22abaf/rpds_py-2026.9.1-pp312-pypy312_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:6c0dbbcc19735fe5f8b0a54c07659d154a9e69f47e15d0a6ab7299215daf62cb", size = 500384, upload-time = "2026-10-04T16:32:13.478Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7a/88/ddda9d28adfe33119c75b2e733d4ba7e326f41d7e1b1093951771768ca48/rpds_py-2026.9.1-pp312-pypy312_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:684fd492fff4fead00587544e059be2bbcb6f93454f21fa2a91b66fc7508be82", size = 401661, upload-time = "2026-10-04T16:32:15.653Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/32/c3/bb59ba16a6b4a57995d15b8c04c00f28df3b938c4dcdde48fbe0f73edb0c/rpds_py-2026.9.1-pp312-pypy312_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d1028417bb44037eb3069c1009bd7b7277212876cda22fbe565b0bca9fab6d2c", size = 380748, upload-time = "2026-10-04T16:32:17.727Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f5/b8/0580faa1c5a32ddc160130dd7ae54d2d007272a89a40a6b5db3e50f9f585/rpds_py-2026.9.1-pp312-pypy312_pp73-manylinux_2_31_riscv64.whl", hash = "sha256:492e5e428cbe126221611f47e068f01660352feec4ad18bc0f5ea9b2ae88fb14", size = 385731, upload-time = "2026-10-04T16:32:19.855Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/bf/70/f73564642bbe3322c7eeef2c2f258cf040b71c41a430b532e59d04a1b838/rpds_py-2026.9.1-pp312-pypy312_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:88b5268892fde430d5531f95bc560b6efbbd67c929662c586afd729a96e7461c", size = 412131, upload-time = "2026-10-04T16:32:22.285Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8c/5d/17fff2e1f8f68721a2afb5cb48f7e442c751bb6b4883157c5abaf618cd08/rpds_py-2026.9.1-pp312-pypy312_pp73-musllinux_1_2_aarch64.whl", hash = "sha256:01445c8d194aa032a08e944f16567672da1c62dbdbefd8b6d0693032e290cf68", size = 558193, upload-time = "2026-10-04T16:32:24.4Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2a/d0/869ab6fb08531f97aae4a780b4d61b4190d8aef48fe054ee28bcf7114466/rpds_py-2026.9.1-pp312-pypy312_pp73-musllinux_1_2_i686.whl", hash = "sha256:eef6a03b0b6d08d0835ccfa8ec8d1bc70525e3801387567137b50c557695e6da", size = 622448, upload-time = "2026-10-04T16:32:26.844Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ba/a5/9672e532fe02cd3b92c78cfd8743abca939d96853006c63835009a1ecb6f/rpds_py-2026.9.1-pp312-pypy312_pp73-musllinux_1_2_x86_64.whl", hash = "sha256:6b9bf3135b4ad5981df9a73d71a35272d650a2985ae9c2746357b24d59de2448", size = 591126, upload-time = "2026-10-04T16:32:29.274Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/80/ef/3a9f8e4279c7920af561deed4cb7ebebed2794a8f608cf404d5eef14a105/rpds_py-2026.9.1-pp312-pypy312_pp80-macosx_10_12_x86_64.whl", hash = "sha256:56c6952a9b15047466d0c2347c446a761d4527f89976156341e68f0ce5cc08b0", size = 357108, upload-time = "2026-10-04T16:32:31.641Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b7/55/4b2fa381a583760aea5c92841e4e928a358e1c6511b129219e9c2826a226/rpds_py-2026.9.1-pp312-pypy312_pp80-macosx_11_0_arm64.whl", hash = "sha256:b242c27c8f836305a4a72df9cdd564386ac57b807bd252a063223331c9316b37", size = 346877, upload-time = "2026-10-04T16:32:34.061Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "ruff"
|
||||||
|
version = "0.16.10"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/c4/49/23802c45f093eb14bde54b141d2b2f058edfa63a06db7beed047308cc08f/ruff-0.16.10.tar.gz", hash = "sha256:eff4728c4eaae93f0955cd264d24b2ab348e74bf59986ccf282ba6dc16b3b017", size = 4958724, upload-time = "2026-10-01T18:03:21.697Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2f/21/ebce22e1d90cdb2cd691026b9c6e9bec6499f0b396089481755b5d49efff/ruff-0.16.10-py3-none-linux_armv6l.whl", hash = "sha256:488b0fe3f3574210e5cf80d9f59b9e3ab17a127a8155de3f307b392589cfb511", size = 10095557, upload-time = "2026-10-01T18:02:36.072Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/cb/98/a54de85876a8b2612bfa0d84c7b9abfb39c6a3354aee7800b09c1649b9e3/ruff-0.16.10-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:e748ff95c934c4e978783b8e687bc174e7bd84e8ad24e3243e1ecfcda5e0282d", size = 10340577, upload-time = "2026-10-01T18:02:39.258Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/9f/16/1a5a4a2657effe4806110f2b907313802f1367fcdbb29e8122766e407fab/ruff-0.16.10-py3-none-macosx_11_0_arm64.whl", hash = "sha256:3031a4a2e8e7b8a46f70be45f198c35a11ece509a94b80334d8d397a33c67550", size = 9774282, upload-time = "2026-10-01T18:02:41.79Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/6e/fb/470085af734da396e68cd80fb0a3e7c109a459716ae59588c0f6fab8a17d/ruff-0.16.10-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:494401c86df4c4c25f69b9419605d944467ee98c42fb6ad405ef4fa40b8fb67d", size = 9920895, upload-time = "2026-10-01T18:02:44.458Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/57/de/f10cffe4f88a37ea6615ec460f01bf76f0bc1477737a35d9ee61a630a78b/ruff-0.16.10-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:d203abc0ff2b773ee33d00ab8df0bb67046fbc7c07b119332f08b9b345cf8221", size = 9902707, upload-time = "2026-10-01T18:02:47.041Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ff/44/3fdcedf83ae60ef837dd239170e480dce606a9142cfa2db911afdf855fd9/ruff-0.16.10-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:bd83d1235a5258d318477bc5b576303974cbdef5df0c01a1bff14efcc23bd12a", size = 10619287, upload-time = "2026-10-01T18:02:49.485Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c1/62/02e76a5574002153618eafbb70e159468addc72a5d2c488e65cbb4639d2d/ruff-0.16.10-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:bc2610fb269fa56dd8a68669ae470fa6272902668c0fc2ebc3aa112b2633d5b8", size = 11339942, upload-time = "2026-10-01T18:02:52.008Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/6b/c4/cde27d47ad8d4126c587e608c67c46e7feac11763f606d261a1bc8a489e6/ruff-0.16.10-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3e70175e29cc94c26ea296c80e470180b744b7419026898e58f520c6ab32578e", size = 10934316, upload-time = "2026-10-01T18:02:54.811Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e4/03/17234145f302a645a123e8c3bb2411ecf4669fbc350de3b0d803230a1729/ruff-0.16.10-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f33f43a864a8483eebd160e713336c8bab02c934feaff0a33cf5ccb41546d09a", size = 10387968, upload-time = "2026-10-01T18:02:57.497Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/71/29/2493af60240ee7644b38a4b821f5f9c3b5a4fa3178770fe1d0c685217395/ruff-0.16.10-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:1dfc6f0088149fb6a362c1c446bcbb3fd2157b3852fe2fa68409276eab9ad9b3", size = 10537906, upload-time = "2026-10-01T18:03:00.006Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2e/9a/f56b28f3b143bb9e518c34fb89ab191b626bca8b70dbf8af0d2e2d473572/ruff-0.16.10-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:6553498afc35f580f036030795810b9e6bcea31604b0fd9e8d352795473042e3", size = 10012938, upload-time = "2026-10-01T18:03:03.006Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c2/c4/fca37362848ea4d4d80712df13632e645e7c7cfb1bdedf140699ac7a090b/ruff-0.16.10-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:a3b8471dea115d37f123882be852bed13403746d3a76c11de4a19ec5f9ff5a03", size = 9897945, upload-time = "2026-10-01T18:03:05.818Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7d/c7/e0bc57664d6e0c61fd22f620af260fe9662d7e1ddb320ea7f160e76ef165/ruff-0.16.10-py3-none-musllinux_1_2_i686.whl", hash = "sha256:92e59a70bcbd9d3a5483656da906ec28edfdacfce00afd99edb8b4e9d15644be", size = 10333134, upload-time = "2026-10-01T18:03:08.25Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/21/aa/5c9f3b68737c0e4a33a1db5dfab44f7b786d96ca91a7233112ddab1e6dc2/ruff-0.16.10-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:7ae7375f803b5520dc9f546bed7e3a0acb70b91812e9bb4b19927de22f25b77d", size = 10741702, upload-time = "2026-10-01T18:03:10.775Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/78/fa/0f9c2020dc316c53d983be011720b8157cc052b97e3f4dfa7db6d0f880a6/ruff-0.16.10-py3-none-win32.whl", hash = "sha256:2a12e01cb9156c10c466f63b46eaae5ecea28dfbd21b5836353ae498e7d1349a", size = 10139176, upload-time = "2026-10-01T18:03:13.267Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/99/29/cfb0df9448d4d4ad48c2de029ada9ebd71baa6da983a6c77ee6c6cd0fe82/ruff-0.16.10-py3-none-win_amd64.whl", hash = "sha256:97f2015c92aa97105b0eab19eb5d224884399281cfc5da86a92db4ab5e7fb2ca", size = 10584734, upload-time = "2026-10-01T18:03:16.006Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/fc/05/c16957eb287c3fc062e032619a25868d93d408a844b725bcf514f7a378ff/ruff-0.16.10-py3-none-win_arm64.whl", hash = "sha256:25a65fe998c4e6861ec079ada5826a2fc605e6cbccbe9dcd7fac1f54e791621b", size = 10366440, upload-time = "2026-10-01T18:03:19.04Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "sse-starlette"
|
||||||
|
version = "3.0.3"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "anyio" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/db/3c/fa6517610dc641262b77cc7bf994ecd17465812c1b0585fe33e11be758ab/sse_starlette-3.0.3.tar.gz", hash = "sha256:88cfb08747e16200ea990c8ca876b03910a23b547ab3bd764c0d8eb81019b971", size = 21943, upload-time = "2025-10-30T18:44:20.117Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/23/a0/984525d19ca5c8a6c33911a0c164b11490dd0f90ff7fd689f704f84e9a11/sse_starlette-3.0.3-py3-none-any.whl", hash = "sha256:af5bf5a6f3933df1d9c7f8539633dc8444ca6a97ab2e2a7cd3b6e431ac03a431", size = 11765, upload-time = "2025-10-30T18:44:18.834Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "starlette"
|
||||||
|
version = "0.41.3"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "anyio" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/1a/4c/9b5764bd22eec91c4039ef4c55334e9187085da2d8a2df7bd570869aae18/starlette-0.41.3.tar.gz", hash = "sha256:0e4ab3d16522a255be6b28260b938eae2482f98ce5cc934cb08dce8dc3ba5835", size = 2574159, upload-time = "2024-11-18T19:45:04.283Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/96/00/2b325970b3060c7cecebab6d295afe763365822b1306a12eeab198f74323/starlette-0.41.3-py3-none-any.whl", hash = "sha256:44cedb2b7c77a9de33a8b74b2b90e9f50d11fcf25d8270ea525ad71a25374ff7", size = 73225, upload-time = "2024-11-18T19:45:02.027Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "types-jsonschema"
|
||||||
|
version = "4.26.0.20261006"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "referencing" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/29/d2/1f742605f5a6d39f993134885b8de41c98af606871d3ebddaf3e776bd1eb/types_jsonschema-4.26.0.20261006.tar.gz", hash = "sha256:3eb7db61b6819d40addfdaac7173e749071a7d4a9a4394f0c844598ec84b2500", size = 16843, upload-time = "2026-10-06T08:16:07.318Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8b/a0/4f2e3c0dc3d5cad958006f2e0307065cc8fe4fbf0393ff0b69d55ee9bbe5/types_jsonschema-4.26.0.20261006-py3-none-any.whl", hash = "sha256:29301f4e65e3928540cdf5e23ad716e38bb0c0b416a48dada213edd3d70206ec", size = 16113, upload-time = "2026-10-06T08:16:06.355Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "typing-extensions"
|
||||||
|
version = "4.16.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/f6/cc/6253133b5bb138fc3306cebfbda2c520f545d36b5be2c7255cc528bb45d6/typing_extensions-4.16.0.tar.gz", hash = "sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5", size = 113555, upload-time = "2026-07-02T08:40:05.92Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl", hash = "sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8", size = 45571, upload-time = "2026-07-02T08:40:04.659Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "typing-inspection"
|
||||||
|
version = "0.4.4"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "typing-extensions" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/a3/26/b09b8010994eccc3c09092e6b34058f36a460eea2d4c3e8b910c695975a0/typing_inspection-0.4.4.tar.gz", hash = "sha256:547274fa6b0a561ccf549cc9524b999a578e737d015d8709d021f9d0d13bea47", size = 76928, upload-time = "2026-08-12T12:37:25.997Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/67/81/4add07e5172b7ac40d8ed5ff580409a7801a4fe26d529bdd915401dabfbe/typing_inspection-0.4.4-py3-none-any.whl", hash = "sha256:65b8397ba37ccbce054456aaccddfc91e6e3083c92824df348d96ca832f3f147", size = 14750, upload-time = "2026-08-12T12:37:24.648Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "uvicorn"
|
||||||
|
version = "0.34.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "click" },
|
||||||
|
{ name = "h11" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/4b/4d/938bd85e5bf2edeec766267a5015ad969730bb91e31b44021dfe8b22df6c/uvicorn-0.34.0.tar.gz", hash = "sha256:404051050cd7e905de2c9a7e61790943440b3416f49cb409f965d9dcd0fa73e9", size = 76568, upload-time = "2024-12-15T13:33:30.42Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/61/14/33a3a1352cfa71812a3a21e8c9bfb83f60b0011f5e36f2b1399d51928209/uvicorn-0.34.0-py3-none-any.whl", hash = "sha256:023dc038422502fa28a09c7a30bf2b6991512da7dcdb8fd35fe57cfc154126f4", size = 62315, upload-time = "2024-12-15T13:33:27.467Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[package.optional-dependencies]
|
||||||
|
standard = [
|
||||||
|
{ name = "colorama", marker = "sys_platform == 'win32'" },
|
||||||
|
{ name = "httptools" },
|
||||||
|
{ name = "python-dotenv" },
|
||||||
|
{ name = "pyyaml" },
|
||||||
|
{ name = "uvloop", marker = "platform_python_implementation != 'PyPy' and sys_platform != 'cygwin' and sys_platform != 'win32'" },
|
||||||
|
{ name = "watchfiles" },
|
||||||
|
{ name = "websockets" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "uvloop"
|
||||||
|
version = "0.23.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/fa/42/02c739ce85fb2ee8d99212c61417da8140c6b87e9d97c430bea520d76044/uvloop-0.23.0.tar.gz", hash = "sha256:28d160f51ab4da3b187063652e643dea6831072add4adc1e6d62afbe73b6be27", size = 2559185, upload-time = "2026-10-01T03:17:04.4Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/05/98/04e766a6de99e6f7f955ecb7829e8d5a557de3427cb85be2236de54dda0c/uvloop-0.23.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:93935ab27b6eaef4c3e5489aebc84284f0644592f7ab516df60ee1b27eaf5eb3", size = 1393055, upload-time = "2026-10-01T03:15:42.526Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/33/8a/499e7b863a848ede009539bce39806b66205da5f8779354228e785601144/uvloop-0.23.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:4448e9124537620f9c25d004c227bb5104440b58955c19bbd312d910af919a63", size = 768909, upload-time = "2026-10-01T03:15:43.974Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3d/95/a880f8ce3b87ac5b307c354e8ee480be4658d24bf01f87921d57e3530b4a/uvloop-0.23.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f7548ede3ee908cfabc0d068106e303a9a2d811af959cdf6ab85676344cedcda", size = 4419106, upload-time = "2026-10-01T03:15:45.551Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/51/27/c1d2f9fa977f8f42ea294604166df10e0027e6dc6cd17f85ede386c9bf36/uvloop-0.23.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:090865d8ce7a03986755a3ce711b7dd0d4b44eb14ab74368b717f3fad1180208", size = 4532597, upload-time = "2026-10-01T03:15:47.258Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/42/dd/2cb6a2c8a30ca55c07a882dd4ae4ceae0fa7d8c15b25b3b7cb9a4b6cf4ca/uvloop-0.23.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:bd6f2f81c7b9da99d301c0b16b82044e76fe887086e42e1590ecf520b94dbdac", size = 4230048, upload-time = "2026-10-01T03:15:49.119Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f4/52/29989cbaa4022dc4ef35c1dd60a4ab989e4c2065f341ed483ae71d2bd950/uvloop-0.23.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:a6ac96da66c35bf789bdcde78a88dc7d56b7907d8379648c54adc1c61594575d", size = 4394152, upload-time = "2026-10-01T03:15:50.829Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "virtualenv"
|
||||||
|
version = "21.14.5"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "distlib" },
|
||||||
|
{ name = "filelock" },
|
||||||
|
{ name = "packaging" },
|
||||||
|
{ name = "platformdirs" },
|
||||||
|
{ name = "python-discovery" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/c4/f9/f323b3b6058cff3853b31cf6a49c0425ed797bf9611572ec10d065e08bac/virtualenv-21.14.5.tar.gz", hash = "sha256:c4cb6c13e46b57225a999c7e22a09b163393878facc7ac4c059a57f46faa1647", size = 5467127, upload-time = "2026-10-02T17:42:25.613Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/df/93/084f10ebbecdedf7902ed68fbecc903f0e9c5742155bfffb05eccbc071da/virtualenv-21.14.5-py3-none-any.whl", hash = "sha256:b0651e0174982bba17cc6f0aaa2ef496730cb85a3f35eb5d1119d1c904f7d1da", size = 5487482, upload-time = "2026-10-02T17:42:23.38Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "vocero-api"
|
||||||
|
version = "0.1.0"
|
||||||
|
source = { editable = "." }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "fastapi" },
|
||||||
|
{ name = "httpx" },
|
||||||
|
{ name = "jsonschema" },
|
||||||
|
{ name = "mcp" },
|
||||||
|
{ name = "pydantic" },
|
||||||
|
{ name = "python-dotenv" },
|
||||||
|
{ name = "uvicorn", extra = ["standard"] },
|
||||||
|
{ name = "websockets" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[package.dev-dependencies]
|
||||||
|
dev = [
|
||||||
|
{ name = "mypy" },
|
||||||
|
{ name = "pre-commit" },
|
||||||
|
{ name = "pytest" },
|
||||||
|
{ name = "ruff" },
|
||||||
|
{ name = "types-jsonschema" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[package.metadata]
|
||||||
|
requires-dist = [
|
||||||
|
{ name = "fastapi", specifier = "==0.115.6" },
|
||||||
|
{ name = "httpx", specifier = "==0.28.1" },
|
||||||
|
{ name = "jsonschema", specifier = "==4.26.0" },
|
||||||
|
{ name = "mcp", specifier = "==1.29.0" },
|
||||||
|
{ name = "pydantic", specifier = "==2.13.5" },
|
||||||
|
{ name = "python-dotenv", specifier = "==1.2.2" },
|
||||||
|
{ name = "uvicorn", extras = ["standard"], specifier = "==0.34.0" },
|
||||||
|
{ name = "websockets", specifier = "==17.2" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[package.metadata.requires-dev]
|
||||||
|
dev = [
|
||||||
|
{ name = "mypy", specifier = ">=1.16,<2" },
|
||||||
|
{ name = "pre-commit", specifier = ">=4.4,<5" },
|
||||||
|
{ name = "pytest", specifier = ">=9,<10" },
|
||||||
|
{ name = "ruff", specifier = ">=0.12,<1" },
|
||||||
|
{ name = "types-jsonschema", specifier = ">=4.26.0.20261006" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "watchfiles"
|
||||||
|
version = "1.3.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "anyio" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/b3/68/e6aa0b77d217b31f8f486ec0cdfe5e00e6e38dc0be657e7d85819b9faf0a/watchfiles-1.3.0.tar.gz", hash = "sha256:99aee4a07847c06820765fd7b1b49ceac4f3f711ccb7d104655a33231de1c207", size = 108119, upload-time = "2026-09-21T09:08:55.664Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/68/fa/c0b840d5d8bafe640925408ce11095948c8945bb61be1e67d0ab629b872a/watchfiles-1.3.0-cp310-abi3-macosx_10_12_x86_64.whl", hash = "sha256:000b9688fc8133037a8b075c8ebf98f32844ff8964dda61e85c1db547dafc441", size = 399703, upload-time = "2026-09-21T09:08:13.518Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c7/8a/894799b485fe9473ad10422a0d9668e53fb78e3a2b5cc6061159572844ad/watchfiles-1.3.0-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:bbc1198edfdc90fda0600f825aa94150f428dfcbf8138746f55998e0e660d64c", size = 397647, upload-time = "2026-09-21T09:08:15.16Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/5a/61/f277cd5ead05f5d37b98b41fbdd0c943ee0c04d49738f62e902cb291cf43/watchfiles-1.3.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4af9c464d410e8c44b58ddf1cbec7c3a02c1cb2c6c80df9c74659ba2fbda0ab5", size = 454994, upload-time = "2026-09-21T09:08:16.587Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/46/66/ea4c01382975e53b1ac1b0a0ea1b353416b8127fb6e04f5c1c313119aab6/watchfiles-1.3.0-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:eb94e40b9a0db19636b5e3fed0a9803452ecc7381df13b385272abb779ee83ca", size = 460105, upload-time = "2026-09-21T09:08:17.961Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/31/c3/1a501b597817096c9831c255f6270c124ce0e2faeb74124be24dcc93b86d/watchfiles-1.3.0-cp310-abi3-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:7398930a2b76b9dc67a4e6b8bfc7baf30946892a86f417a3920c07bdd6febc30", size = 493761, upload-time = "2026-09-21T09:08:19.327Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/5a/16/77352d05e152c7786a016e9f1decdb4507e8aa5bcf3343c72142a0413d14/watchfiles-1.3.0-cp310-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:025b108f2d5cc2799cb941f79380df3e8b22bfb589c218e801767fe78d598c95", size = 577500, upload-time = "2026-09-21T09:08:20.723Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/95/fe/81001c55b24466fe9911135e50269fd3a29f43e446627a92adfab54553b6/watchfiles-1.3.0-cp310-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:be9ef3cd403d756a304a0a08a112163c63da7ec72c8c9d083c98818291ff8f29", size = 469209, upload-time = "2026-09-21T09:08:22.346Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8a/cf/e3ad894ed5909a7c44b0707e89bc25d19a3d78b3a7aec846119443b6ae56/watchfiles-1.3.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b5768b49e426fd5b550b012c866db347cdf15c398ef98dc557b6e6b72fa74cd1", size = 458353, upload-time = "2026-09-21T09:08:23.7Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7f/f5/2f106829e6e2bc91ad74614730f6df484202b9a4d20d927dbf2f08d37d44/watchfiles-1.3.0-cp310-abi3-manylinux_2_31_riscv64.whl", hash = "sha256:d36c72fcdc08f143d87316a4c4c3b064e340da2c7383d39803950740ed418935", size = 468707, upload-time = "2026-09-21T09:08:25.167Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/15/37/2741ab776b978669c1f3bba1e573f1a184f727ab8b4d3a6d9c61090800ed/watchfiles-1.3.0-cp310-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:f0c9865240e065a2247f4b5248528ae3d01e0e8ae071462c250c884dc16e3ee3", size = 632510, upload-time = "2026-09-21T09:08:26.532Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/48/64/db4a4684275fe9b696ee5118c194705c1423b3d42181c3f8ee70194d5c73/watchfiles-1.3.0-cp310-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:380f513d26cc2e598266b88d66456e07d67ef4be8f0f435a1154d9c44b43d509", size = 661723, upload-time = "2026-09-21T09:08:28.004Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/52/80/95ddb24f6a2f595c031c24432ab7b8aa7d6d1ada0c6d63a552166e06e8d1/watchfiles-1.3.0-cp310-abi3-win32.whl", hash = "sha256:509d9f74d2bec5c1f4cd868ddcfe0c9601f7ea53066dfcb19d8ede742c4de661", size = 277761, upload-time = "2026-09-21T09:08:29.335Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/48/a1/1c4a1b3c8030cd03206232612be8a6e8799093490c220b0fcb935d02b75a/watchfiles-1.3.0-cp310-abi3-win_amd64.whl", hash = "sha256:1acabde19b67e673274e89a04d613b7fb1f122e2c4b8ec9a581e59c149264b61", size = 290643, upload-time = "2026-09-21T09:08:30.45Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/82/e0/d2db577c632d26d08078c4390dd870b8555c4e558df25d22270f0166c71f/watchfiles-1.3.0-cp310-abi3-win_arm64.whl", hash = "sha256:d978cc1dd7ba44f5590d7de74f7a9c8be7262d9e469c2c0efb8bbfc2574321dc", size = 285510, upload-time = "2026-09-21T09:08:31.561Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "websockets"
|
||||||
|
version = "17.2"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/01/89/3f825ab71c242fffb62ea8fe638741c290f62f8d7aadf8125ff897747af3/websockets-17.2.tar.gz", hash = "sha256:36c2fb94c990cc2545143b12690e2de6c16300f9dbe5b4f33fa300cf57dc8792", size = 188355, upload-time = "2026-10-03T14:56:53.5Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/bc/de/87854af9b38fe4738fd85f7f21c5b49558ae20aec898880894e435f33375/websockets-17.2-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:916ebdfd82e7fc68041d36b2b5f60361b9abce1e087454da15f8bd004839e090", size = 217757, upload-time = "2026-10-03T14:53:23.029Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3a/2e/1e80b5efa41544f626d56bd15ccb53dbfc56bf28bf80ab9cd6f82c4b1d20/websockets-17.2-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:3621f3686397708b8eeabfd0a9d75267c1f29a7537d2fe31e65d099e71587fa4", size = 215439, upload-time = "2026-10-03T14:53:24.531Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3b/6e/82c78b595aee05be76a7ee78539323da1593c1848e4fef51c704c696568f/websockets-17.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:a81e19710d48da88653473b6b9c366d47e99fe4f58e37ce415be47966748f31f", size = 215703, upload-time = "2026-10-03T14:53:26.226Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f8/c4/905ef6aa80423c03dba99e1e26fc0acf63a2a9a6a2d9e8c0e6a63caaf952/websockets-17.2-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:f2731f9067976c8c4127212c0d2f2ada42d497d935e470419e029802365b12bb", size = 225023, upload-time = "2026-10-03T14:53:27.744Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/03/c0/a6d8be9c43e4456fb9597fdf8b5e0ce1f0a5df41503acce6d869536e4e23/websockets-17.2-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:6627b913b8586b1c06db9516b31dd0dfbc621de3bb9312616d92a7e44f268a5b", size = 225299, upload-time = "2026-10-03T14:53:29.171Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2f/d4/976d34b5491258b0a86c2ce9b9aabb9fdd68919ffd7fe65999c14a502a98/websockets-17.2-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0198c4ec6a3406a2f7557c032967de426474c2c995c81076585e09d29a9f407b", size = 226540, upload-time = "2026-10-03T14:53:31.635Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/83/2f/c4cfd42f53c697a8ed123fd82b8f85fcd13b6360d47f9f1d1d45d6ec6627/websockets-17.2-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:88c6a42c2632ff469e84155e44f6ed92cb15ccb047bf5fcb59225ae5a12fd33d", size = 229371, upload-time = "2026-10-03T14:53:33.061Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e7/55/9a221b29c6232ff9282eecb2fc102402cb9e42a3479264db0e5fc4fe6835/websockets-17.2-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:eb0023e6cdb4b8ece0b33875188dd16104ad8c335361d396a98394f99e30ff7a", size = 227173, upload-time = "2026-10-03T14:53:34.502Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8f/07/125e6d010c56c253d3d2b93cabaea0f96d33898151a16b49066a594acecf/websockets-17.2-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c1c09d5d4646eb96bda2cfb97493bcea21a0956a981de116e6b1f4a9de07f3fd", size = 225929, upload-time = "2026-10-03T14:53:36.071Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/23/a8/aad3bd902aee84e1b261ad6ab83b405e4a564af43101b8ad1dc0293ff4f4/websockets-17.2-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:0360c4dc13ac569cc245e0efa2f4d4b1e4733d24c47b8ab3f3747227b1356348", size = 223167, upload-time = "2026-10-03T14:53:37.528Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1f/f4/ec8ab9be1a5310b4fea829f088c7aa2b7a58b61d34bce1b2a9338635ff12/websockets-17.2-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:76693a16dead737946b651375ee3109d7db7ad9569a1c55c60aaed3ef85cfcc6", size = 225974, upload-time = "2026-10-03T14:53:38.959Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/65/45/ba6503f8257d3f98b0f07ebaad0fd099c9023eae744fd5b775416743597e/websockets-17.2-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:77a42cc507993ec5471b5283f7eef869239173b6000031543e3938a86d1af0fd", size = 224581, upload-time = "2026-10-03T14:53:40.496Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d0/45/05cca59a876c6776727d96fc7ba59e0b6f9aa496afbf13e7e04ad0b63678/websockets-17.2-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:3bbc5543e39ee025d524077c5c15c2d67bc11c9f6676afe5b531839e24d701f6", size = 225347, upload-time = "2026-10-03T14:53:42.061Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1c/00/cf0e43292ae949b13f67535be84317102891d69fd1986ec2bf2ead42747b/websockets-17.2-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:8da58558bfb0ca6ccac2419773521f1111e40654038b1afabdfc69c02cb82614", size = 226457, upload-time = "2026-10-03T14:53:43.575Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/79/0d/9a5c61a18f0cc9876d94c70ccb3daf7614a9fee56abbb37c0e64e757fb96/websockets-17.2-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:01420cb1cb47433e8e7075d32cb8017ad3ffed0654bd1e48c0251b865920dec3", size = 224011, upload-time = "2026-10-03T14:53:45.077Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/34/ed/991c1ab80ab2ce40e1c939fef6fa8f971c3ef3b21caf988a7a107e0ad27d/websockets-17.2-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:c49c9edd47d0e44d360299e2d8865e2950d2fcf1b4098782c9d7dcd070919e5a", size = 224990, upload-time = "2026-10-03T14:53:46.8Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e7/7a/363c835d17923e967fb66376188e67b9a261c85d826a0cd5e4dd3471221d/websockets-17.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:96f6c8d0fe21930d1f982bfce2382789d2e8d005d2ab63d21280660f95ef8fe1", size = 225265, upload-time = "2026-10-03T14:53:48.382Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c8/90/6c51f6d78636bd1cd6781fae8ea5ea7bf1d5b4059354f3c1f5f8de793338/websockets-17.2-cp312-cp312-win32.whl", hash = "sha256:b25659ab2d655d742701487d5591e3f98e8f8b329fc999e05e3d59691ab344a1", size = 218228, upload-time = "2026-10-03T14:53:49.867Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c6/2a/90008411c652dcfae34345a2169f4becd066a4ba71eebfa8dd801e0445e1/websockets-17.2-cp312-cp312-win_amd64.whl", hash = "sha256:faa763b677e96f1beccc6b4d7e8c079dfeed2f249f57a19debc321b519ee64ec", size = 218528, upload-time = "2026-10-03T14:53:51.486Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1f/a1/b8ad6c17f8e75ba2215422fffe0d7f0c4b690dcff1c47c0473db0d253d51/websockets-17.2-cp312-cp312-win_arm64.whl", hash = "sha256:63499fc49efe48bccc2fca40723bc7adb198866cbe159093dd979905316994b6", size = 218457, upload-time = "2026-10-03T14:53:52.938Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8a/58/835cd51934d6780fa586f275b5d9901eead6d81569b4343b3767cdbaae4c/websockets-17.2-py3-none-any.whl", hash = "sha256:6aa59f0ef92e796b2db6f5f26550c4713c0e4036899fadf02f55e2ed4db0b7ae", size = 211883, upload-time = "2026-10-03T14:56:51.898Z" },
|
||||||
|
]
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
"""Validate service-owned response structure without rewriting the wire body."""
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
|
||||||
|
def validated_json_response(
|
||||||
|
content: Any,
|
||||||
|
model: type[BaseModel],
|
||||||
|
*,
|
||||||
|
status_code: int = 200,
|
||||||
|
) -> JSONResponse:
|
||||||
|
model.model_validate(content)
|
||||||
|
# model_dump would insert defaults, potentially coerce values, and reorder
|
||||||
|
# fields. Keeping the original JSONResponse serialization is intentional.
|
||||||
|
return JSONResponse(content=content, status_code=status_code)
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
"""Register routes in their original order without changing public paths."""
|
||||||
|
|
||||||
|
from fastapi import APIRouter
|
||||||
|
|
||||||
|
from vocero_api.api.routes import caller, config, conversations, search, sessions
|
||||||
|
from vocero_api.models.responses import ReasoningResponse
|
||||||
|
from vocero_api.realtime.proxy import realtime_proxy
|
||||||
|
from vocero_api.services.reasoning import reasoning
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
router.include_router(config.router)
|
||||||
|
router.include_router(conversations.router)
|
||||||
|
router.add_api_route("/api/reasoning", reasoning, methods=["POST"], response_model=ReasoningResponse)
|
||||||
|
router.add_api_websocket_route("/v1/realtime", realtime_proxy)
|
||||||
|
router.include_router(caller.router)
|
||||||
|
router.include_router(search.router)
|
||||||
|
router.include_router(sessions.router)
|
||||||
@@ -0,0 +1,293 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from urllib.parse import urlencode
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from fastapi import APIRouter, HTTPException
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
|
||||||
|
from vocero_api.api.responses import validated_json_response
|
||||||
|
from vocero_api.core import config as settings
|
||||||
|
from vocero_api.core import logging as app_logging
|
||||||
|
from vocero_api.models.responses import ReportsResponse, UserProfileResponse
|
||||||
|
from vocero_api.providers import caller as providers_caller
|
||||||
|
from vocero_api.services import context as services_context
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/api/user/phone", response_model=UserProfileResponse)
|
||||||
|
async def user_by_phone(phoneNumber: str) -> JSONResponse:
|
||||||
|
"""HTTP passthrough: fetches caller identity / profile from the external
|
||||||
|
info endpoint set via VOCERO_USER_PHONE_AUTH_URL and forwards the raw
|
||||||
|
JSON to the browser.
|
||||||
|
|
||||||
|
The proxy NEVER inspects this payload nor injects it into the realtime
|
||||||
|
WebSocket session. It is only used by:
|
||||||
|
- the browser UI to render a caller name header (#ident-user-name)
|
||||||
|
- the /api/reasoning HTTP body user_context field (browser's choice).
|
||||||
|
|
||||||
|
When VOCERO_USER_PHONE_AUTH_URL is unset we return 503 so the UI can
|
||||||
|
silently skip identity loading (backward compatible with generic deploys).
|
||||||
|
|
||||||
|
When S2S_LOGIN_BY_PHONE is set, the proxy first performs a login-by-phone
|
||||||
|
POST with the caller phone and (if a bearer token is extracted) forwards
|
||||||
|
it as Authorization: Bearer on the subsequent profile GET.
|
||||||
|
"""
|
||||||
|
|
||||||
|
if settings.S2S_MCP_URL:
|
||||||
|
raise HTTPException(409, "Usa la sesión de conversación para consultar AquaPortal.")
|
||||||
|
if not settings.VOCERO_USER_PHONE_AUTH_URL:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=503,
|
||||||
|
detail="VOCERO_USER_PHONE_AUTH_URL no está configurado en este despliegue.",
|
||||||
|
)
|
||||||
|
phone = (phoneNumber or "").strip()
|
||||||
|
if not phone:
|
||||||
|
raise HTTPException(status_code=400, detail="Se requiere el campo phoneNumber.")
|
||||||
|
|
||||||
|
# 0) Login-by-phone (optional — only if S2S_LOGIN_BY_PHONE env is set).
|
||||||
|
login_token = None
|
||||||
|
if settings.S2S_LOGIN_BY_PHONE:
|
||||||
|
login_token, _login_obj = await providers_caller._login_by_phone(phone)
|
||||||
|
|
||||||
|
target_url = f"{settings.VOCERO_USER_PHONE_AUTH_URL}?{urlencode({'phoneNumber': phone})}"
|
||||||
|
app_logging.logger.debug(
|
||||||
|
"[EXT][USER-BY-PHONE] GET upstream=%s phoneLen=%d bearer=%s",
|
||||||
|
settings.VOCERO_USER_PHONE_AUTH_URL,
|
||||||
|
len(phone),
|
||||||
|
"yes" if login_token else "no",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
headers = {}
|
||||||
|
if login_token:
|
||||||
|
headers["Authorization"] = f"Bearer {login_token}"
|
||||||
|
async with httpx.AsyncClient(timeout=15.0, follow_redirects=True) as http:
|
||||||
|
resp = await http.get(target_url, headers=headers or None)
|
||||||
|
except httpx.RequestError as exc:
|
||||||
|
app_logging.logger.warning("[EXT][USER-BY-PHONE] unreachable: %r", exc)
|
||||||
|
raise HTTPException(status_code=502, detail="No se puede conectar con el servicio de información del usuario.") # noqa: B904
|
||||||
|
# Forward the upstream status + JSON verbatim; never rewrite the payload.
|
||||||
|
if resp.status_code >= 500:
|
||||||
|
app_logging.logger.warning(
|
||||||
|
"[EXT][USER-BY-PHONE] upstream HTTP %s: %.200s",
|
||||||
|
resp.status_code,
|
||||||
|
resp.text,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
payload = resp.json()
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException( # noqa: B904
|
||||||
|
status_code=502,
|
||||||
|
detail="El servicio de información del usuario devolvió un JSON no válido.",
|
||||||
|
)
|
||||||
|
return validated_json_response(
|
||||||
|
payload, UserProfileResponse, status_code=resp.status_code if resp.status_code != 204 else 200
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/api/user/reports", response_model=ReportsResponse)
|
||||||
|
async def user_reports(phoneNumber: str | None = None) -> JSONResponse:
|
||||||
|
"""Fetch + normalize the scheduled reports list for the caller UI.
|
||||||
|
|
||||||
|
Mirrors the same sequence used in the WebSocket connect handler and
|
||||||
|
/api/reasoning:
|
||||||
|
0. login-by-phone (optional) → bearer token
|
||||||
|
1. GET S2S_REPORT_LIST with ?phoneNumber= (if phone provided) + Bearer auth
|
||||||
|
2. run the same resolver used for prompt injection: filter 3 fields,
|
||||||
|
sort DESC, build hints + entries → JSON returned verbatim so the
|
||||||
|
browser can render a "reports loaded" summary in the UI header.
|
||||||
|
|
||||||
|
Returns 503 when S2S_REPORT_LIST is not configured (generic deploy with no
|
||||||
|
reports feature enabled) so the browser silently skips UI rendering.
|
||||||
|
Response shape on success:
|
||||||
|
{
|
||||||
|
"count": N,
|
||||||
|
"wrappedLen": X,
|
||||||
|
"hints": [ "• Report #1: '<label>' — scheduled on <date>", ... ],
|
||||||
|
"entries": [
|
||||||
|
{"sourceScheduleLabel": "...", "date": "...", "podcastScript": "..."},
|
||||||
|
...
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
USER FIX 2026-09-11: remove mass injection of scheduled reports.
|
||||||
|
• Even this BROWSER-FACING endpoint (/api/user/reports) now honours the new
|
||||||
|
S2S_INJECT_REPORTS default=False gate: when S2S_INJECT_REPORTS is not
|
||||||
|
explicitly true, we return a polite 403 + empty response — the endpoint
|
||||||
|
does NOT execute any upstream S2S_REPORT_LIST HTTP call, does NOT expose
|
||||||
|
any podcastScript bodies, and does NOT emit entries/hints. This avoids
|
||||||
|
a subtle loophole where the browser could still fetch scheduled reports
|
||||||
|
through /api/user/reports while the LLM prompt paths were gated.
|
||||||
|
"""
|
||||||
|
if settings.S2S_MCP_URL:
|
||||||
|
raise HTTPException(409, "Los reportes se consultan durante la conversación con AquaPortal.")
|
||||||
|
# ── MASS-INJECTION KILL SWITCH (user request) ─────────────────────────
|
||||||
|
if not settings.S2S_INJECT_REPORTS:
|
||||||
|
return validated_json_response(
|
||||||
|
model=ReportsResponse,
|
||||||
|
status_code=200,
|
||||||
|
content={
|
||||||
|
"count": 0,
|
||||||
|
"wrappedLen": 0,
|
||||||
|
"hints": [],
|
||||||
|
"entries": [],
|
||||||
|
"skipped": True,
|
||||||
|
"reason": "S2S_INJECT_REPORTS está deshabilitado en este despliegue; no se muestran reportes programados en la interfaz ni se envían al modelo.",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
# ───────────────────────────────────────────────────────────────────────
|
||||||
|
if not settings.S2S_REPORT_LIST:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=503,
|
||||||
|
detail="S2S_REPORT_LIST no está configurado en este despliegue.",
|
||||||
|
)
|
||||||
|
phone = (phoneNumber or "").strip() or None
|
||||||
|
# 0) Login-by-phone if enabled. Bearer token used only if login succeeded
|
||||||
|
# (same rule as WS connect / reasoning handlers — never-fail).
|
||||||
|
login_token = None
|
||||||
|
if settings.S2S_LOGIN_BY_PHONE and phone:
|
||||||
|
login_token, _login_obj = await providers_caller._login_by_phone(phone)
|
||||||
|
|
||||||
|
reports_raw = await providers_caller._fetch_reports_list(phone, login_token)
|
||||||
|
# Re-run the same resolver used for prompt injection → identical logic
|
||||||
|
# guarantees UI and LLM see exactly the same filtered/sorted list.
|
||||||
|
ctx_str = services_context._resolve_reports_context(reports_raw)
|
||||||
|
if not ctx_str or not isinstance(reports_raw, list):
|
||||||
|
return validated_json_response(
|
||||||
|
{
|
||||||
|
"count": 0,
|
||||||
|
"wrappedLen": 0,
|
||||||
|
"hints": [],
|
||||||
|
"entries": [],
|
||||||
|
"skipped": True,
|
||||||
|
"reason": "Lista vacía, error del servicio de origen o ningún contenido disponible.",
|
||||||
|
},
|
||||||
|
ReportsResponse,
|
||||||
|
)
|
||||||
|
# Rebuild hints + normalized entries with the exact filtered output
|
||||||
|
items_list: list = reports_raw
|
||||||
|
# Re-derive filtered + sorted list (same as _resolve_reports_context) so
|
||||||
|
# we don't accidentally expose raw unfiltered upstream payload to the UI.
|
||||||
|
filtered: list[dict] = []
|
||||||
|
for item in items_list:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
cleaned: dict = {}
|
||||||
|
label_val: object = None
|
||||||
|
if item.get("sourceScheduleLabel") is not None:
|
||||||
|
label_val = item["sourceScheduleLabel"]
|
||||||
|
elif isinstance(item.get("sourceSchedule"), dict):
|
||||||
|
ss = item["sourceSchedule"]
|
||||||
|
for k in ("label", "name", "title"):
|
||||||
|
if ss.get(k) is not None:
|
||||||
|
label_val = ss[k]
|
||||||
|
break
|
||||||
|
if label_val is not None:
|
||||||
|
cleaned["sourceScheduleLabel"] = label_val
|
||||||
|
date_val: object = None
|
||||||
|
if item.get("date") is not None:
|
||||||
|
date_val = item["date"]
|
||||||
|
elif isinstance(item.get("sourceSchedule"), dict):
|
||||||
|
ss = item["sourceSchedule"]
|
||||||
|
for k in ("date", "scheduledAt", "publishedAt"):
|
||||||
|
if ss.get(k) is not None:
|
||||||
|
date_val = ss[k]
|
||||||
|
break
|
||||||
|
if date_val is not None:
|
||||||
|
cleaned["date"] = date_val
|
||||||
|
script_val: object = None
|
||||||
|
for k in ("podcastScript", "script", "text", "content"):
|
||||||
|
if item.get(k) is not None:
|
||||||
|
script_val = item[k]
|
||||||
|
break
|
||||||
|
if script_val is not None:
|
||||||
|
cleaned["podcastScript"] = script_val
|
||||||
|
if cleaned:
|
||||||
|
filtered.append(cleaned)
|
||||||
|
filtered.sort(key=lambda r: services_context._parse_date_key(r.get("date")))
|
||||||
|
# 2026-09-11 user fix: configure report quick-hint limits through .env.
|
||||||
|
# Apply the SAME cap used by the LLM prompt resolver: after sort by newest,
|
||||||
|
# keep only first S2S_REPORTS_QUICK_HINT_MAX entries. This keeps counts between
|
||||||
|
# /api/user/reports UI JSON and the LLM prompt identical — avoids confusing
|
||||||
|
# mismatches where the UI shows 20 hints but the prompt only saw 5.
|
||||||
|
total_raw_ui = len(filtered)
|
||||||
|
cap_ui = max(1, int(settings.S2S_REPORTS_QUICK_HINT_MAX or 5))
|
||||||
|
truncated_ui_note: str | None = None
|
||||||
|
if total_raw_ui > cap_ui:
|
||||||
|
filtered = filtered[:cap_ui]
|
||||||
|
truncated_ui_note = (
|
||||||
|
f"Se muestran los {cap_ui} reportes más recientes de un total de {total_raw_ui}. "
|
||||||
|
"Ajuste la variable de entorno S2S_REPORTS_QUICK_HINT_MAX para cambiar la cantidad."
|
||||||
|
)
|
||||||
|
import datetime as _dt
|
||||||
|
|
||||||
|
# Deduplicate by label for the UI JSON response only.
|
||||||
|
# NOTE: the LLM prompt injection uses the full non-deduped list (different
|
||||||
|
# dates = different content even when the scheduled report name repeats).
|
||||||
|
# We only dedup entries_out, hints_list, and entries for the browser payload.
|
||||||
|
seen_labels: set[str] = set()
|
||||||
|
dedup_filtered: list[dict] = []
|
||||||
|
for r in filtered:
|
||||||
|
lbl_raw = r.get("sourceScheduleLabel")
|
||||||
|
if lbl_raw is None:
|
||||||
|
# Items with no label are kept as-is (can't meaningfully dedupe)
|
||||||
|
dedup_filtered.append(r)
|
||||||
|
continue
|
||||||
|
key = str(lbl_raw).strip()
|
||||||
|
if not key:
|
||||||
|
dedup_filtered.append(r)
|
||||||
|
continue
|
||||||
|
if key in seen_labels:
|
||||||
|
continue
|
||||||
|
seen_labels.add(key)
|
||||||
|
dedup_filtered.append(r)
|
||||||
|
hints_list: list[str] = []
|
||||||
|
entries_out: list[dict] = []
|
||||||
|
for idx, r in enumerate(dedup_filtered, start=1):
|
||||||
|
entries_out.append(r)
|
||||||
|
label_piece = str(r["sourceScheduleLabel"]).strip() if r.get("sourceScheduleLabel") else ""
|
||||||
|
date_piece = ""
|
||||||
|
if r.get("date") is not None:
|
||||||
|
raw_d = r["date"]
|
||||||
|
try:
|
||||||
|
if isinstance(raw_d, (int, float)) and not isinstance(raw_d, bool):
|
||||||
|
ts_sec = float(raw_d)
|
||||||
|
if ts_sec > 1e12:
|
||||||
|
ts_sec = ts_sec / 1000.0
|
||||||
|
dt = _dt.datetime.fromtimestamp(ts_sec, tz=_dt.UTC)
|
||||||
|
date_piece = dt.strftime("%Y-%m-%d %H:%M UTC")
|
||||||
|
else:
|
||||||
|
s = str(raw_d).strip()
|
||||||
|
if "T" in s and len(s) >= 16:
|
||||||
|
date_piece = s.replace("T", " ")[:16]
|
||||||
|
else:
|
||||||
|
date_piece = s
|
||||||
|
except Exception:
|
||||||
|
date_piece = str(raw_d)[:32]
|
||||||
|
if label_piece and date_piece:
|
||||||
|
hints_list.append(f"• Reporte #{idx}: '{label_piece}' — programado para {date_piece}")
|
||||||
|
elif label_piece:
|
||||||
|
hints_list.append(f"• Reporte #{idx}: '{label_piece}'")
|
||||||
|
elif date_piece:
|
||||||
|
hints_list.append(f"• Reporte #{idx}: (sin título) — programado para {date_piece}")
|
||||||
|
resp_body = {
|
||||||
|
"count": len(entries_out),
|
||||||
|
"rawCount": total_raw_ui,
|
||||||
|
"cap": cap_ui,
|
||||||
|
"wrappedLen": len(ctx_str),
|
||||||
|
"hints": hints_list,
|
||||||
|
# Strip the long podcastScript from the summary JSON to keep the
|
||||||
|
# browser response lightweight (UI only needs label/date).
|
||||||
|
"entries": [
|
||||||
|
{
|
||||||
|
"sourceScheduleLabel": r.get("sourceScheduleLabel"),
|
||||||
|
"date": r.get("date"),
|
||||||
|
}
|
||||||
|
for r in entries_out
|
||||||
|
],
|
||||||
|
}
|
||||||
|
if truncated_ui_note is not None:
|
||||||
|
resp_body["truncated"] = True
|
||||||
|
resp_body["note"] = truncated_ui_note
|
||||||
|
return validated_json_response(resp_body, ReportsResponse)
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, Request
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
|
||||||
|
from vocero_api.api.responses import validated_json_response
|
||||||
|
from vocero_api.core import config as settings
|
||||||
|
from vocero_api.core import logging as app_logging
|
||||||
|
from vocero_api.core import urls as core_urls
|
||||||
|
from vocero_api.models.responses import ConfigResponse
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
def _effective_s2s_url(request: Request) -> str:
|
||||||
|
return settings.SPEECH_TO_SPEECH_URL or core_urls._proxy_public_url(request)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/api/config", response_model=ConfigResponse)
|
||||||
|
async def config(request: Request) -> JSONResponse:
|
||||||
|
"""Client bootstrap: whether web search is available, whether the deploy runs
|
||||||
|
behind a load balancer (so the browser uses the /api/session proxy),
|
||||||
|
and whether the user may instead set a direct
|
||||||
|
s2s server URL. The LB address itself is intentionally NOT included."""
|
||||||
|
app_logging.logger.debug(
|
||||||
|
"[INTERNAL] /api/config requested client=%s", request.client.host if request.client else "?"
|
||||||
|
)
|
||||||
|
primary_prompt = settings.S2S_PRIMARY_PROMPT
|
||||||
|
out = {
|
||||||
|
"search": bool(settings.SERPER_KEY),
|
||||||
|
"lb": bool(settings.LOAD_BALANCER_URL),
|
||||||
|
"allowDirect": not settings.LOAD_BALANCER_URL,
|
||||||
|
"s2sUrl": _effective_s2s_url(request),
|
||||||
|
"primaryPrompt": primary_prompt,
|
||||||
|
"reasoningType": settings.S2S_REASONING_TYPE,
|
||||||
|
"injectProfile": settings.S2S_INJECT_PROFILE,
|
||||||
|
"debugPrompt": settings.S2S_DEBUG_PROMPT,
|
||||||
|
"startSessionMode": settings.S2S_START_SESSION_MODE,
|
||||||
|
"theme": settings.S2S_THEME,
|
||||||
|
}
|
||||||
|
if settings.S2S_MCP_URL:
|
||||||
|
if settings.S2S_REASONING_TYPE == "PROXY" and (
|
||||||
|
not settings.S2S_PROXY_TARGET or settings.SPEECH_TO_SPEECH_URL or settings.LOAD_BALANCER_URL
|
||||||
|
):
|
||||||
|
raise HTTPException(503, "MCP en modo PROXY requiere la conexión de voz a través de Vocero.")
|
||||||
|
out["mcp"] = True
|
||||||
|
app_logging.logger.debug(
|
||||||
|
"[INTERNAL] /api/config returned promptLen=%d reasoning=%s startSessionMode=%s theme=%s mcp=%s",
|
||||||
|
len(primary_prompt),
|
||||||
|
settings.S2S_REASONING_TYPE,
|
||||||
|
settings.S2S_START_SESSION_MODE or "-",
|
||||||
|
settings.S2S_THEME or "-",
|
||||||
|
bool(settings.S2S_MCP_URL),
|
||||||
|
)
|
||||||
|
return validated_json_response(out, ConfigResponse)
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
"""Browser-owned conversations; backend credentials never cross this boundary."""
|
||||||
|
|
||||||
|
import secrets
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, Request, Response
|
||||||
|
|
||||||
|
from vocero_api.core import config as settings
|
||||||
|
from vocero_api.models.mcp import ConversationRequest, ConversationResponse, IntegrationError, McpCatalogResponse
|
||||||
|
from vocero_api.providers import backend_login
|
||||||
|
from vocero_api.services import content_context, conversations
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
def require_mcp() -> None:
|
||||||
|
if not settings.S2S_MCP_URL:
|
||||||
|
raise HTTPException(404, "No encontrado.")
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/api/conversations", response_model=ConversationResponse)
|
||||||
|
async def create_conversation(body: ConversationRequest, request: Request, response: Response) -> ConversationResponse:
|
||||||
|
require_mcp()
|
||||||
|
try:
|
||||||
|
conversations.check_origin(request)
|
||||||
|
login = await backend_login.login(body.phoneNumber.strip())
|
||||||
|
linked_content = await content_context.from_link(
|
||||||
|
login.sessionToken.get_secret_value(), body.report_id, body.episode_id
|
||||||
|
)
|
||||||
|
browser = request.cookies.get(conversations.COOKIE_NAME) or secrets.token_urlsafe(32)
|
||||||
|
item = conversations.store.create(browser, login, linked_content)
|
||||||
|
except IntegrationError as exc:
|
||||||
|
raise HTTPException(exc.status, exc.message) from None
|
||||||
|
response.set_cookie(
|
||||||
|
conversations.COOKIE_NAME,
|
||||||
|
browser,
|
||||||
|
httponly=True,
|
||||||
|
secure=request.url.scheme == "https" or request.headers.get("x-forwarded-proto") == "https",
|
||||||
|
samesite="strict",
|
||||||
|
)
|
||||||
|
response.headers["Cache-Control"] = "no-store"
|
||||||
|
return ConversationResponse(
|
||||||
|
conversationId=item.id, expiresIn=settings.S2S_CONVERSATION_TTL_SEC, profile=item.profile
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/api/conversations/current", status_code=204)
|
||||||
|
async def end_conversation(request: Request) -> Response:
|
||||||
|
require_mcp()
|
||||||
|
try:
|
||||||
|
item = conversations.resolve(request)
|
||||||
|
conversations.store.remove(item.id)
|
||||||
|
except IntegrationError as exc:
|
||||||
|
raise HTTPException(exc.status, exc.message) from None
|
||||||
|
return Response(status_code=204)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/api/conversations/current/interrupt", status_code=204)
|
||||||
|
async def interrupt_conversation(request: Request) -> Response:
|
||||||
|
require_mcp()
|
||||||
|
try:
|
||||||
|
item = conversations.resolve(request)
|
||||||
|
if request.headers.get("x-vocero-turn") == str(item.client_turn):
|
||||||
|
item.interrupt()
|
||||||
|
except IntegrationError as exc:
|
||||||
|
raise HTTPException(exc.status, exc.message) from None
|
||||||
|
return Response(status_code=204)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/api/mcp/tools", response_model=McpCatalogResponse)
|
||||||
|
async def list_tools(request: Request, response: Response) -> McpCatalogResponse:
|
||||||
|
require_mcp()
|
||||||
|
item: conversations.Conversation | None = None
|
||||||
|
try:
|
||||||
|
item = conversations.resolve(request)
|
||||||
|
catalog = await content_context.catalog(item.backend_token)
|
||||||
|
item.require_active()
|
||||||
|
item.catalog = catalog
|
||||||
|
response.headers["Cache-Control"] = "no-store"
|
||||||
|
return McpCatalogResponse(tools=catalog)
|
||||||
|
except IntegrationError as exc:
|
||||||
|
if exc.status == 401 and item is not None:
|
||||||
|
conversations.store.remove(item.id)
|
||||||
|
raise HTTPException(exc.status, exc.message) from None
|
||||||
@@ -0,0 +1,92 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from fastapi import APIRouter, HTTPException
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
|
||||||
|
from vocero_api.api.responses import validated_json_response
|
||||||
|
from vocero_api.core import config as settings
|
||||||
|
from vocero_api.core import logging as app_logging
|
||||||
|
from vocero_api.models import requests as models_requests
|
||||||
|
from vocero_api.models.responses import SearchResponse
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/api/search", response_model=SearchResponse)
|
||||||
|
async def search(req: models_requests.SearchRequest) -> JSONResponse:
|
||||||
|
"""Proxy a Google search via Serper.dev. The key stays on the server unless
|
||||||
|
the user brought their own (then theirs is used for this request only)."""
|
||||||
|
query = (req.query or "").strip()
|
||||||
|
if not query:
|
||||||
|
raise HTTPException(status_code=400, detail="La consulta está vacía.")
|
||||||
|
|
||||||
|
key = (req.key or "").strip() or settings.SERPER_KEY
|
||||||
|
has_user_key = bool((req.key or "").strip())
|
||||||
|
if not key:
|
||||||
|
# No server key and the user didn't supply one — search is unavailable.
|
||||||
|
raise HTTPException(status_code=503, detail="La búsqueda no está configurada.")
|
||||||
|
|
||||||
|
headers = {"X-API-KEY": key, "Content-Type": "application/json"}
|
||||||
|
payload = {"q": query, "num": settings.MAX_RESULTS}
|
||||||
|
app_logging.logger.debug(
|
||||||
|
"[EXT][SERPER] search POST %s query=%s userKey=%s num=%d",
|
||||||
|
settings.SERPER_URL,
|
||||||
|
json.dumps(query, ensure_ascii=True)[:200],
|
||||||
|
has_user_key,
|
||||||
|
settings.MAX_RESULTS,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
async with httpx.AsyncClient(timeout=12.0) as http:
|
||||||
|
resp = await http.post(settings.SERPER_URL, headers=headers, json=payload)
|
||||||
|
except httpx.RequestError as exc:
|
||||||
|
app_logging.logger.warning("Serper unreachable: %r", exc)
|
||||||
|
raise HTTPException(status_code=502, detail="No se puede conectar con el proveedor de búsqueda.") # noqa: B904
|
||||||
|
app_logging.logger.debug(
|
||||||
|
"[EXT][SERPER] search RESP status=%s len=%dB",
|
||||||
|
resp.status_code,
|
||||||
|
len(resp.content or b""),
|
||||||
|
)
|
||||||
|
|
||||||
|
if resp.status_code != 200:
|
||||||
|
# Serper's error body carries the real reason (e.g. "Not enough
|
||||||
|
# credits") and contains no key, so it's safe to log and relay.
|
||||||
|
body = resp.text[:300]
|
||||||
|
app_logging.logger.warning("Serper error %s: %s", resp.status_code, body)
|
||||||
|
msg = None
|
||||||
|
try:
|
||||||
|
msg = resp.json().get("message")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
detail = f"Error del proveedor de búsqueda ({resp.status_code})"
|
||||||
|
if msg:
|
||||||
|
detail += f": {msg}"
|
||||||
|
raise HTTPException(status_code=502, detail=detail)
|
||||||
|
|
||||||
|
data = resp.json()
|
||||||
|
results = []
|
||||||
|
for item in (data.get("organic") or [])[: settings.MAX_RESULTS]:
|
||||||
|
results.append(
|
||||||
|
{
|
||||||
|
"title": item.get("title", ""),
|
||||||
|
"snippet": item.get("snippet", ""),
|
||||||
|
"url": item.get("link", ""),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
# A direct answer when Google has one — saves the model a hop.
|
||||||
|
box = data.get("answerBox") or {}
|
||||||
|
answer = box.get("answer") or box.get("snippet") or None
|
||||||
|
if not answer:
|
||||||
|
kg = data.get("knowledgeGraph") or {}
|
||||||
|
answer = kg.get("description") or None
|
||||||
|
|
||||||
|
app_logging.logger.debug(
|
||||||
|
"[EXT][SERPER] search PARSED hasAnswer=%s resultCount=%d titles=%s",
|
||||||
|
bool(answer),
|
||||||
|
len(results),
|
||||||
|
[r.get("title", "")[:40] for r in results],
|
||||||
|
)
|
||||||
|
return validated_json_response({"query": query, "answer": answer, "results": results}, SearchResponse)
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user