client sdk

UPDATES.md

@@ -5,6 +5,267 @@ Entries are written in English and ordered newest-first.
 
 ---
 
+## 2026-10-21 — Feat: demo routes audio tool results to the external source
+
+### Changed
+
+- **Cue asset layout** — filler, start-filler and backchannel banks now live
+  under `<base>/<lang>/<voice_profile>/` instead of `<base>/<lang>/`, so
+  multiple TTS voice configurations can coexist on disk without overwriting one
+  another.
+- **[`src/config.rs`](src/config.rs)** — added a resolved cue-asset voice
+  profile (`Config::cue_voice_profile()`), with optional explicit override via
+  `ASSET_VOICE_PROFILE`. When unset, the engine derives the profile from the
+  active voice setup (prefer `TTS_VOICE` when it is not `default`, otherwise
+  fall back to the reference-audio file stem, then to `default`).
+- **[`src/assets.rs`](src/assets.rs)** / **[`src/main.rs`](src/main.rs)** /
+  **[`src/orchestrator/turn_manager.rs`](src/orchestrator/turn_manager.rs)** /
+  **[`src/ws/session.rs`](src/ws/session.rs)** — cue-bank bootstrap, runtime
+  language switching and cue metadata now all resolve the same
+  `<lang>/<voice_profile>` directory layout.
+- **Legacy cue-bank migration** — when old single-voice banks still exist
+  directly under `<base>/<lang>/`, startup now copies their flat files
+  (`*.raw`, `texts.txt`, `.voice_signature`) into the active
+  `<base>/<lang>/<voice_profile>/` directory if the target files are missing.
+  The legacy files are kept in place.
+- **[`run/refresh_voice_assets.sh`](run/refresh_voice_assets.sh)** — the voice
+  asset refresh script now targets the current cue voice profile and refreshes
+  directories under `<base>/<lang>/<voice_profile>/`, keeping other voice
+  profiles intact.
+- **[`README.md`](README.md)** — documented the new cue-bank directory layout
+  and the optional `ASSET_VOICE_PROFILE` override.
+- **[`sdk/demo.html`](sdk/demo.html)** — a tool definition may now carry an
+  extra `audio: true` flag (ignored by the backend, which only reads
+  `name`/`description`/`parameters`/`url`/`method`/`headers`). When the model
+  calls such a tool, the demo rebuilds the call's effective URL (arguments of
+  read-style verbs are appended as query parameters, mirroring the backend's
+  argument placement), writes it into the external audio source field and
+  starts playback on the engine's dedicated channel. `get_today_podcast_mp3` is
+  flagged this way, so a podcast request is played back instead of only being
+  returned to the model. The manual Play button now shares the same
+  `playExternalSource()` helper.
+- **[`sdk/demo.html`](sdk/demo.html)** — the default session prompt and the
+  built-in tool descriptions now explicitly tell the model to call
+  `get_today_podcast_mp3` on the first relevant podcast/audio request and
+  `get_public_day_report` on the first relevant report/alerts request, to
+  reduce cases where the user has to repeat the same intent before tool calling
+  is triggered.
+- **Markdown document mode** — when the user explicitly asks for a Markdown
+  document, the demo prompt now instructs the model to keep a very short spoken
+  preface outside the block and place the full document inside
+  `<chat_markdown>...</chat_markdown>`.
+- **[`sdk/demo.html`](sdk/demo.html)** — audio tool results are queued on
+  `tool_call` and autoplay after a fixed post-`assistant_end` delay, rather
+  than waiting for the SDK player to become idle. This avoids false timeouts in
+  deployments where the engine keeps sending mixed 20 ms PCM frames
+  continuously, even during silence, while keeping the post-reply timing
+  deterministic.
+- **[`src/tools.rs`](src/tools.rs)** / **[`src/orchestrator/turn_manager.rs`](src/orchestrator/turn_manager.rs)** —
+  the engine now strips `<chat_markdown>...</chat_markdown>` blocks out of the
+  spoken stream, emits them separately as `assistant_markdown` events, and
+  keeps the document text in the assistant history for follow-up turns.
+- **[`sdk/demo.html`](sdk/demo.html)** — the SDK demo renders
+  `assistant_markdown` payloads as formatted Markdown bubbles in chat instead
+  of showing raw source text.
+- **[`sdk/audio-agent-client.js`](sdk/audio-agent-client.js)** /
+  **[`sdk/audio-agent-client.d.ts`](sdk/audio-agent-client.d.ts)** — the SDK now
+  exposes the `assistant_markdown` event type for clients that want chat-only,
+  non-TTS Markdown documents.
+- **[`sdk/audio-agent-client.js`](sdk/audio-agent-client.js)** /
+  **[`sdk/audio-agent-client.d.ts`](sdk/audio-agent-client.d.ts)** — `PcmPlayer`
+  now exposes `isIdle()` and emits a local `idle` event when its scheduled
+  audio has fully drained, giving browser clients a reliable post-playback
+  signal.
+
+---
+
+## 2026-10-21 — Chore: voice-asset refresh helper script
+
+### Added
+
+- **[`run/refresh_voice_assets.sh`](run/refresh_voice_assets.sh)** — bash helper
+  that reads the current `.env`, analyzes the active TTS voice / reference /
+  sampling / cue-text settings, computes a voice signature, inspects every cue
+  bank (`fillers`, `start_filler`, `backchannel`) across the discovered
+  languages, and decides which generated assets are stale.
+- The script stores a per-directory `.voice_signature`, supports `--dry-run`,
+  `--force`, `--lang <code>`, `--base-lang-only` and `--no-restart`, deletes the
+  generated `*.raw` clips (plus `texts.txt` for text-driven banks) when a
+  refresh is needed, restarts `pm2-piperust`, and writes the updated signature
+  markers back to the asset directories.
+
+---
+
+## 2026-10-21 — Chore: SDK demo installs a default day-report tool
+
+### Changed
+
+- **[`sdk/demo.html`](sdk/demo.html)** — the demo now installs a default
+  runtime tool set through `POST /api/tools` right after connect:
+  `get_public_day_report`, mapped to
+  `GET https://aquachile.azurianlab.com/api/v1/day-report` with query
+  parameters `limit` and `date`, plus `get_today_podcast_mp3`, mapped to
+  `GET https://aquachile.azurianlab.com/api/v1/podcast/today/mp3` with query
+  parameter `download`. This lets the demo exercise real tool calling without
+  relying on an external `LLM_TOOLS_FILE`.
+
+---
+
+## 2026-10-21 — Feat: phone-login proxy (`POST /api/auth/phone-login`)
+
+### Added
+
+- **`POST /api/auth/phone-login`** ([`src/auth_api.rs`](src/auth_api.rs)) — a
+  server-side proxy that forwards `{ "phoneNumber": "…" }` to the AquaChile auth
+  API (`AUTH_API_BASE` + `AUTH_PHONE_LOGIN_PATH`) and returns its status and body
+  unchanged (on success: `sessionToken` + `user`). It exists because that API
+  rejects browser calls from the client origin (CORS preflight `400 Disallowed
+  CORS origin`, no `Access-Control-Allow-Origin`), so a direct `fetch` from the
+  demo is blocked; going through the engine makes the call same-origin. When the
+  request also carries a live `session_id`, the engine stores a sanitized user
+  profile (`name`, `email`, `businessRoles`, `permissionSet`, `centers`,
+  `permissions`) as hidden context for that session's LLM. Errors: `400` for an
+  empty/invalid body or missing `phoneNumber`, `404` for an unknown
+  `session_id`, `502` when the upstream is unreachable.
+- Config `AUTH_API_BASE` (default `https://aquachile.azurianlab.com`) and
+  `AUTH_PHONE_LOGIN_PATH` (default `/api/v1/auth/phone-login`).
+- Swagger UI (`/docs`) documents the endpoint (new `auth` tag,
+  `PhoneLoginRequest` schema).
+
+### Changed
+
+- **[JavaScript SDK](sdk/audio-agent-client.js)** — added
+  `phoneLogin(phoneNumber, sessionId?)` to `AudioAgentRest`; types updated in
+  [`sdk/audio-agent-client.d.ts`](sdk/audio-agent-client.d.ts) and the reference
+  in [`sdk/README.md`](sdk/README.md).
+- **[`sdk/demo.html`](sdk/demo.html)** — at session start it reads `phoneNumber`
+  from the page URL (`?phoneNumber=+44…`) and authenticates through the engine
+  proxy, showing the resolved user in the header, a chat chip and the event log.
+  (Replaces the earlier direct cross-origin `fetch`, which the browser blocked.)
+- **Hidden LLM user context** — the session user block now instructs the model
+  to treat `permissions`, `permissionSet`, `businessRoles` and `centers` as
+  access/domain scope, avoid fabricating unavailable access, constrain KPI /
+  report / alert interpretation to the verified user scope, and suggest next
+  steps aligned with the user's role.
+
+---
+
+## 2026-10-21 — Feat: per-session system prompt (`GET` / `POST /api/prompt`)
+
+### Added
+
+- **Server-assigned session id** — every WebSocket session is given an id
+  (`s-<epoch_ms>-<seq>`, the same stem used for its audit log) by a new
+  per-session prompt registry ([`src/prompt.rs`](src/prompt.rs)) and the id is
+  pushed to the client right after connect, before `cues`, as
+  `{"type":"session","id":"…"}` ([`src/ws/session.rs`](src/ws/session.rs)).
+  `SessionAudit::new` now receives that id instead of generating its own, so the
+  audit file and the live session share one identifier.
+- **`GET` / `POST /api/prompt`** ([`src/prompt_api.rs`](src/prompt_api.rs)) — read
+  or override the LLM system prompt at runtime, **per session** (with a
+  `session_id`) or process-wide (without one). `GET` returns the effective prompt
+  plus metadata (`source`: `session` / `global` / `env` / `default` / `none`,
+  `has_override`, `direct_answers_applied`); `POST` installs an override
+  (`prompt: null` / `""` clears it). Overrides live **in memory only** — a
+  restart restores the configured default and a session override is dropped on
+  disconnect; an unknown session id returns `404`.
+- The LLM turn now resolves the system prompt through the registry
+  ([`src/orchestrator/turn_manager.rs`](src/orchestrator/turn_manager.rs)):
+  session override → process-wide override → `LLM_SYSTEM_PROMPT` → built-in
+  default. With `/api/prompt` never called the behaviour is unchanged.
+- `Config::llm_system_prompt_from_env` records whether the configured prompt came
+  from `LLM_SYSTEM_PROMPT` (reported as `source: "env"`).
+- Swagger UI (`/docs`) documents the new endpoint (new `prompt` tag, schemas
+  `PromptResponse` / `SetPromptRequest`).
+
+### Changed
+
+- **[JavaScript SDK](sdk/audio-agent-client.js)** — added the `session` event to
+  `CLIENT_EVENTS` and `getPrompt(sessionId?)` / `setPrompt(prompt, sessionId?)`
+  to `AudioAgentRest`; types updated in
+  [`sdk/audio-agent-client.d.ts`](sdk/audio-agent-client.d.ts) and the reference
+  in [`sdk/README.md`](sdk/README.md).
+- **[`sdk/demo.html`](sdk/demo.html)** — captures the session id from the
+  `session` event and adds a **System prompt** panel to reload / override / clear
+  the prompt of the current session over REST. It also applies a built-in
+  `SESSION_PROMPT` (the Vocero / AquaChile assistant instructions) automatically
+  at the start of every session, right after the `session` event.
+
+---
+
+## 2026-10-21 — Chore: English-only code and documentation convention
+
+### Changed
+
+- **All code is now English-only.** Every comment, doc-comment, error/log string
+  and test description across `src/` and `sdk/` was reviewed and the remaining
+  Italian was translated — e.g. [`src/stt/tokenizer.rs`](src/stt/tokenizer.rs).
+  Per-language *content* is intentionally left as-is: the voice-cloning reading
+  phrases in [`src/customvoice.rs`](src/customvoice.rs) and the clip phrase
+  tables in [`src/lang.rs`](src/lang.rs) must stay in their own language.
+- **Documentation is now English-only.** [`TOOL_CALLING.md`](TOOL_CALLING.md) and
+  [`sdk/README.md`](sdk/README.md) dropped their `Italiano` / `Español`
+  sections and [`README.md`](README.md) examples switched to English sample text.
+  This supersedes the previous IT/EN/ES documentation convention.
+
+---
+
+## 2026-10-21 — Feat: JavaScript client SDK + opt-in CORS
+
+### Added
+
+- **`sdk/audio-agent-client.js`** — a dependency-free, single-file ESM client
+  SDK (browser + Node) with types in
+  [`sdk/audio-agent-client.d.ts`](sdk/audio-agent-client.d.ts) and a
+  reference in [`sdk/README.md`](sdk/README.md). It exists so the
+  client can be **decoupled** from the engine and hosted on another server:
+  - `AudioAgentClient` — the realtime `/ws` session: `sendAudio()` (accepts
+    `Int16Array` / `Float32Array` / `ArrayBuffer` / view / `number[]`, normalized
+    to 16-bit LE mono 44.1 kHz), typed controls (`sendUserText`, `interrupt`,
+    `playAudioSource`, `stopAudioSource`, `setAudioSourceVolume`,
+    `setReference`, `clearReference`, `speak`), optional auto-reconnect with
+    exponential backoff, and one event per server message (`cues`, `user`,
+    `assistant_delta`, `assistant_end`, `cue`, `barge_in`, `tool_call`,
+    `audio_source`, `error`) plus a local `audio` event with the decoded frames.
+  - `AudioAgentRest` — typed REST wrapper: `health`, `getTools` / `setTools`,
+    `getCustomVoicePhrases` / `setCustomVoice` / `clearCustomVoice`,
+    `transcribe`, `openapi`; failures throw an error carrying `status`/`body`.
+  - Browser audio helpers `MicCapture` (mic → 882-sample `Int16Array` frames via
+    an inline AudioWorklet), `PcmPlayer` (jitter-buffered playback with
+    `flush()` for barge-in) and `AudioSession` (wires client + mic + player).
+  - PCM utilities `int16ArrayFromBytes`, `float32ToInt16`, `int16ToFloat32`,
+    `toInt16` and the `AUDIO` constants.
+- **`GET /sdk/audio-agent-client.js`** ([`src/main.rs`](src/main.rs),
+  [`src/web/mod.rs`](src/web/mod.rs)) — serves the SDK (embedded at compile
+  time, like the HTML clients) so a client hosted elsewhere can fetch it.
+- **`sdk/demo.html`** + **`GET /sdk/demo.html`** — a complete demo client built
+  on the SDK (English UI, like all the code in this project): connect/disconnect,
+  live session state, streaming
+  agent replies with word-level *karaoke* highlighting, user/agent chat bubbles,
+  inline chips for `cue` / `tool_call` / `audio_source` events, a barge-in
+  counter, a microphone toggle with a live level meter, a text-turn input, an
+  external audio source (play / stop / volume) and a raw event log. It imports
+  `./audio-agent-client.js`, so it works both served by the engine and copied to
+  another origin; it warns when opened outside a secure context (no mic on
+  `file://`).
+- **Opt-in CORS** — `CORS_ALLOW_ORIGIN` (comma-separated origins, or a single
+  `*`) adds a [`tower-http`](https://docs.rs/tower-http) CORS layer
+  ([`src/main.rs`](src/main.rs), [`src/config.rs`](src/config.rs)) for browser
+  clients on another origin. Empty (the default) adds **no** layer, so the
+  engine behaves exactly as before. The WebSocket `/ws` is unaffected (a
+  handshake is not a CORS request).
+
+### Notes
+
+- No build step for the SDK: copy the file, or import it from the engine origin.
+- `tower-http` was already a transitive dependency; the `cors` feature adds no
+  new crates.
+- Docs updated: [README.md](README.md#client-sdk-javascript) (new section, route
+  table, env table, feature bullet), [`.env.example`](.env.example), and the new
+  [`sdk/README.md`](sdk/README.md) (IT/EN/ES).
+
+---
+
 ## 2026-10-21 — Feat: every HTTP verb for tools (list / get / add / update / delete / …)
 
 ### Changed