feat(mcp): connect authenticated AquaPortal tools to Vocero (#1)

Vocero needs to query AquaPortal on behalf of the caller selected by the existing phone link. This adds server-owned conversations and MCP tool execution for voice and HTTP reasoning while leaving the speech pipeline unchanged.

Release notes:

- Added browser-bound conversations that keep backend credentials server-side and isolate callers.
- Connected the eight-tool AquaPortal catalog through authenticated Streamable HTTP with full input schemas and structured results.
- Added bounded multi-step tool execution with cancellation, expiry, and stale-result protection.
- Updated browser bootstrap, tool progress, reasoning headers, and conversation cleanup.
- Disabled legacy unscoped content preloads in MCP mode and documented configuration and deployment constraints.
- Preserved disabled-mode contracts and added local SDK, WebSocket, and browser regression coverage.

Reviewed-on: AquaPortal/s2s#1
This commit is contained in:
2026-10-08 08:49:39 -04:00
parent ccb1f708ca
commit 96895996d9
37 changed files with 2498 additions and 55 deletions
+70 -4
View File
@@ -1,8 +1,74 @@
# Tool calling
The application has two tool mechanisms: native realtime function calling and
ApiPilot calls detected from model text. A spoken promise to fetch something is
not evidence that either mechanism executed a tool.
The application supports native realtime function calling and server-side
tool selection from model text. AquaPortal content uses authenticated MCP when
`S2S_MCP_URL` is set; otherwise the legacy ApiPilot bridge remains available.
A spoken promise to fetch something is not evidence that a tool executed.
## Authenticated AquaPortal MCP
Vocero uses the official MCP Python SDK with Streamable HTTP. Each discovery
or execution initializes a client with that conversation's opaque backend token
as `Authorization: Bearer ...`. JSON and SSE responses are supported. Neither
browser values nor model arguments choose the authenticated user. There is no
service-token override, anonymous retry, redirect following, or public REST
fallback on this path. AquaPortal checks current permissions on every request,
including its administrator and assigned report-manager access rules.
The backend phone login must return `sessionToken` plus `user.id` and `user.name`.
The only allowed tool names are:
| Tool | Content |
|---|---|
| `list_reports` | Accessible published reports with name, center, and date filters |
| `get_report` | One report or an explicit list of ambiguous selections |
| `list_insights` | Published findings, priorities, recommendations, and sources |
| `get_metric_history` | Dated metric values, units, centers, cages, and source reports |
| `list_podcasts` | Daily and report-associated episodes |
| `get_podcast` | Selected episode summary, highlights, and available media |
| `read_podcast_transcript` | Paginated transcript or report-script passages |
| `get_my_briefing` | Compact overview with actual source dates |
Discovery preserves the complete input schema, including nested `request`,
optional filters, references, and pagination. The executor validates arguments
against that schema and supplies `structuredContent` to reasoning when present,
falling back to MCP text content. It preserves `get_report`'s `result` union and
list `next_offset` values. Tool errors are data for the model to explain; expired
authentication stops the conversation. Tool text is labeled as reference data,
never as instructions or authorization.
Both reasoning paths can discover reports, retrieve details, and follow up with
another tool before answering. The HTTP loop retains bounded completed-turn
history. The realtime relay injects results into the speech service's existing
conversation. Attempts are bounded per user turn (six by default), identical
calls within a turn are rejected, and tool results are limited to 64,000
characters individually and 96,000 per turn. Each MCP operation has a timeout;
HTTP reasoning also has a 180-second total timeout.
Selection uses an internal text marker, not a new speech protocol event:
```text
<<<VOCERO_MCP_CALL:NAME=get_report,ARGUMENTS={"request":{"report_id":"example-id"}}>>>
```
The distinct marker is necessary because the current speech pipeline removes
`APIPILOT_CALL` before emitting assistant text. The relay buffers undecided
text/audio, handles transcripts emitted before `response.created`, executes
the MCP selection, and requests another response. Normal speech continues using
the existing PCM16 protocol. This leaves `speech_to_speech/` unchanged, although
the speech service may spend time synthesizing audio that the relay suppresses.
Malformed selections fail with a user-facing error rather than permissive parsing.
New speech, cancellation, disconnect, and expiry invalidate pending work.
Generation checks reject late results even if the transport does not stop in
time. The browser receives `vocero.tool.started` / `vocero.tool.failed` status
events, without backend credentials or raw tool arguments.
Automatic current-episode context from `episode_id` is not wired yet. The model
must resolve a podcast with the catalog rather than fetching an unauthenticated
episode URL. See [usage](usage.md#authenticated-aquaportal-mcp) for configuration,
conversation endpoints, single-worker constraints, and the retained unsafe
phone selector.
## Native realtime function calls
@@ -72,7 +138,7 @@ model `<code>` extraction and native remote-provider tool calls.
## ApiPilot discovery and execution
`S2S_APIPILOT_TOOLS_URL` enables the current dynamic-tool bridge. The backend
When MCP is disabled, `S2S_APIPILOT_TOOLS_URL` enables the legacy dynamic-tool bridge. The backend
discovers tools through its `/tools` route and executes through `/execute`.
`S2S_APIPILOT_TOOLS_EXECUTE_URL` can override the execution target. The same-origin UI
server exposes: