feat(mcp): connect authenticated AquaPortal tools to Vocero (#1)
Vocero needs to query AquaPortal on behalf of the caller selected by the existing phone link. This adds server-owned conversations and MCP tool execution for voice and HTTP reasoning while leaving the speech pipeline unchanged. Release notes: - Added browser-bound conversations that keep backend credentials server-side and isolate callers. - Connected the eight-tool AquaPortal catalog through authenticated Streamable HTTP with full input schemas and structured results. - Added bounded multi-step tool execution with cancellation, expiry, and stale-result protection. - Updated browser bootstrap, tool progress, reasoning headers, and conversation cleanup. - Disabled legacy unscoped content preloads in MCP mode and documented configuration and deployment constraints. - Preserved disabled-mode contracts and added local SDK, WebSocket, and browser regression coverage. Reviewed-on: AquaPortal/s2s#1
This commit is contained in:
@@ -1,8 +1,74 @@
|
||||
# Tool calling
|
||||
|
||||
The application has two tool mechanisms: native realtime function calling and
|
||||
ApiPilot calls detected from model text. A spoken promise to fetch something is
|
||||
not evidence that either mechanism executed a tool.
|
||||
The application supports native realtime function calling and server-side
|
||||
tool selection from model text. AquaPortal content uses authenticated MCP when
|
||||
`S2S_MCP_URL` is set; otherwise the legacy ApiPilot bridge remains available.
|
||||
A spoken promise to fetch something is not evidence that a tool executed.
|
||||
|
||||
## Authenticated AquaPortal MCP
|
||||
|
||||
Vocero uses the official MCP Python SDK with Streamable HTTP. Each discovery
|
||||
or execution initializes a client with that conversation's opaque backend token
|
||||
as `Authorization: Bearer ...`. JSON and SSE responses are supported. Neither
|
||||
browser values nor model arguments choose the authenticated user. There is no
|
||||
service-token override, anonymous retry, redirect following, or public REST
|
||||
fallback on this path. AquaPortal checks current permissions on every request,
|
||||
including its administrator and assigned report-manager access rules.
|
||||
|
||||
The backend phone login must return `sessionToken` plus `user.id` and `user.name`.
|
||||
The only allowed tool names are:
|
||||
|
||||
| Tool | Content |
|
||||
|---|---|
|
||||
| `list_reports` | Accessible published reports with name, center, and date filters |
|
||||
| `get_report` | One report or an explicit list of ambiguous selections |
|
||||
| `list_insights` | Published findings, priorities, recommendations, and sources |
|
||||
| `get_metric_history` | Dated metric values, units, centers, cages, and source reports |
|
||||
| `list_podcasts` | Daily and report-associated episodes |
|
||||
| `get_podcast` | Selected episode summary, highlights, and available media |
|
||||
| `read_podcast_transcript` | Paginated transcript or report-script passages |
|
||||
| `get_my_briefing` | Compact overview with actual source dates |
|
||||
|
||||
Discovery preserves the complete input schema, including nested `request`,
|
||||
optional filters, references, and pagination. The executor validates arguments
|
||||
against that schema and supplies `structuredContent` to reasoning when present,
|
||||
falling back to MCP text content. It preserves `get_report`'s `result` union and
|
||||
list `next_offset` values. Tool errors are data for the model to explain; expired
|
||||
authentication stops the conversation. Tool text is labeled as reference data,
|
||||
never as instructions or authorization.
|
||||
|
||||
Both reasoning paths can discover reports, retrieve details, and follow up with
|
||||
another tool before answering. The HTTP loop retains bounded completed-turn
|
||||
history. The realtime relay injects results into the speech service's existing
|
||||
conversation. Attempts are bounded per user turn (six by default), identical
|
||||
calls within a turn are rejected, and tool results are limited to 64,000
|
||||
characters individually and 96,000 per turn. Each MCP operation has a timeout;
|
||||
HTTP reasoning also has a 180-second total timeout.
|
||||
|
||||
Selection uses an internal text marker, not a new speech protocol event:
|
||||
|
||||
```text
|
||||
<<<VOCERO_MCP_CALL:NAME=get_report,ARGUMENTS={"request":{"report_id":"example-id"}}>>>
|
||||
```
|
||||
|
||||
The distinct marker is necessary because the current speech pipeline removes
|
||||
`APIPILOT_CALL` before emitting assistant text. The relay buffers undecided
|
||||
text/audio, handles transcripts emitted before `response.created`, executes
|
||||
the MCP selection, and requests another response. Normal speech continues using
|
||||
the existing PCM16 protocol. This leaves `speech_to_speech/` unchanged, although
|
||||
the speech service may spend time synthesizing audio that the relay suppresses.
|
||||
Malformed selections fail with a user-facing error rather than permissive parsing.
|
||||
|
||||
New speech, cancellation, disconnect, and expiry invalidate pending work.
|
||||
Generation checks reject late results even if the transport does not stop in
|
||||
time. The browser receives `vocero.tool.started` / `vocero.tool.failed` status
|
||||
events, without backend credentials or raw tool arguments.
|
||||
|
||||
Automatic current-episode context from `episode_id` is not wired yet. The model
|
||||
must resolve a podcast with the catalog rather than fetching an unauthenticated
|
||||
episode URL. See [usage](usage.md#authenticated-aquaportal-mcp) for configuration,
|
||||
conversation endpoints, single-worker constraints, and the retained unsafe
|
||||
phone selector.
|
||||
|
||||
## Native realtime function calls
|
||||
|
||||
@@ -72,7 +138,7 @@ model `<code>` extraction and native remote-provider tool calls.
|
||||
|
||||
## ApiPilot discovery and execution
|
||||
|
||||
`S2S_APIPILOT_TOOLS_URL` enables the current dynamic-tool bridge. The backend
|
||||
When MCP is disabled, `S2S_APIPILOT_TOOLS_URL` enables the legacy dynamic-tool bridge. The backend
|
||||
discovers tools through its `/tools` route and executes through `/execute`.
|
||||
`S2S_APIPILOT_TOOLS_EXECUTE_URL` can override the execution target. The same-origin UI
|
||||
server exposes:
|
||||
|
||||
Reference in New Issue
Block a user