valenti/vocero-client
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore(vocero): remove the legacy ApiPilot integration (#3)

Browse Source 
Vocero now retrieves AquaPortal content through authenticated MCP. This removes the obsolete ApiPilot integration from the API and browser while preserving the separately deployed speech pipeline.

Release notes:

- Removed ApiPilot endpoints, configuration, credential overrides, and HTTP/realtime tool execution.
- Removed browser ApiPilot catalog loading and prompt generation while retaining authenticated MCP discovery.
- Updated regression coverage and operating documentation for the retired routes and settings.

Reviewed-on: AquaPortal/s2s#3
This commit is contained in:
alvaro
2026-10-08 11:23:41 -04:00
parent 0311d3f1ff
commit dfed03ac7c
39 changed files with 198 additions and 3314 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/client/tool-calling.md
+17 -54
View File
@@ -2,7 +2,7 @@
The application supports native realtime function calling and server-side
tool selection from model text. AquaPortal content uses authenticated MCP when
`S2S_MCP_URL` is set; otherwise the legacy ApiPilot bridge remains available.
`S2S_MCP_URL` is set. Without MCP, the native browser tools remain available.
A spoken promise to fetch something is not evidence that a tool executed.
## Authenticated AquaPortal MCP
@@ -172,69 +172,32 @@ result should be spoken. A `function_call_output` item alone does not start
generation. See the [realtime API guide](../speech/realtime-api.md) for local
model `<code>` extraction and native remote-provider tool calls.
## ApiPilot discovery and execution
When MCP is disabled, `S2S_APIPILOT_TOOLS_URL` enables the legacy dynamic-tool bridge. The backend
discovers tools through its `/tools` route and executes through `/execute`.
`S2S_APIPILOT_TOOLS_EXECUTE_URL` can override the execution target. The same-origin UI
server exposes:
- `GET /api/apipilot/tools?include_unauthorized=true`: discovery envelope with
`data.tools`, `data.count`, and `data.include_unauthorized`.
- `POST /api/apipilot/execute`: body `{"name":"tool_name","arguments":{}}` and
a `success/data/error` envelope.
Tool definitions contain `name`, `description`, JSON Schema `parameters`, and
`requires_auth`. Execution looks up the current discovery cache, refreshes once
if the tool is missing, and applies an authorization gate and argument
validation before forwarding. Caller `Authorization` and
`x-demo-session` can be forwarded; configured service credentials and
login-by-phone provide additional server paths. Display `error.message` to the
user; keep `error.code`, `error.status_code`, and `error.raw` for diagnostics.
The discovery cache defaults to 300 seconds server-side
(`S2S_APIPILOT_TTL_SEC`) and can be refreshed with `refresh=true`. The browser
also keeps a per-user catalog cache. ApiPilot configuration is independent of
`S2S_INJECT_PROFILE`.
## Marker-based two-pass reasoning
The browser and proxy add an ApiPilot catalog as **instruction text**, including
tool names, descriptions, authentication requirements and required parameters.
The model is instructed to emit a call marker when it chooses a matching tool:
```text
<<<APIPILOT_CALL:NAME=tool_name,ARGUMENTS={"parameter":"value"}>>>
```
Tool names are exact identifiers and must not be translated. The local realtime
proxy accumulates assistant text, detects the marker, validates and executes
the ApiPilot call, then injects the result as context and requests a new
response. The HTTP reasoning route has analogous discovery/execution and
second-answer handling. Internal marker text and its speculative audio are
filtered from user output where those handlers run.
This marker is not `response.function_call_arguments.done`, and ApiPilot prompt
discovery is not the same as declaring a native `session.tools` array. Direct
browser connections bypass the local relay's marker executor even if the
browser includes catalog instructions.
## Retired documentation assumptions
ApiPilot discovery/execution endpoints, catalog injection, credential overrides,
and two-pass marker handling have been removed from Vocero. Its former HTTP
routes return 404 regardless of MCP configuration. The separate speech service
still has its existing legacy marker sanitizers; no ApiPilot requests can be
made through Vocero. See [usage](usage.md#authenticated-aquaportal-mcp) for
deployment configuration cleanup.
The former `HOWITWORKS.md` described `/api/mcp/tool`, remote
`S2S_AGENT_PROFILE` loading, and Spanish/Italian-to-English client ranking.
Those APIs/rankers are absent from the current client/server path. The retained
agent-profile JSON is an illustrative historical/example shape. Add or modify
current ApiPilot tool definitions in the provider; do not restore obsolete
endpoints solely to match older documentation.
agent-profile JSON is an illustrative historical/example shape. Content tool
definitions and permissions belong to the AquaPortal MCP backend; Vocero's
provider enforces its allowlist. Do not restore obsolete endpoints solely to
match older documentation.
## Verify a tool turn
1. Confirm `/api/config` reports the intended feature and connection mode.
2. Inspect `session.update`: native `tools` and ApiPilot instruction text are
separate evidence.
2. For MCP, confirm conversation creation and authenticated `/api/mcp/tools`
discovery succeed. Inspect the relay's `session.update` for MCP instructions;
these are separate from native `session.tools` declarations.
3. For native calls, find `response.function_call_arguments.done`, then the
browser `toolcall` and executor request.
4. For ApiPilot, find the discovery, call-marker detection and execution logs.
4. For MCP, find `mcp_tool_completed` for the expected user/tool and any
`vocero.tool.started` / `vocero.tool.failed` voice events.
5. Confirm the tool result reaches the model and follow-up text/audio reaches
the browser. A spoken acknowledgement alone does not complete the flow.