docs: consolidate English documentation under root docs

2026-10-05 10:11:35 -03:00
parent b91762a927
commit 6b8c27fd76
38 changed files with 2261 additions and 5594 deletions
@@ -0,0 +1,137 @@
+# Tool calling
+
+The application has two tool mechanisms: native realtime function calling and
+ApiPilot calls detected from model text. A spoken promise to fetch something is
+not evidence that either mechanism executed a tool.
+
+## Native realtime function calls
+
+The browser sends tool declarations inside the `session` object:
+
+```json
+{
+  "type": "session.update",
+  "session": {
+    "type": "realtime",
+    "tools": [
+      {
+        "type": "function",
+        "name": "web_search",
+        "description": "Search the web for information.",
+        "parameters": {
+          "type": "object",
+          "properties": {"query": {"type": "string"}},
+          "required": ["query"]
+        }
+      }
+    ],
+    "tool_choice": "auto"
+  }
+}
+```
+
+Keep configuration inside `session`, not at the event root. The backend has a
+legacy flat-payload compatibility path, but new clients should use the nested
+shape. The current browser's `activeToolDefs()` declares `web_search` when
+enabled and a deploy/user search key is available. It does not load the old
+remote agent-profile MCP catalog into native session tools.
+
+For a native call, the backend emits:
+
+```json
+{
+  "type": "response.function_call_arguments.done",
+  "call_id": "call_example",
+  "name": "web_search",
+  "arguments": "{\"query\":\"example query\"}"
+}
+```
+
+`arguments` is a JSON-encoded string. The
+[WebSocket client](../../s2s-client/ws/s2s-ws-client.js) parses it and dispatches
+`toolcall`; [main.js](../../s2s-client/main.js) runs the supported executor.
+Web search uses `POST /api/search`, keeping the deployment's Serper key on the
+server. Tool output returns to the model as:
+
+```json
+{
+  "type": "conversation.item.create",
+  "item": {
+    "type": "function_call_output",
+    "call_id": "call_example",
+    "output": "Tool result text"
+  }
+}
+```
+
+The client requests follow-up generation with `response.create` when the
+result should be spoken. A `function_call_output` item alone does not start
+generation. See the [realtime API guide](../speech/realtime-api.md) for local
+model `<code>` extraction and native remote-provider tool calls.
+
+## ApiPilot discovery and execution
+
+`S2S_APIPILOT_TOOLS_URL` enables the current dynamic-tool bridge. The backend
+discovers tools through its `/tools` route and executes through `/execute`.
+`S2S_APIPILOT_TOOLS_EXECUTE_URL` can override the execution target. The same-origin UI
+server exposes:
+
+- `GET /api/apipilot/tools?include_unauthorized=true`: discovery envelope with
+  `data.tools`, `data.count`, and `data.include_unauthorized`.
+- `POST /api/apipilot/execute`: body `{"name":"tool_name","arguments":{}}` and
+  a `success/data/error` envelope.
+
+Tool definitions contain `name`, `description`, JSON Schema `parameters`, and
+`requires_auth`. Execution looks up the current discovery cache, refreshes once
+if the tool is missing, and applies an authorization gate and argument
+validation before forwarding. Caller `Authorization` and
+`x-demo-session` can be forwarded; configured service credentials and
+login-by-phone provide additional server paths. Display `error.message` to the
+user; keep `error.code`, `error.status_code`, and `error.raw` for diagnostics.
+
+The discovery cache defaults to 300 seconds server-side
+(`S2S_APIPILOT_TTL_SEC`) and can be refreshed with `refresh=true`. The browser
+also keeps a per-user catalog cache. ApiPilot configuration is independent of
+`S2S_INJECT_PROFILE`.
+
+## Marker-based two-pass reasoning
+
+The browser and proxy add an ApiPilot catalog as **instruction text**, including
+tool names, descriptions, authentication requirements and required parameters.
+The model is instructed to emit a call marker when it chooses a matching tool:
+
+```text
+<<<APIPILOT_CALL:NAME=tool_name,ARGUMENTS={"parameter":"value"}>>>
+```
+
+Tool names are exact identifiers and must not be translated. The local realtime
+proxy accumulates assistant text, detects the marker, validates and executes
+the ApiPilot call, then injects the result as context and requests a new
+response. The HTTP reasoning route has analogous discovery/execution and
+second-answer handling. Internal marker text and its speculative audio are
+filtered from user output where those handlers run.
+
+This marker is not `response.function_call_arguments.done`, and ApiPilot prompt
+discovery is not the same as declaring a native `session.tools` array. Direct
+browser connections bypass the local relay's marker executor even if the
+browser includes catalog instructions.
+
+## Retired documentation assumptions
+
+The former `HOWITWORKS.md` described `/api/mcp/tool`, remote
+`S2S_AGENT_PROFILE` loading, and Spanish/Italian-to-English client ranking.
+Those APIs/rankers are absent from the current client/server path. The retained
+agent-profile JSON is an illustrative historical/example shape. Add or modify
+current ApiPilot tool definitions in the provider; do not restore obsolete
+endpoints solely to match older documentation.
+
+## Verify a tool turn
+
+1. Confirm `/api/config` reports the intended feature and connection mode.
+2. Inspect `session.update`: native `tools` and ApiPilot instruction text are
+   separate evidence.
+3. For native calls, find `response.function_call_arguments.done`, then the
+   browser `toolcall` and executor request.
+4. For ApiPilot, find the discovery, call-marker detection and execution logs.
+5. Confirm the tool result reaches the model and follow-up text/audio reaches
+   the browser. A spoken acknowledgement alone does not complete the flow.