docs: consolidate English documentation under root docs
This commit is contained in:
@@ -0,0 +1,106 @@
|
||||
# Reasoning modes
|
||||
|
||||
Reasoning determines the assistant's response text. The realtime backend always
|
||||
provides the speech channel; optional HTTP reasoning can observe a transcript or
|
||||
produce a separate answer. Set `S2S_REASONING_TYPE` to select the mode.
|
||||
|
||||
| Mode | Response owner | HTTP hook |
|
||||
|---|---|---|
|
||||
| `PROXY` (default) | Realtime backend | Browser does not call `/api/reasoning`; that endpoint returns 409 |
|
||||
| `NODE_RED` | Realtime backend for `log_only=true`; external answer otherwise | Server forwards to `S2S_REASONING_NODE_RED_URL/api/reasoning` |
|
||||
| `OPENAI` | OpenAI-compatible Chat Completions response | Server calls `OPENAI_BASE_URL/chat/completions` |
|
||||
|
||||
The mode is normalized case-insensitively. `NODERED` and `NODE-RED` resolve to
|
||||
`NODE_RED`; unknown values resolve to `PROXY`.
|
||||
|
||||
## Shared instruction assembly
|
||||
|
||||
In [main.js](../../s2s-client/main.js), `effectiveReasoningInstructions()` starts
|
||||
with `pinnedPrompt || settings.instructions`. `/api/config` pins
|
||||
`S2S_PRIMARY_PROMPT` when non-empty; otherwise the settings prompt or
|
||||
`DEFAULT_INSTRUCTIONS` is used. The current server does not fetch a remote agent
|
||||
profile to choose this base prompt.
|
||||
|
||||
The browser adds caller context when `injectProfile` is enabled, custom RAG text
|
||||
when the operator has applied it, and an available ApiPilot catalog block.
|
||||
`effectiveInstructions()` adds `OPENAI_RENDER_HINT` only in `OPENAI` mode.
|
||||
The WebSocket client packages these instructions in `session.update.session`;
|
||||
the local proxy may then add server-resolved context. Details belong in
|
||||
[context injection](context-injection.md).
|
||||
|
||||
## Native realtime reasoning (`PROXY`)
|
||||
|
||||
The realtime backend receives session instructions and enabled native tools,
|
||||
then processes each voice turn through STT, model generation and TTS.
|
||||
Native function-call events are executed by the browser. ApiPilot also has a
|
||||
separate marker-based path handled by the local proxy. These are distinct
|
||||
protocols, described in [tool calling](tool-calling.md).
|
||||
|
||||
Using a direct browser URL bypasses local WebSocket proxy processing. Browser
|
||||
instructions still apply, but server-side session context and marker execution
|
||||
require the local `/v1/realtime` route.
|
||||
|
||||
## Node-RED reasoning
|
||||
|
||||
Configure a Node-RED base URL reachable by the UI server:
|
||||
|
||||
```env
|
||||
S2S_REASONING_TYPE=NODE_RED
|
||||
S2S_REASONING_NODE_RED_URL=http://localhost:1881
|
||||
```
|
||||
|
||||
`maybeRunOpenAiReasoning()` handles both external modes despite its historical
|
||||
function name. It runs after a complete user transcript and posts a body with
|
||||
`transcript`, `instructions`, `language`, `user_context`, `phoneNumber`, and
|
||||
`episode_id`. The server builds context and forwards the reasoning fields to
|
||||
Node-RED with a 30-second timeout.
|
||||
|
||||
The checked-in Node-RED flow exposes `POST /api/reasoning` and
|
||||
`POST /api/log/system-response`. Its reasoning response uses `log_only=true` and
|
||||
an empty answer. The browser returns early in that case, preserving the native
|
||||
backend's response. Assistant-text logging is forwarded separately by the relay
|
||||
and is not an audio transport.
|
||||
|
||||
An answer-producing flow must return a non-empty `answer` and omit `log_only` or
|
||||
set it to `false`. That activates the external-answer behavior below.
|
||||
|
||||
## OpenAI-compatible HTTP reasoning
|
||||
|
||||
```env
|
||||
S2S_REASONING_TYPE=OPENAI
|
||||
OPENAI_API_KEY=YOUR_API_KEY
|
||||
OPENAI_BASE_URL=https://api.openai.com/v1
|
||||
OPENAI_REASONING_MODEL=gpt-4.1-mini
|
||||
```
|
||||
|
||||
`OPENAI_ORGANIZATION` and `OPENAI_PROJECT` are optional headers. The model value
|
||||
is a runtime default, not a recommendation about provider availability.
|
||||
|
||||
The server combines instructions and context into model messages, requests a
|
||||
final answer, and returns `answer` with available usage metadata. ApiPilot can
|
||||
append discovery instructions, execute a detected call marker, and request a
|
||||
second response based on the tool result. The endpoint does not forward the
|
||||
browser's native `session.tools` catalog as OpenAI Chat Completions `tools`.
|
||||
|
||||
For a non-log-only answer, the browser cancels/suppresses the active native
|
||||
response and appends the external assistant text. It uses browser
|
||||
`SpeechSynthesis` when available. Its compatibility fallback sends
|
||||
`[VOCERO_RENDER_REPLY]` plus the answer to the realtime backend and requests a
|
||||
response. The render hint asks the model to read only the answer after that
|
||||
marker. This instruction is model behavior, not a guarantee of exact playback.
|
||||
|
||||
External reasoning adds an HTTP operation to each completed transcript and has
|
||||
separate provider token usage. It is not the speech pipeline's `responses-api`
|
||||
or `chat-completions` LLM backend setting; those configure the model inside the
|
||||
realtime backend instead.
|
||||
|
||||
## Where to change behavior
|
||||
|
||||
| Goal | Source |
|
||||
|---|---|
|
||||
| Change the base agent prompt | `S2S_PRIMARY_PROMPT` or browser settings when unpinned |
|
||||
| Change browser context assembly | `effectiveReasoningInstructions()` in `main.js` |
|
||||
| Change HTTP provider messages | `reasoning()` in `server.py` |
|
||||
| Change external-answer rendering | `maybeRunOpenAiReasoning()` in `main.js` |
|
||||
| Change Node-RED processing | `s2s-reasoning-engine/flows.json` |
|
||||
| Change native model/tool generation | `speech_to_speech/LLM` and realtime handlers |
|
||||
Reference in New Issue
Block a user