# Reasoning modes

Reasoning determines the assistant's response text. The realtime backend always
provides the speech channel; optional HTTP reasoning can observe a transcript or
produce a separate answer. Set `S2S_REASONING_TYPE` to select the mode.

| Mode | Response owner | HTTP hook |
|---|---|---|
| `PROXY` (default) | Realtime backend | Browser does not call `/api/reasoning`; that endpoint returns 409 |
| `NODE_RED` | Realtime backend for `log_only=true`; external answer otherwise | Server forwards to `S2S_REASONING_NODE_RED_URL/api/reasoning` |
| `OPENAI` | OpenAI-compatible Chat Completions response | Server calls `OPENAI_BASE_URL/chat/completions` |

The mode is normalized case-insensitively. `NODERED` and `NODE-RED` resolve to
`NODE_RED`; unknown values resolve to `PROXY`.

## Shared instruction assembly

In [main.js](../../s2s-client/main.js), `effectiveReasoningInstructions()` starts
with `pinnedPrompt || settings.instructions`. `/api/config` pins
`S2S_PRIMARY_PROMPT` when non-empty; otherwise the settings prompt or
`DEFAULT_INSTRUCTIONS` is used. The current server does not fetch a remote agent
profile to choose this base prompt.

The browser adds caller context when `injectProfile` is enabled, custom RAG text
when the operator has applied it, and an available ApiPilot catalog block.
`effectiveInstructions()` adds `OPENAI_RENDER_HINT` only in `OPENAI` mode.
The WebSocket client packages these instructions in `session.update.session`;
the local proxy may then add server-resolved context. Details belong in
[context injection](context-injection.md).

## Native realtime reasoning (`PROXY`)

The realtime backend receives session instructions and enabled native tools,
then processes each voice turn through STT, model generation and TTS.
Native function-call events are executed by the browser. ApiPilot also has a
separate marker-based path handled by the local proxy. These are distinct
protocols, described in [tool calling](tool-calling.md).

Using a direct browser URL bypasses local WebSocket proxy processing. Browser
instructions still apply, but server-side session context and marker execution
require the local `/v1/realtime` route.

## Node-RED reasoning

Configure a Node-RED base URL reachable by the UI server:

```env
S2S_REASONING_TYPE=NODE_RED
S2S_REASONING_NODE_RED_URL=http://localhost:1881
```

`maybeRunOpenAiReasoning()` handles both external modes despite its historical
function name. It runs after a complete user transcript and posts a body with
`transcript`, `instructions`, `language`, `user_context`, `phoneNumber`, and
`episode_id`. The server builds context and forwards the reasoning fields to
Node-RED with a 30-second timeout.

The checked-in Node-RED flow exposes `POST /api/reasoning` and
`POST /api/log/system-response`. Its reasoning response uses `log_only=true` and
an empty answer. The browser returns early in that case, preserving the native
backend's response. Assistant-text logging is forwarded separately by the relay
and is not an audio transport.

An answer-producing flow must return a non-empty `answer` and omit `log_only` or
set it to `false`. That activates the external-answer behavior below.

## OpenAI-compatible HTTP reasoning

```env
S2S_REASONING_TYPE=OPENAI
OPENAI_API_KEY=YOUR_API_KEY
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_REASONING_MODEL=gpt-4.1-mini
```

`OPENAI_ORGANIZATION` and `OPENAI_PROJECT` are optional headers. The model value
is a runtime default, not a recommendation about provider availability.

The server combines instructions and context into model messages, requests a
final answer, and returns `answer` with available usage metadata. ApiPilot can
append discovery instructions, execute a detected call marker, and request a
second response based on the tool result. The endpoint does not forward the
browser's native `session.tools` catalog as OpenAI Chat Completions `tools`.

For a non-log-only answer, the browser cancels/suppresses the active native
response and appends the external assistant text. It uses browser
`SpeechSynthesis` when available. Its compatibility fallback sends
`[VOCERO_RENDER_REPLY]` plus the answer to the realtime backend and requests a
response. The render hint asks the model to read only the answer after that
marker. This instruction is model behavior, not a guarantee of exact playback.

External reasoning adds an HTTP operation to each completed transcript and has
separate provider token usage. It is not the speech pipeline's `responses-api`
or `chat-completions` LLM backend setting; those configure the model inside the
realtime backend instead.

## Where to change behavior

| Goal | Source |
|---|---|
| Change the base agent prompt | `S2S_PRIMARY_PROMPT` or browser settings when unpinned |
| Change browser context assembly | `effectiveReasoningInstructions()` in `main.js` |
| Change HTTP provider messages | `reasoning()` in `server.py` |
| Change external-answer rendering | `maybeRunOpenAiReasoning()` in `main.js` |
| Change Node-RED processing | `s2s-reasoning-engine/flows.json` |
| Change native model/tool generation | `speech_to_speech/LLM` and realtime handlers |