# Reasoning modes

Reasoning determines the assistant's response text. The realtime backend always
provides the speech channel; optional HTTP reasoning can observe a transcript or
produce a separate answer. Set `S2S_REASONING_TYPE` to select the mode.

| Mode | Response owner | HTTP hook |
|---|---|---|
| `PROXY` (default) | Realtime backend | Browser does not call `/api/reasoning`; that endpoint returns 409 |
| `NODE_RED` | Realtime backend for `log_only=true`; external answer otherwise | Server forwards to `S2S_REASONING_NODE_RED_URL/api/reasoning` |
| `OPENAI` | OpenAI-compatible Chat Completions response | Server calls `OPENAI_BASE_URL/chat/completions` |

The mode is normalized case-insensitively. `NODERED` and `NODE-RED` resolve to
`NODE_RED`; unknown values resolve to `PROXY`.

## Shared instruction assembly

In [context.ts](../../s2s-frontend/src/lib/context.ts), `buildInstructions()` starts
with `config.primaryPrompt || settings.instructions`. `/api/config` pins
`S2S_PRIMARY_PROMPT` when non-empty; otherwise the settings prompt or
`DEFAULT_INSTRUCTIONS` is used. The current server does not fetch a remote agent
profile to choose this base prompt.

The browser adds caller context when `injectProfile` is enabled, custom RAG text
when the operator has applied it, and an available ApiPilot catalog block.
`speechInstructions()` adds the speech-rendering hint only in `OPENAI` mode.
The WebSocket client packages these instructions in `session.update.session`;
the local proxy may then add server-resolved context. Details belong in
[context injection](context-injection.md).

## Native realtime reasoning (`PROXY`)

The realtime backend receives session instructions and enabled native tools,
then processes each voice turn through STT, model generation and TTS.
Native function-call events are executed by the browser. ApiPilot also has a
separate marker-based path handled by the local proxy. These are distinct
protocols, described in [tool calling](tool-calling.md).

Using a direct browser URL bypasses local WebSocket proxy processing. Browser
instructions still apply, but server-side session context and marker execution
require the local `/v1/realtime` route.

## Node-RED reasoning

Configure a Node-RED base URL reachable by the UI server:

```env
S2S_REASONING_TYPE=NODE_RED
S2S_REASONING_NODE_RED_URL=http://localhost:1881
```

`VoiceController.reason()` in [controller.ts](../../s2s-frontend/src/realtime/controller.ts)
handles both external modes. It runs after a complete user transcript and posts a body with
`transcript`, `instructions`, `language`, `user_context`, `phoneNumber`, and
`episode_id`. The server builds context and forwards the reasoning fields to
Node-RED with a 30-second timeout.

The checked-in Node-RED flow exposes `POST /api/reasoning` and
`POST /api/log/system-response`. Its reasoning response uses `log_only=true` and
an empty answer. The browser returns early in that case, preserving the native
backend's response. Assistant-text logging is forwarded separately by the relay
and is not an audio transport.

An answer-producing flow must return a non-empty `answer` and omit `log_only` or
set it to `false`. That activates the external-answer behavior below.

## OpenAI-compatible HTTP reasoning

```env
S2S_REASONING_TYPE=OPENAI
OPENAI_API_KEY=YOUR_API_KEY
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_REASONING_MODEL=gpt-4.1-mini
```

`OPENAI_ORGANIZATION` and `OPENAI_PROJECT` are optional headers. The model value
is a runtime default, not a recommendation about provider availability.

The server combines instructions and context into model messages, requests a
final answer, and returns `answer` with available usage metadata. ApiPilot can
append discovery instructions, execute a detected call marker, and request a
second response based on the tool result. The endpoint does not forward the
browser's native `session.tools` catalog as OpenAI Chat Completions `tools`.

For a non-log-only answer, the browser cancels/suppresses the active native
response and appends the external assistant text. It uses browser
`SpeechSynthesis` when available. The React client adds the answer to the realtime
conversation as an assistant item; without browser speech synthesis it remains
visible as text.

External reasoning adds an HTTP operation to each completed transcript and has
separate provider token usage. It is not the speech pipeline's `responses-api`
or `chat-completions` LLM backend setting; those configure the model inside the
realtime backend instead.

## Where to change behavior

| Goal | Source |
|---|---|
| Change the base agent prompt | `S2S_PRIMARY_PROMPT` or browser settings when unpinned |
| Change browser context assembly | `buildInstructions()` in `s2s-frontend/src/lib/context.ts` |
| Change HTTP provider messages | `reasoning()` in `s2s-client/s2s_client/services/reasoning.py` |
| Change external-answer rendering | `VoiceController.reason()` and `RealtimeClient.renderExternalAnswer()` |
| Change Node-RED processing | `s2s-reasoning-engine/flows.json` |
| Change native model/tool generation | `speech_to_speech/LLM` and realtime handlers |