fixes tool callong

This commit is contained in:
valenti
2026-10-21 10:44:49 +01:00
parent 79f927d968
commit 1cafe3dd3c
14 changed files with 2308 additions and 61 deletions
+144
View File
@@ -5,6 +5,150 @@ Entries are written in English and ordered newest-first.
---
## 2026-10-21 — Feat: every HTTP verb for tools (list / get / add / update / delete / …)
### Changed
- **`method` now accepts any HTTP verb** ([`src/tools.rs`](src/tools.rs)),
case-insensitive: `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, `HEAD`, `OPTIONS`,
`TRACE` and custom verbs (e.g. `PURGE`). Before this change anything other than
`GET` fell through to `POST`, so `PUT`/`PATCH`/`DELETE` tools silently became
POSTs.
- **Argument placement follows the verb.** The arguments left after the URL
placeholders go to the **query string** for the read-style verbs
(`GET`, `HEAD`, `OPTIONS`, `TRACE`, `DELETE`) and to the **JSON body** for
every other verb (`POST`, `PUT`, `PATCH`, custom); body-style verbs always send
a body (`{}` when nothing is left). This makes a CRUD surface expressible:
`GET /items?limit=10`, `GET /items/{id}`, `POST /items`,
`PATCH|PUT /items/{id}`, `DELETE /items/{id}`, `HEAD /items/{id}`.
- **New helpers** `http_method()`, `args_in_body()` and `place_args()` in
[`src/tools.rs`](src/tools.rs); `ToolRegistry::execute` now builds the request
through `Client::request(method, url)` instead of separate `get`/`post`
branches.
- An invalid verb is **warned at load** and, at execution time, returned to the
model as `Error: invalid HTTP method '<verb>' for tool '<name>'` (never a
panic, never a blocking failure).
### Notes
- `DELETE` is **query-style**: put the id in the URL (`/items/{id}`) and any
extra argument ends up as a query parameter.
- Tests: two new unit tests (`every_http_verb_is_accepted`,
`args_placement_follows_the_verb`) — `cargo test --bin audio-agent-engine --
tools::` → 9 passed.
- Docs updated: [TOOL_CALLING.md](TOOL_CALLING.md) (IT/EN/ES, per-verb argument
table + full CRUD example set) and [README.md](README.md#tool-calling).
---
## 2026-10-21 — Feat: runtime tool definitions — `GET` / `POST /api/tools`
### Added
- **New REST endpoint `POST /api/tools`** ([`src/tools_api.rs`](src/tools_api.rs)).
The body is the **same JSON array** used by `LLM_TOOLS_FILE`; it installs a new
set of tool definitions at runtime, **without a restart** (until now the file
was only read once at startup). The definitions apply from the **next turn**
on.
- **`GET /api/tools`** reports the active definitions (in the OpenAI `tools`
shape they are advertised to the model with) plus their origin: `source` is
`file`, `request` or `none`.
- **Errors:** `400` empty body / non-UTF-8 / invalid JSON / JSON array with no
tool definition, `503` LLM disabled. Responses use the shared
`{ok, …}` / `{ok:false,error}` shapes.
- **OpenAPI/Swagger UI:** both handlers are documented (tag `tools`, schema
`ToolsResponse`) in [`src/apidoc.rs`](src/apidoc.rs) and appear at `/docs`.
### Changed
- **`ToolRegistry` can be built from a string** — `ToolRegistry::from_json`
([`src/tools.rs`](src/tools.rs)) is the new core builder; `load` now reads the
file and delegates to it, so a body accepted by the endpoint is also valid as
an `LLM_TOOLS_FILE` (and vice versa). Added `len()` / `names()`.
- **The LLM client's registry is swappable** — `LlmClient::tools` moved from
`Option<Arc<ToolRegistry>>` to an `Arc<RwLock<…>>` with `set_tools(…)` /
`tools_from_request()` ([`src/llm.rs`](src/llm.rs)). The registry is re-read on
**every** request and at the start of every turn
([`turn_manager.rs`](src/orchestrator/turn_manager.rs)), so an override takes
effect on the next turn while a session in flight is unaffected.
- Docs: new [TOOL_CALLING.md](TOOL_CALLING.md) (IT/EN/ES operating guide) and a
"Runtime definitions" subsection in [README.md](README.md#tool-calling).
### Notes
- The override lives **in memory only**: it never writes the tools file, and a
restart restores the `LLM_TOOLS_FILE` definitions. When the endpoint is never
called, the file definitions stay in use — that remains the default.
- Verified on the running service: `GET` before any POST →
`{"count":2,"source":"file"}`; after a POST → `{"count":1,"source":"request"}`;
invalid JSON and `[]` → `400`; restart → back to
`{"count":2,"source":"file"}`.
---
## 2026-10-17 — Feat: `POST /api/transcribe` — Parakeet transcription of an uploaded audio file
### Added
- **New REST endpoint `POST /api/transcribe`** ([`src/transcribe.rs`](src/transcribe.rs)).
The body is the raw audio file (any container `ffmpeg` can read: wav, mp3,
m4a, ogg, flac…). The file is decoded to **mono 16-bit PCM at 16 kHz**
(`ffmpeg -ac 1 -ar 16000 -f s16le pipe:1`) and the **already-loaded** Parakeet
TDT model from [`AppState::stt`](src/ws/handler.rs) transcribes the whole take
in a single inference on a blocking thread (the inference is CPU-bound and
holds the ONNX session lock). Response:
`{"ok": true, "text": …, "duration_ms": …, "tokens": …}`.
- **Errors:** `400` empty body, `503` STT model not loaded, `502` ffmpeg decode
failed, `500` temp-file / inference failure. Uploads capped at **32 MB** (same
as `/api/customvoice`).
- **OpenAPI/Swagger UI:** the endpoint is documented (tag `transcribe`, schema
`TranscribeResponse`) in [`src/apidoc.rs`](src/apidoc.rs) and appears at
`/docs`.
### Notes
- **Whole-file, single transcript** by design (no VAD segmentation) and **no
`lang` parameter**: Parakeet TDT v3 is multilingual, the language is
auto-detected.
- No second ONNX environment is created — the model is shared with the live
session (respects the one-environment-per-process constraint). The upload is
spilled to a temp file (ffmpeg needs a seekable input) and removed afterwards.
- Verified end-to-end: `voice_profiles/solis.wav` (14.9 s, Spanish) →
`duration_ms=14918`, `tokens=83`, correct transcript; empty body → `400`.
---
## 2026-10-17 — Feat: OpenAPI document + Swagger UI at `/docs`
### Added
- **Swagger UI for the REST surface.** The engine now serves an interactive
Swagger UI at **`GET /docs`** backed by an OpenAPI 3 document at
**`GET /api-docs/openapi.json`**. The document is derived from the handlers
with [`utoipa`](https://docs.rs/utoipa) (`utoipa = "5"`, `axum_extras`) and
rendered by [`utoipa-swagger-ui`](https://docs.rs/utoipa-swagger-ui) v8
(axum 0.7). New module [`src/apidoc.rs`](src/apidoc.rs) holds the
`#[derive(OpenApi)]` document (`ApiDoc`) and the response schemas
(`PhrasesResponse`, `PhraseEntry`, `SetVoiceResponse`, `ClearResponse`,
`ErrorResponse`); the endpoints carry `#[utoipa::path]` annotations.
- **Documented endpoints:** `GET /health`, `GET /api/customvoice/phrases`,
`POST /api/customvoice` (query params `lang` / `text` via
`#[derive(IntoParams)]` on `SetParams`; raw `application/octet-stream` body),
`POST /api/customvoice/clear`. WebSocket (`/ws`) and the HTML client pages are
intentionally excluded (streaming / static assets).
### Notes
- The `zip` crate is pinned to the **2.2.x** line in `Cargo.toml`: the
`utoipa-swagger-ui` 8.1 build script (which unpacks the embedded UI assets)
targets the older zip 2.x API and does not compile against zip 2.3+/2.6. The
root `axum` stays at 0.7.9 — `utoipa-swagger-ui` 8.1 resolves to it, so no
duplicate Axum is pulled in.
- Swagger UI is only for the REST endpoints; `/docs` requires the same
HTTP/HTTPS origin as the engine.
---
## 2026-10-17 — Feat: custom voice accepts an uploaded audio file + typed transcript
### Added