good

.env.example

@@ -73,11 +73,13 @@ FILLER_GAIN_IDLE=1.0
 FILLER_GAIN_DUCK=0.15
 # TTS jitter buffer (ms): how much synthesized audio the mixer buffers before it
 # starts playing the reply. Lower values start speaking sooner; higher values
-# absorb uneven TTS chunk arrival at the cost of added latency. Drama-3 delivers
+# absorb uneven TTS chunk arrival at the cost of added latency. Drama-3's vocoder
-# ~1.08x real time in bursts every ~2.7s, so raising this reduces drop-outs on
+# emits audio in windows of `stream_followup_stride` codec frames (2048 samples =
-# long replies. Mid-reply underruns resume as soon as any audio is queued again
+# 46.4 ms each); with the recommended server setting (stride 16, ramp
-# (no full refill).
+# "[10,12,16]") the first window is 10 frames = 0.464 s, so 450 ms starts
-TTS_PREBUFFER_MS=900
+# playback on that very first window. Mid-reply underruns resume as soon as any
+# audio is queued again (no full refill).
+TTS_PREBUFFER_MS=450
 # Number of TTS requests kept in flight at once. Reply chunks are synthesized
 # ahead of playback (and replayed in order) so each request's start-up latency
 # hides behind the previous chunk's audio instead of leaving an audible gap.

README.md

@@ -172,7 +172,7 @@ See [`.env.example`](.env.example) for a copy-paste template.
 | `FILLER_DELAY_MS` | `2500` | How long (ms) to wait for the response audio to start before playing the filler. If the reply starts earlier, the filler is dropped instead of talking over the beginning of the answer. |
 | `FILLER_GAIN_IDLE` | `1.0` | Filler gain when TTS is idle. |
 | `FILLER_GAIN_DUCK` | `0.15` | Filler gain while TTS is active (ducking). |
-| `TTS_PREBUFFER_MS` | `900` | Synthesized audio (ms) buffered on the TTS channel before the reply starts playing. Lower = the agent speaks sooner; higher = fewer drop-outs on uneven TTS chunk arrival. Only awaited at the start of a reply; mid-reply underruns resume as soon as any audio is queued again. |
+| `TTS_PREBUFFER_MS` | `450` | Synthesized audio (ms) buffered on the TTS channel before the reply starts playing. Lower = the agent speaks sooner; higher = fewer drop-outs on uneven TTS chunk arrival. Only awaited at the start of a reply; mid-reply underruns resume as soon as any audio is queued again. |
 | `TTS_PIPELINE_DEPTH` | `3` | How many TTS requests may be in flight at once. Chunks are synthesized ahead of playback and replayed in order, so each request's start-up latency hides behind the previous chunk's audio. `1` restores strictly serial synthesis. |
 | `BACKCHANNEL_ENABLED` | `true` | Master switch for during-utterance acknowledgments. |
 | `BACKCHANNEL_DIR` | `assets/backchannel` | Directory of `*.raw` backchannel PCM clips (16-bit LE mono, 44.1 kHz). |
@@ -266,6 +266,51 @@ An unknown value panics at startup with a clear message.
 - Supports voice cloning via the `references` field.
 - Honors barge-in cancellation.
 
+##### Starting the Drama 3 backend (SGLang-Omni)
+
+The `fish` engine talks to an **SGLang-Omni** server running Drama 3. Start it
+with the vocoder streaming window narrowed, otherwise the vocoder emits ~3 s
+bursts and the client is forced into a large pre-buffer:
+
+```bash
+sgl-omni serve \
+    --model-path fishaudio/drama-3 \
+    --host 0.0.0.0 --port 8000 \
+    --tts_engine.gpu_memory_fraction 0.45 \
+    --tts_engine.factory.max_running_requests 4 \
+    --vocoder.factory.stream_followup_stride 16 \
+    --vocoder.factory.stream_followup_stride_ramp "[10,12,16]"
+```
+
+In this deployment the launcher is `/root/drama3/start-drama3.sh` and the server
+is kept alive by **pm2** as the process `start-drama3`:
+
+```bash
+pm2 restart start-drama3 --update-env   # after editing the launcher
+pm2 logs start-drama3 --lines 100       # CUDA-graph rebuild takes ~2 min at start
+```
+
+The two `--vocoder.factory.*` overrides are the key part:
+
+| Flag | Meaning |
+|---|---|
+| `stream_followup_stride` | Steady-state window, in codec frames (`2048` samples = `46.4 ms` each). `16` ⇒ `0.743 s` bursts; the upstream default `64` ⇒ `2.97 s`. |
+| `stream_followup_stride_ramp` | Window sizes for the first windows, as a JSON tuple. `[10,12,16]` emits a `0.464 s` then a `0.557 s` window first, so playback can start sooner. Upstream default `(10,12,32)`. |
+
+Two gotchas when writing the command:
+
+- The flags are **dotted keys into the pydantic config**, so they go *without* a
+  `stages.` prefix — `--vocoder.factory.stream_followup_stride`, not
+  `--stages.vocoder.factory.stream_followup_stride` (the latter is rejected).
+- The ramp tuple must be passed as a **JSON string** — `"[10,12,16]"` — because
+  the config manager parses non-scalar paths with JSON.
+
+Smaller stride = finer delivery granularity but more compute: each window
+re-decodes `stream_overlap_tokens` (default `32`) of context, so `stride 16`
+decodes 48 frames per 16 emitted. Keep the stride at or below the client's
+`TTS_PREBUFFER_MS` budget (see *Mixer & jitter buffering*) — with the defaults
+above, `TTS_PREBUFFER_MS=450` starts playback on the first window.
+
 #### `kokoro` — Kokoro TTS *(stub)*
 
 [`src/tts/kokoro.rs`](src/tts/kokoro.rs)
@@ -630,6 +675,12 @@ A ready-to-use speech-to-speech example client is served at **`/mivocero`**
   (44.1 kHz, 16-bit LE mono, 20 ms frames).
 - **Chat box** — shows the user transcript (from STT) and the assistant reply
   **as it streams**, plus messages typed in the composer.
+- **Karaoke highlighting** — the assistant bubble is rendered word by word: each
+  word lights up as it is *spoken*, driven by the audio playhead (the playback
+  worklet reports the samples it has actually emitted). Words already spoken are
+  bright, the word currently sounding is accented, and the rest stay dim. The
+  rate is an estimate (`KARAOKE_CPS`, ~15 chars/s) — it only affects the visual
+  highlight, never the audio.
 - **Text composer** — typing and sending a message runs a full LLM → TTS turn
   without using the microphone (`{"type":"user_text","text":…}`).
 - **Interrupt button** — flushes audio and cancels the in-flight reply
@@ -690,13 +741,18 @@ Constants are defined in [`src/config.rs`](src/config.rs):
   `0.15`); the backchannel gain is fixed (`BACKCHANNEL_GAIN`, default `0.35`).
   Output is clamped to `i16`.
 - **Pre-buffer:** TTS playback does not start until the queue holds
-  ≥ `TTS_PREBUFFER_MS` (default **900 ms**) of synthesized audio. It absorbs
+  ≥ `TTS_PREBUFFER_MS` (default **450 ms**) of synthesized audio. It absorbs
   uneven TTS chunk arrival; lower it to start speaking sooner, raise it for fewer
   drop-outs. With pipelined synthesis (above) the pre-buffer only has to cover
   arrival jitter, not the TTS engine's per-request start-up, so it can stay
-  small. Note: Drama-3 delivers ~1.08× real time in bursts every ~2.7 s, so a
+  small. Note: Drama-3 delivers ~1.04× real time, and its *granularity* is set by
-  ~900 ms buffer favours latency over perfect gaplessness on very long replies —
+  the server's vocoder window
-  raising it towards ~3 s is what actually removes the remaining drop-outs.
+  (`--vocoder.factory.stream_followup_stride`, in codec frames of 2048 samples =
+  46.4 ms). With stride `16` and ramp `"[10,12,16]"` the first window is 10
+  frames = 0.464 s, so `450 ms` starts playback on that very first window (the
+  steady state then delivers a 0.743 s burst every ~0.69 s). With the upstream
+  default (`64`, a 2.97 s burst every ~2.7 s) gapless playback would instead
+  require a ~3 s pre-buffer.
 - **Resilient resume:** the full pre-buffer is only required at the *first*
   playback of a reply. If the queue empties mid-reply (after the 200 ms drain
   grace) playback pauses, but resumes as soon as *any* audio is queued again —

UPDATES.md

@@ -5,6 +5,126 @@ Entries are written in English and ordered newest-first.
 
 ---
 
+## 2026-10-16 — Mivocero: karaoke word highlighting on the assistant reply
+
+### Changed
+
+- **The `/mivocero` client now reveals the assistant reply in karaoke style**
+  ([`src/web/mivocero.html`](src/web/mivocero.html)) instead of appending plain
+  text. Each word is a `<span>`; words light up as they are *spoken*. The
+  highlight is driven by the **audio playhead**: the playback AudioWorklet now
+  counts the samples it has actually emitted and reports them (`played`) on its
+  existing per-block message. The client maps that count onto a character offset
+  with an estimated speaking rate (`KARAOKE_CPS`, ~15 chars/s) and marks each
+  word `done` / `now` / pending.
+- The old blinking-caret "pending" style was removed; the whole reply is shown at
+  once (dim) and progressively illuminated.
+- A safety net finishes the highlight: once `assistant_end` has been received and
+  playback has been idle for 600 ms *after it had actually started*, any word
+  still pending is marked done, so a reply always ends fully lit. New turns and
+  interrupts cancel the loop.
+
+### Notes
+
+- The mapping from audio to text is inherently approximate (the server streams
+  text and PCM on the same socket without per-word timing), so `KARAOKE_CPS` is a
+  tunable estimate for the visual effect only — it never touches the audio.
+- `played` counts only real samples, so the playback jitter buffer's silence and
+  any underrun pauses do not advance the highlight.
+
+---
+
+## 2026-10-16 — End-to-end validation → `TTS_PREBUFFER_MS` 700 → 450 ms; Drama-3 startup documented
+
+### Changed
+
+- **`TTS_PREBUFFER_MS` lowered `700` → `450`** (`.env`, `.env.example`,
+  [`src/config.rs`](src/config.rs)). With the vocoder ramp `"[10,12,16]"` the
+  *first* window is 10 codec frames = 0.464 s, so `450 ms` sits just below it and
+  playback now starts on that **first** window instead of waiting for the second
+  one that `700 ms` required.
+- **README:** new *"Starting the Drama 3 backend (SGLang-Omni)"* subsection under
+  the `fish` plugin, with the exact `sgl-omni serve` command, the **pm2** workflow,
+  the meaning of the two `--vocoder.factory.*` flags and the two CLI gotchas (no
+  `stages.` prefix; the ramp tuple passed as a JSON string).
+
+### Why
+
+- An end-to-end browser test (`RUST_LOG=…,audio_agent_engine::tts=debug,…`) on the
+  live engine confirmed the narrower windows: bursts of 10/12/16 frames
+  (0.464 / 0.557 / 0.743 s) instead of a single 2.97 s block, `prebuffer=30870`
+  samples (= 700 ms) and time-to-first-audio ≈ 1.1 s from dispatch (~0.64 s of
+  which is the server's TTFB). Barge-in fired correctly on three interrupts.
+- Three **tail underruns** were still observed at the end of short replies: e.g.
+  2.46 s of audio produced in ~2.6 s (≈1.07× real time including the fixed
+  ~0.64 s TTFB), so the queue drained just before the last words. The mixer
+  recovered via resilient resume, but the gap is audible. `450 ms` targets the
+  first 0.464 s window and trades a little more tail risk for a markedly earlier
+  start.
+
+### Notes
+
+- The residual deficit is dominated by the server's **fixed ~0.64 s
+  time-to-first-byte**, which no pre-buffer value can remove. Going lower would
+  need a faster first token (model / preprocessing), not a smaller stride.
+
+---
+
+## 2026-10-16 — Finer TTS delivery granularity (Drama-3 vocoder window) → `TTS_PREBUFFER_MS` 700 ms
+
+### Changed
+
+- **The SGLang Drama-3 server now emits audio in ~0.74 s windows instead of
+  ~2.97 s bursts.** Two flags were added to the server launcher
+  (`/root/drama3/start-drama3.sh`, managed by `pm2` as `start-drama3`):
+  `--vocoder.factory.stream_followup_stride 16` and
+  `--vocoder.factory.stream_followup_stride_ramp "[10,12,16]"` — upstream
+  defaults were `64` and `(10,12,32)`.
+- **`TTS_PREBUFFER_MS` default lowered `900` → `700`** (`.env`, `.env.example`,
+  [`src/config.rs`](src/config.rs)). 700 ms is deliberately just under one 0.743 s
+  window, so playback starts on the *first* burst rather than waiting for a
+  second one.
+
+### Why
+
+- Drama-3's streaming granularity is a **vocoder** setting, not an engine one:
+  [`Drama3VocoderFactoryArgs`](../drama3/sglang-omni/sglang_omni/models/fishaudio_drama3/config.py)
+  buffers `stream_followup_stride` codec frames before decoding and emitting, and
+  `frame_length = hop_length * 4 = 2048` samples (46.4 ms at 44.1 kHz). The
+  upstream `64` therefore means a 131 072-sample burst every ~2.7 s.
+- Measured against the live endpoint (single, uncontended request, PCM stream):
+
+  | | burst | period | ratio | first audio |
+  |---|---|---|---|---|
+  | before (`stride` 64) | 131 072 samples / 2.97 s | 2.70–2.78 s | 1.08× | 0.96 s |
+  | after (`stride` 16) | 32 768 samples / 0.743 s | 0.69 s (σ ≈ 0.01 s) | 1.04× | 1.34 s |
+
+- The finer window is what makes the small pre-buffer viable: the producer still
+  runs slightly faster than real time (1.04×), but it now tops the queue up every
+  0.69 s instead of every ~2.7 s, so arrival jitter is bounded by one window and
+  no longer forces a multi-second buffer.
+- Prefix caching (RadixAttention) was evaluated and **rejected**: the client
+  already sends a constant prefix (`model` / `voice` / `references`), yet an A/B
+  on the endpoint showed no TTFB difference (ref 0.62–0.64 s, no-ref 0.52 s) —
+  that start-up is decode/vocoder, not prompt prefill.
+
+### Notes
+
+- Each window re-decodes `stream_overlap_tokens` (32) frames of context, so a
+  smaller stride costs more vocoder compute: 96 frames decoded per 64 emitted
+  becomes 48 per 16. The measured ratio moved 1.08× → 1.04×, still above real
+  time; `stride` 8 (0.371 s windows, 40 frames decoded per 8 emitted) is the next
+  step down and should only be tried with the ratio re-measured.
+- The window lengths feed the vocoder's CUDA-graph capture set, so changing them
+  requires a server restart.
+- `stream_stride` (first window) and `stream_followup_stride_ramp` (the windows
+  after it) were left near their defaults — only the steady-state value drives
+  the granularity of a long reply.
+- (Superseded: the `TTS_PREBUFFER_MS` default is now `450`, targeting the ramp's
+  0.464 s *first* window — see the entry above.)
+
+---
+
 ## 2026-10-16 — Resilient mixer resume + `TTS_PREBUFFER_MS` default 900 ms
 
 ### Changed
@@ -42,6 +162,8 @@ Entries are written in English and ordered newest-first.
 - Default `900` is a deliberate latency-first choice: it keeps time-to-first-audio
   low (~1.8 s) and accepts a rare short drop-out on very long replies. Raising
   `TTS_PREBUFFER_MS` towards ~3000 trades that latency back for gaplessness.
+  (Superseded: the default is now `700`, once the server's vocoder window was
+  narrowed to ~0.74 s — see the entry above.)
 
 ### Notes
 

run/server.pid

@@ -1 +1 @@
-3004143
+3212397

src/config.rs

@@ -84,7 +84,7 @@ pub struct Config {
     /// per turn.
     pub filler_delay_ms: u64,
     /// TTS jitter buffer (ms) the mixer must fill before the response audio
-    /// starts playing (`TTS_PREBUFFER_MS`, default `900`). Larger values absorb
+    /// starts playing (`TTS_PREBUFFER_MS`, default `450`). Larger values absorb
     /// uneven TTS chunk arrival (fewer drop-outs) at the cost of added
     /// time-to-first-audio. The full buffer is only required at the start of a
     /// reply; after a mid-reply underrun the mixer resumes as soon as any audio
@@ -184,7 +184,7 @@ impl Default for Config {
             fillers_dir: PathBuf::from("assets/fillers"),
             filler_enabled: true,
             filler_delay_ms: 2500,
-            tts_prebuffer_ms: 900,
+            tts_prebuffer_ms: 450,
             tts_pipeline_depth: 3,
             filler_gain_idle: 1.0,
             filler_gain_duck: 0.15,

src/web/mivocero.html

@@ -89,12 +89,10 @@
   .msg.agent { align-self: flex-start; }
   .msg.agent .bubble { background: var(--bg); border: 1px solid var(--border); }
   .msg.error .bubble { border-color: var(--err); color: var(--err); }
-  .msg .bubble.pending::after {
+  /* Karaoke: words light up as they are spoken by the agent. */
-    content: "▍";
+  .bubble.karaoke .kw { color: var(--muted); transition: color .18s ease; }
-    animation: blink 1s step-start infinite;
+  .bubble.karaoke .kw.done { color: var(--text); }
-    color: var(--accent);
+  .bubble.karaoke .kw.now { color: var(--accent); text-shadow: 0 0 8px rgba(79,140,255,.55); }
-  }
-  @keyframes blink { 50% { opacity: 0; } }
   .hint { color: var(--muted); font-size: 13px; text-align: center; margin: auto; }
 
   footer { border-top: 1px solid var(--border); padding: 12px 18px; }
@@ -198,6 +196,17 @@ let playbackNode = null;
 let outputGain = null;
 let agentBubble = null;   // bubble being streamed by the assistant
 let agentText = '';       // accumulated assistant text for that bubble
+let agentWords = [];      // [{el, start}] word spans of the current reply
+let agentPlayedAt = 0;    // playback sample counter when the reply started
+let agentEnded = false;   // assistant_end received for the current reply
+let playedSamples = 0;    // cumulative samples emitted by the playback worklet
+let karaokeRAF = 0;       // requestAnimationFrame id while the reply streams
+let karaokePrevPlayed = 0;
+let karaokePrevTs = 0;
+let karaokeIdleMs = 0;
+// Approximate speaking rate used to map the audio playhead onto characters.
+// Only drives the visual highlight — the spoken audio is unaffected.
+const KARAOKE_CPS = 15;
 
 function log(msg) {
   const ts = new Date().toLocaleTimeString();
@@ -237,20 +246,95 @@ function addMessage(role, text, cls = '') {
 
 function startAgentMessage() {
   agentText = '';
+  agentWords = [];
   agentBubble = addMessage('agent', '');
-  agentBubble.classList.add('pending');
+  agentBubble.classList.add('karaoke');
+  agentPlayedAt = playedSamples;   // count only the audio spoken after this point
+  startKaraoke();
 }
 
 function appendAgentDelta(text) {
   if (!agentBubble) startAgentMessage();
   agentText += text;
-  agentBubble.textContent = agentText;
+  renderKaraoke();
+  applyKaraoke(karaokeChars());
   scrollDown(false);
 }
 
+// Rebuild the bubble as one <span> per word, keeping the whitespace between
+// them so the layout is identical to plain text.
+function renderKaraoke() {
+  if (!agentBubble) return;
+  agentWords = [];
+  agentBubble.textContent = '';
+  let idx = 0;
+  for (const tok of agentText.split(/(\s+)/)) {
+    if (tok === '') continue;
+    if (/^\s+$/.test(tok)) {
+      agentBubble.appendChild(document.createTextNode(tok));
+    } else {
+      const span = document.createElement('span');
+      span.className = 'kw';
+      span.textContent = tok;
+      agentBubble.appendChild(span);
+      agentWords.push({ el: span, start: idx });
+    }
+    idx += tok.length;
+  }
+}
+
+// Characters the playhead has "reached", from the audio actually emitted.
+function karaokeChars() {
+  const played = Math.max(0, playedSamples - agentPlayedAt);
+  return Math.floor((played / SAMPLE_RATE_OUT) * KARAOKE_CPS);
+}
+
+function applyKaraoke(chars) {
+  for (const w of agentWords) {
+    const end = w.start + w.el.textContent.length;
+    w.el.classList.toggle('done', end <= chars);
+    w.el.classList.toggle('now', w.start < chars && end > chars);
+  }
+}
+
+function startKaraoke() {
+  stopKaraoke();
+  agentEnded = false;
+  karaokePrevPlayed = playedSamples;
+  karaokeIdleMs = 0;
+  karaokePrevTs = performance.now();
+  karaokeRAF = requestAnimationFrame(karaokeTick);
+}
+
+function stopKaraoke() {
+  if (karaokeRAF) cancelAnimationFrame(karaokeRAF);
+  karaokeRAF = 0;
+}
+
+function karaokeTick(ts) {
+  karaokeRAF = 0;
+  if (!agentBubble) return;
+  const dt = ts - karaokePrevTs;
+  karaokePrevTs = ts;
+  const played = Math.max(0, playedSamples - agentPlayedAt);
+  if (played > karaokePrevPlayed) { karaokePrevPlayed = played; karaokeIdleMs = 0; }
+  else karaokeIdleMs += dt;
+
+  // Once the text has ended and the audio has drained (playback idle for a
+  // beat after it had actually started), light up anything still pending so
+  // the reply always ends fully highlighted.
+  if (agentEnded && karaokePrevPlayed > 0 && karaokeIdleMs > 600) {
+    for (const w of agentWords) { w.el.classList.add('done'); w.el.classList.remove('now'); }
+    return;
+  }
+  applyKaraoke(karaokeChars());
+  karaokeRAF = requestAnimationFrame(karaokeTick);
+}
+
 function endAgentMessage() {
-  if (agentBubble) agentBubble.classList.remove('pending');
+  stopKaraoke();
   agentBubble = null;
+  agentWords = [];
   agentText = '';
 }
 
@@ -314,6 +398,7 @@ class PlaybackProcessor extends AudioWorkletProcessor {
     this.buffering = true;
     this.underrunSamples = 0;
     this.underrunThreshold = Math.floor(sampleRate * 0.10);
+    this.played = 0;   // samples actually emitted (drives the karaoke highlight)
     this.port.onmessage = (e) => {
       const pcm = e.data;
       for (let i = 0; i < pcm.length; i++) {
@@ -337,6 +422,7 @@ class PlaybackProcessor extends AudioWorkletProcessor {
           s = this.buf[this.tail];
           this.tail = (this.tail + 1) % this.buf.length;
           this.underrunSamples = 0;
+          this.played++;
         } else {
           this.underrunSamples++;
           if (this.underrunSamples >= this.underrunThreshold) { this.buffering = true; this.underrunSamples = 0; }
@@ -345,7 +431,7 @@ class PlaybackProcessor extends AudioWorkletProcessor {
       out[i] = s;
       level += s * s;
     }
-    this.port.postMessage({ rms: Math.sqrt(level / out.length) });
+    this.port.postMessage({ rms: Math.sqrt(level / out.length), played: this.played });
     return true;
   }
 }
@@ -379,6 +465,7 @@ async function startAudio() {
 
   playbackNode = new AudioWorkletNode(audioCtx, 'playback-processor');
   playbackNode.port.onmessage = (e) => {
+    if (e.data.played !== undefined) playedSamples = e.data.played;
     if (e.data.rms !== undefined) outMeter.style.width = Math.min(100, e.data.rms * 200) + '%';
   };
   outputGain = audioCtx.createGain();
@@ -421,7 +508,7 @@ function handleServerText(text) {
       appendAgentDelta(event.text);
       break;
     case 'assistant_end':
-      endAgentMessage();
+      agentEnded = true;   // keep the bubble: karaoke finishes when audio drains
       break;
     case 'error':
       endAgentMessage();