docs: consolidate English documentation under root docs
This commit is contained in:
@@ -0,0 +1,151 @@
|
||||
# Text-to-speech backends
|
||||
|
||||
## Available TTS modes (`--tts`)
|
||||
|
||||
Runtime-supported values in [s2s_pipeline.py](../../speech_to_speech/s2s_pipeline.py):
|
||||
|
||||
- `chatTTS` → `chatTTS_handler.py`
|
||||
- `facebookMMS` → `facebookmms_handler.py`
|
||||
- `pocket` → `pocket_tts_handler.py`
|
||||
- `kokoro` → `kokoro_handler.py`
|
||||
- `qwen3` → `qwen3_tts_handler.py`
|
||||
|
||||
Run examples from the repository root in an environment with the speech
|
||||
dependencies installed. The UI requirements do not install TTS dependencies.
|
||||
|
||||
## Usage
|
||||
|
||||
### 1) ChatTTS (`--tts chatTTS`)
|
||||
|
||||
Primary args prefix: `--chat_tts_*`
|
||||
|
||||
```bash
|
||||
python -m speech_to_speech.s2s_pipeline \
|
||||
--tts chatTTS \
|
||||
--chat_tts_device cuda \
|
||||
--chat_tts_stream true \
|
||||
--chat_tts_chunk_size 512
|
||||
```
|
||||
|
||||
### 2) Facebook MMS (`--tts facebookMMS`)
|
||||
|
||||
Primary args prefix: `--facebook_mms_*` plus `--tts_language`
|
||||
|
||||
```bash
|
||||
python -m speech_to_speech.s2s_pipeline \
|
||||
--tts facebookMMS \
|
||||
--facebook_mms_device cuda \
|
||||
--tts_language en
|
||||
```
|
||||
|
||||
This handler maps STT language codes (e.g. `en`, `fr`, `es`) to MMS model suffixes (e.g. `eng`, `fra`, `spa`) and reloads the model on language changes.
|
||||
|
||||
### 3) Pocket TTS (`--tts pocket`)
|
||||
|
||||
Primary args prefix: `--pocket_tts_*`
|
||||
|
||||
```bash
|
||||
python -m speech_to_speech.s2s_pipeline \
|
||||
--tts pocket \
|
||||
--pocket_tts_voice jean \
|
||||
--pocket_tts_device cpu \
|
||||
--pocket_tts_sample_rate 16000
|
||||
```
|
||||
|
||||
Available preset voices include:
|
||||
`alba`, `marius`, `javert`, `jean`, `fantine`, `cosette`, `eponine`, `azelma`.
|
||||
|
||||
### 4) Kokoro (`--tts kokoro`)
|
||||
|
||||
Primary args prefix: `--kokoro_*`
|
||||
|
||||
```bash
|
||||
python -m speech_to_speech.s2s_pipeline \
|
||||
--tts kokoro \
|
||||
--kokoro_device auto \
|
||||
--kokoro_voice bm_fable \
|
||||
--kokoro_lang_code b
|
||||
```
|
||||
|
||||
Behavior:
|
||||
- Uses MLX backend on Apple Silicon (`mlx-community/Kokoro-82M-bf16`)
|
||||
- Uses native kokoro pipeline otherwise (`hexgrad/Kokoro-82M`)
|
||||
- Can auto-switch voice/language based on STT language code mapping
|
||||
|
||||
### 5) Qwen3-TTS (`--tts qwen3`)
|
||||
|
||||
Primary args prefix: `--qwen3_tts_*`
|
||||
|
||||
```bash
|
||||
python -m speech_to_speech.s2s_pipeline \
|
||||
--tts qwen3 \
|
||||
--qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
|
||||
--qwen3_tts_device cuda \
|
||||
--qwen3_tts_backend ggml \
|
||||
--qwen3_tts_speaker Aiden \
|
||||
--qwen3_tts_language auto \
|
||||
--qwen3_tts_non_streaming_mode True
|
||||
```
|
||||
|
||||
Behavior:
|
||||
- Uses `faster-qwen3-tts` on non-macOS platforms, defaulting to the GGML backend. Pass `--qwen3_tts_backend torch` to use the CUDA-graphs backend instead.
|
||||
- Uses `mlx-audio` on Apple Silicon and auto-maps `Qwen/...` model IDs to `mlx-community/...`, defaulting to the `6bit` MLX variant unless the model name already pins a suffix.
|
||||
- Supports MLX quantization overrides on Apple Silicon via `--qwen3_tts_mlx_quantization bf16|4bit|6bit|8bit`.
|
||||
- Keeps the existing voice-clone/custom-voice/voice-design handler flow intact.
|
||||
- Defaults to the CustomVoice model with speaker `Aiden`, so no reference audio is required. Voice-clone/base models can still use `--qwen3_tts_ref_audio`.
|
||||
|
||||
Install notes for Linux GGML:
|
||||
- The default PyPI `qwentts-cpp-python` wheel targets CUDA 12.8.
|
||||
- If that wheel does not match your CUDA runtime, install one of the Hugging Face wheelhouse builds before installing `speech-to-speech`.
|
||||
|
||||
```bash
|
||||
pip install "qwentts-cpp-python==0.3.1+cu130" \
|
||||
-f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu130
|
||||
pip install speech-to-speech
|
||||
```
|
||||
|
||||
Available wheelhouse directories include `cu124`, `cu128`, `cu130`, and `cpu`.
|
||||
|
||||
Example for Apple Silicon using the default 6-bit MLX variant:
|
||||
|
||||
```bash
|
||||
python -m speech_to_speech.s2s_pipeline \
|
||||
--tts qwen3 \
|
||||
--qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
|
||||
--qwen3_tts_speaker Aiden
|
||||
```
|
||||
|
||||
You can override the default and select `bf16`, `4bit`, or `8bit` explicitly:
|
||||
|
||||
```bash
|
||||
python -m speech_to_speech.s2s_pipeline \
|
||||
--tts qwen3 \
|
||||
--qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
|
||||
--qwen3_tts_mlx_quantization 4bit \
|
||||
--qwen3_tts_speaker Aiden
|
||||
```
|
||||
|
||||
For voice-cloning examples, an optional reference clip is retained at
|
||||
[docs/examples/reference-audio.wav](../examples/reference-audio.wav). Supply an
|
||||
appropriate base/voice-cloning model and `--qwen3_tts_ref_audio` explicitly;
|
||||
CustomVoice mode does not need this clip. The repository does not include a TTS
|
||||
benchmark script.
|
||||
|
||||
## Setup
|
||||
|
||||
### Low-latency GPU setup
|
||||
|
||||
```bash
|
||||
python -m speech_to_speech.s2s_pipeline \
|
||||
--stt whisper \
|
||||
--tts pocket
|
||||
```
|
||||
|
||||
### Apple Silicon setup
|
||||
|
||||
```bash
|
||||
python -m speech_to_speech.s2s_pipeline \
|
||||
--local_mac_optimal_settings
|
||||
```
|
||||
|
||||
`--tts pocket` and `--tts kokoro` are also valid options on macOS.
|
||||
Reference in New Issue
Block a user