valenti/spark-pocket-tts
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add default voice for each language (#180)

Browse Source 
Fix https://github.com/kyutai-labs/pocket-tts/issues/166
This commit is contained in:
Gabriel de Marmiesse
2026-05-04 15:35:22 +02:00
committed by GitHub
parent de010ea890
commit 7db278ee57
8 changed files with 94 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+27 -22
View File
@@ -48,33 +48,38 @@ uvx pocket-tts generate
pocket-tts generate pocket-tts generate
``` ```
Modify the voice with `--voice` and the text with `--text`. We provide a small catalog of voices. Modify the voice with `--voice` and the text with `--text`. We provide a small catalog of voices.
Choose a pretrained language model with `--language` when running `generate`, `export-voice`, or `serve` (default: `english`). Choose a pretrained language model with `--language` when running `generate`, `export-voice`, or `serve` (default: `english`). Non-english languages have also biggers 24 layers variants that are higher quality but slower. You can select them by using for example `--language italian_24l`.
The `--config` option accepts only a local YAML path for custom weights. The `--config` option accepts only a local YAML path for custom weights.
You can take a look at [this page](https://huggingface.co/kyutai/tts-voices) which details the licenses You can take a look at [this page](https://huggingface.co/kyutai/tts-voices) which details the licenses
for each voice. for each voice.
* [alba](https://huggingface.co/kyutai/tts-voices/blob/main/alba-mackenna/casual.wav) * [alba](https://huggingface.co/kyutai/tts-voices/blob/main/alba-mackenna/casual.wav) (en)
* [anna](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p228_023_enhanced.wav) * [giovanni](https://huggingface.co/kyutai/pocket-tts/blob/add_lang_not_documented/common_voice_it_36520747-enhanced-v2.mp3) (it)
* [azelma](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p303_023_enhanced.wav) * [lola](https://huggingface.co/kyutai/pocket-tts/blob/add_lang_not_documented/common_voice_es_19762977-enhanced-v2.mp3) (es)
* [bill_boerst](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/bill_boerst.wav) * [juergen](https://huggingface.co/kyutai/pocket-tts/blob/add_lang_not_documented/de-DE-juergen.mp3) (de)
* [caro_davy](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/caro_davy.wav) * [rafael](https://huggingface.co/kyutai/pocket-tts/blob/add_lang_not_documented/g-Vi8PgmSY0-enhanced-v2.wav) (pt)
* [charles](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p254_023_enhanced.wav) * [estelle](https://huggingface.co/kyutai/tts-voices/blob/main/unmute-prod-website/developpeuse-3.wav) (fr)
* [cosette](https://huggingface.co/kyutai/tts-voices/blob/main/expresso/ex04-ex02_confused_001_channel1_499s.wav) * [anna](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p228_023_enhanced.wav) (en)
* [eponine](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p262_023_enhanced.wav) * [azelma](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p303_023_enhanced.wav) (en)
* [eve](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p361_023_enhanced.wav) * [bill_boerst](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/bill_boerst.wav) (en)
* [fantine](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p244_023_enhanced.wav) * [caro_davy](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/caro_davy.wav) (en)
* [george](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p315_023_enhanced.wav) * [charles](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p254_023_enhanced.wav) (en)
* [jane](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p339_023_enhanced.wav) * [cosette](https://huggingface.co/kyutai/tts-voices/blob/main/expresso/ex04-ex02_confused_001_channel1_499s.wav) (en)
* [jean](https://huggingface.co/kyutai/tts-voices/blob/main/ears/p010/freeform_speech_01_enhanced.wav) * [eponine](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p262_023_enhanced.wav) (en)
* [javert](https://huggingface.co/kyutai/tts-voices/blob/main/voice-donations/Butter.wav) * [eve](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p361_023_enhanced.wav) (en)
* [marius](https://huggingface.co/kyutai/tts-voices/blob/main/voice-donations/Selfie.wav) * [fantine](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p244_023_enhanced.wav) (en)
* [mary](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p333_023_enhanced.wav) * [george](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p315_023_enhanced.wav) (en)
* [michael](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p360_023_enhanced.wav) * [jane](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p339_023_enhanced.wav) (en)
* [paul](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p259_023_enhanced.wav) * [jean](https://huggingface.co/kyutai/tts-voices/blob/main/ears/p010/freeform_speech_01_enhanced.wav) (en)
* [peter_yearsley](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/peter_yearsley.wav) * [javert](https://huggingface.co/kyutai/tts-voices/blob/main/voice-donations/Butter.wav) (en)
* [stuart_bell](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/stuart_bell.wav) * [marius](https://huggingface.co/kyutai/tts-voices/blob/main/voice-donations/Selfie.wav) (en)
* [vera](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p229_023_enhanced.wav) * [mary](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p333_023_enhanced.wav) (en)
* [michael](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p360_023_enhanced.wav) (en)
* [paul](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p259_023_enhanced.wav) (en)
* [peter_yearsley](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/peter_yearsley.wav) (en)
* [stuart_bell](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/stuart_bell.wav) (en)
* [vera](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p229_023_enhanced.wav) (en)
The `--voice` argument can also take a plain wav file as input for voice cloning. The `--voice` argument can also take a plain wav file as input for voice cloning.
You can use your own or check out our [voice repository](https://huggingface.co/kyutai/tts-voices). You can use your own or check out our [voice repository](https://huggingface.co/kyutai/tts-voices).
docs/CLI Commands/generate.md
+1 -1
View File
@@ -17,7 +17,7 @@ This will generate a WAV file `./tts_output.wav` with the default text and voice
### Core Options ### Core Options
- `--text TEXT`: Text to generate (default: "Hello world! I am Kyutai Pocket TTS. I'm fast enough to run on small CPUs. I hope you'll like me.") - `--text TEXT`: Text to generate (default: "Hello world! I am Kyutai Pocket TTS. I'm fast enough to run on small CPUs. I hope you'll like me.")
- `--voice VOICE`: Path to audio conditioning file (voice to clone) (default: "hf://kyutai/tts-voices/alba-mackenna/casual.wav"). Urls and local paths are supported. - `--voice VOICE`: Path to audio conditioning file (voice to clone). Defaults to a built-in voice chosen from the language: `giovanni` (it) for italian, `lola` (es) for spanish, `alba` (en) otherwise. Urls and local paths are supported.
- `--output-path OUTPUT_PATH`: Output path for generated audio (default: "./tts_output.wav") - `--output-path OUTPUT_PATH`: Output path for generated audio (default: "./tts_output.wav")
- `--language LANGUAGE`: Language for the TTS model, one of `'english_2026-01'`, `'english_2026-04'`, `'english'`, `'french_24l'`, `'german_24l'`, `'portuguese_24l'`, `'italian_24l'`, `'spanish_24l'` (default: `english`, which is the same model as `'english_2026-04'`). Incompatible with `--config`. The "24l" variants are bigger models, not distilled yet and here only as preview. - `--language LANGUAGE`: Language for the TTS model, one of `'english_2026-01'`, `'english_2026-04'`, `'english'`, `'french_24l'`, `'german_24l'`, `'portuguese_24l'`, `'italian_24l'`, `'spanish_24l'` (default: `english`, which is the same model as `'english_2026-04'`). Incompatible with `--config`. The "24l" variants are bigger models, not distilled yet and here only as preview.
pocket_tts/default_parameters.py
+16 -1
View File
@@ -1,4 +1,3 @@
DEFAULT_AUDIO_PROMPT = "alba"
DEFAULT_LANGUAGE = "english" DEFAULT_LANGUAGE = "english"
DEFAULT_TEMPERATURE = 0.7 DEFAULT_TEMPERATURE = 0.7
DEFAULT_LSD_DECODE_STEPS = 1 DEFAULT_LSD_DECODE_STEPS = 1
@@ -41,9 +40,25 @@ DEFAULT_TEXT_FOR_LANGUAGE = {
), ),
} }
DEFAULT_VOICE_FOR_LANGUAGE = {
"italian": "giovanni",
"spanish": "lola",
"german": "juergen",
"portuguese": "rafael",
"french": "estelle",
}
DEFAULT_VOICE_FALLBACK = "alba"
def get_default_text_for_language(language: str | None) -> str: def get_default_text_for_language(language: str | None) -> str:
for key, text in DEFAULT_TEXT_FOR_LANGUAGE.items(): for key, text in DEFAULT_TEXT_FOR_LANGUAGE.items():
if language is not None and key in language: if language is not None and key in language:
return text return text
return DEFAULT_TEXT_FOR_LANGUAGE[DEFAULT_LANGUAGE] return DEFAULT_TEXT_FOR_LANGUAGE[DEFAULT_LANGUAGE]
def get_default_voice_for_language(language: str | None) -> str:
for key, voice in DEFAULT_VOICE_FOR_LANGUAGE.items():
if language is not None and key in language:
return voice
return DEFAULT_VOICE_FALLBACK
pocket_tts/main.py
+15 -4
View File
@@ -16,7 +16,6 @@ from typing_extensions import Annotated
from pocket_tts.data.audio import stream_audio_chunks from pocket_tts.data.audio import stream_audio_chunks
from pocket_tts.default_parameters import ( from pocket_tts.default_parameters import (
DEFAULT_AUDIO_PROMPT,
DEFAULT_EOS_THRESHOLD, DEFAULT_EOS_THRESHOLD,
DEFAULT_FRAMES_AFTER_EOS, DEFAULT_FRAMES_AFTER_EOS,
DEFAULT_LSD_DECODE_STEPS, DEFAULT_LSD_DECODE_STEPS,
@@ -24,6 +23,7 @@ from pocket_tts.default_parameters import (
DEFAULT_TEMPERATURE, DEFAULT_TEMPERATURE,
MAX_TOKEN_PER_CHUNK, MAX_TOKEN_PER_CHUNK,
get_default_text_for_language, get_default_text_for_language,
get_default_voice_for_language,
) )
from pocket_tts.models.tts_model import TTSModel, export_model_state from pocket_tts.models.tts_model import TTSModel, export_model_state
from pocket_tts.utils.logging_utils import enable_logging from pocket_tts.utils.logging_utils import enable_logging
@@ -136,7 +136,7 @@ def text_to_speech(
raise HTTPException(status_code=400, detail="Text cannot be empty") raise HTTPException(status_code=400, detail="Text cannot be empty")
if voice_url is None and voice_wav is None: if voice_url is None and voice_wav is None:
voice_url = DEFAULT_AUDIO_PROMPT voice_url = get_default_voice_for_language(str(tts_model.origin))
if voice_url is not None and voice_wav is not None: if voice_url is not None and voice_wav is not None:
raise HTTPException(status_code=400, detail="Cannot provide both voice_url and voice_wav") raise HTTPException(status_code=400, detail="Cannot provide both voice_url and voice_wav")
@@ -223,8 +223,17 @@ def serve(
def generate( def generate(
text: Annotated[str, typer.Option(help="Text to generate")] = None, text: Annotated[str, typer.Option(help="Text to generate")] = None,
voice: Annotated[ voice: Annotated[
str, typer.Option(help="Path to audio conditioning file (voice to clone)") str | None,
] = DEFAULT_AUDIO_PROMPT, typer.Option(
help=(
"Path to audio conditioning file (voice to clone). "
"Defaults to a built-in voice chosen from the language: "
"'giovanni' for italian, 'lola' for spanish, 'juergen' for german, "
"'rafael' for portuguese, 'estelle' for french, 'alba' otherwise."
),
show_default=False,
),
] = None,
quiet: Annotated[bool, typer.Option("-q", "--quiet", help="Disable logging output")] = False, quiet: Annotated[bool, typer.Option("-q", "--quiet", help="Disable logging output")] = False,
language: Annotated[ language: Annotated[
str | None, str | None,
@@ -293,6 +302,8 @@ def generate(
) )
tts_model.to(device) tts_model.to(device)
if voice is None:
voice = get_default_voice_for_language(language)
model_state_for_voice = tts_model.get_state_for_audio_prompt(voice) model_state_for_voice = tts_model.get_state_for_audio_prompt(voice)
# Stream audio generation directly to file or stdout # Stream audio generation directly to file or stdout
audio_chunks = tts_model.generate_audio_stream( audio_chunks = tts_model.generate_audio_stream(
pocket_tts/models/tts_model.py
+1 -1
View File
@@ -295,7 +295,7 @@ class TTSModel(nn.Module):
if language is not None: if language is not None:
if language == "french": if language == "french":
raise ValueError( raise ValueError(
"The french model is not ready yet, please use 'french_24l' instead." "For technical reasons, only a larger 24-layer model is available for French. Please use the 'french_24l' language instead."
) )
config = CONFIGS_DIR / f"{language}.yaml" config = CONFIGS_DIR / f"{language}.yaml"
config = Path(config) config = Path(config)
pocket_tts/static/index.html
+6 -4
View File
@@ -40,13 +40,15 @@
id="voice-url-input" id="voice-url-input"
class="w-full border rounded p-2" class="w-full border rounded p-2"
placeholder="hf://kyutai/tts-voices/alba-mackenna/casual.wav" placeholder="hf://kyutai/tts-voices/alba-mackenna/casual.wav"
value="alba"
/> />
<p class="text-xs text-gray-500 mt-1"> <p class="text-xs text-gray-500 mt-1">
Supports: http://, https://, or hf:// URLs.<br> Supports: http://, https://, or hf:// URLs.<br>
You can also use predefined voices:<br> You can also use predefined voices (language code in parentheses):<br>
"alba", "anna", "azelma", "bill_boerst", "caro_davy", "charles", "eponine", "eve", "alba" (en), "anna" (en), "azelma" (en), "bill_boerst" (en), "caro_davy" (en),
"fantine", "george", "jane", "jean", "mary", "michael", "paul", "peter_yearsley", "stuart_bell", "vera". "charles" (en), "cosette" (en), "eponine" (en), "eve" (en), "fantine" (en),
"george" (en), "jane" (en), "javert" (en), "jean" (en),
"marius" (en), "mary" (en), "michael" (en), "paul" (en),
"peter_yearsley" (en), "stuart_bell" (en), "vera" (en), "giovanni" (it), "lola" (es), "juergen" (de), "rafael" (pt), "estelle" (fr).
<br/> <br/>
You can find more voices in our You can find more voices in our
<a <a
pocket_tts/utils/utils.py
+6 -1
View File
@@ -34,11 +34,16 @@ _ORIGINS_OF_PREDEFINED_VOICES = {
"peter_yearsley": "hf://kyutai/tts-voices/voice-zero/peter_yearsley.wav", "peter_yearsley": "hf://kyutai/tts-voices/voice-zero/peter_yearsley.wav",
"stuart_bell": "hf://kyutai/tts-voices/voice-zero/stuart_bell.wav", "stuart_bell": "hf://kyutai/tts-voices/voice-zero/stuart_bell.wav",
"caro_davy": "hf://kyutai/tts-voices/voice-zero/caro_davy.wav", "caro_davy": "hf://kyutai/tts-voices/voice-zero/caro_davy.wav",
"giovanni": "hf://kyutai/pocket-tts/common_voice_it_36520747-enhanced-v2.mp3@64ab7d24c479d736a83b8cc666c4a776fca30fda",
"lola": "hf://kyutai/pocket-tts/common_voice_es_19762977-enhanced-v2.mp3@64ab7d24c479d736a83b8cc666c4a776fca30fda",
"juergen": "hf://kyutai/pocket-tts/de-DE-juergen.mp3@64ab7d24c479d736a83b8cc666c4a776fca30fda",
"rafael": "hf://kyutai/pocket-tts/g-Vi8PgmSY0-enhanced-v2.wav@64ab7d24c479d736a83b8cc666c4a776fca30fda",
"estelle": "hf://kyutai/tts-voices/unmute-prod-website/developpeuse-3.wav@1fc7395b7e012e2bbebfca14b942a4ef62ccc899",
} }
def get_predefined_voice(language: str, name: str) -> str: def get_predefined_voice(language: str, name: str) -> str:
return f"hf://kyutai/pocket-tts-without-voice-cloning/languages/{language}/embeddings/{name}.safetensors@d29db7978e464fb90cb3359ee0c69a273b9142cc" return f"hf://kyutai/pocket-tts-without-voice-cloning/languages/{language}/embeddings/{name}.safetensors@e041936c75475d350b405bc870bcf7c22da4e9e6"
def make_cache_directory() -> Path: def make_cache_directory() -> Path:
scripts/generate_default_voices.py
+20 -17
View File
@@ -1,26 +1,29 @@
import sys from pathlib import Path
import scipy.io.wavfile
from pocket_tts import TTSModel, export_model_state from pocket_tts import TTSModel, export_model_state
from pocket_tts.utils.utils import _ORIGINS_OF_PREDEFINED_VOICES from pocket_tts.utils.utils import _ORIGINS_OF_PREDEFINED_VOICES
model = TTSModel.load_model(language=sys.argv[1]) languages = [x.stem for x in Path("./pocket_tts/config").glob("*.yaml")]
for voice_name, voice_origin in _ORIGINS_OF_PREDEFINED_VOICES.items():
print(f"Processing voice: {voice_name} from origin: {voice_origin}") for language in languages:
model = TTSModel.load_model(language=language)
for voice_name, voice_origin in _ORIGINS_OF_PREDEFINED_VOICES.items():
print(
f"Processing voice: {voice_name} from origin: {voice_origin} for language: {language}"
)
# Export a voice state for fast loading later # Export a voice state for fast loading later
model_state = model.get_state_for_audio_prompt(voice_origin) model_state = model.get_state_for_audio_prompt(voice_origin)
export_model_state(model_state, f"./built-in-voices/{voice_name}.safetensors") dest = f"/projects/huggingface/pocket-tts/languages/{language}/embeddings/{voice_name}.safetensors"
export_model_state(model_state, dest)
model_state_copy = model.get_state_for_audio_prompt( model_state_copy = model.get_state_for_audio_prompt(dest)
f"./built-in-voices/{voice_name}.safetensors"
)
audio = model.generate_audio( # audio = model.generate_audio(
model_state_copy, # model_state_copy,
"An Sommernachmittagen spielten die Kinder fröhlich auf dem Platz der kleinen Stadt.", # "An Sommernachmittagen spielten die Kinder fröhlich auf dem Platz der kleinen Stadt.",
) # )
scipy.io.wavfile.write( # scipy.io.wavfile.write(
f"./built-in-voices-generated/{voice_name}.wav", model.sample_rate, audio.numpy() # f"./built-in-voices-generated/{voice_name}.wav", model.sample_rate, audio.numpy()
) # )