feat(voice/clone): support MP3 and other audio formats for reference voices (#20)

## Summary
- Adds support for MP3, FLAC, OGG, and other audio formats when
uploading reference voices for cloning
- Replace WAV-only `audio_read()` with a `soundfile`-based
implementation
- Update web UI file picker to accept multiple audio formats

## Changes
- `pocket_tts/data/audio.py`: Use `soundfile` library instead of
Python's `wave` module
- `pocket_tts/main.py`: Preserve original file extension in temp files
for format detection
- `pocket_tts/main.py`: Add missing `truncate=True` to CLI generate
command
- `pocket_tts/static/index.html`: Update file input to accept
`.wav,.mp3,.flac,.ogg,.m4a,audio/*`
- `pyproject.toml`: Add `soundfile>=0.12.0` dependency

## Tested
- [x] Upload a WAV file as reference voice and generate speech
- [x] Upload an MP3 file as reference voice and generate speech
- [x] Upload a FLAC file as reference voice and generate speech
- [x] Verify CLI `--voice` option works with MP3 files

---------

Co-authored-by: Gabriel de Marmiesse <gabrieldemarmiesse@gmail.com>
Co-authored-by: Václav Volhejn <8401624+vvolhejn@users.noreply.github.com>
This commit is contained in:
Und3rf10w
2026-01-20 10:12:50 +01:00
committed by GitHub
co-authored by Gabriel de Marmiesse Václav Volhejn
parent 6f9dd250c2
commit bedd1c4098
5 changed files with 155 additions and 15 deletions
+27 -9
View File
@@ -21,17 +21,35 @@ FIRST_CHUNK_LENGTH_SECONDS = float(os.environ.get("FIRST_CHUNK_LENGTH_SECONDS",
def audio_read(filepath: str | Path) -> tuple[torch.Tensor, int]:
"""Read audio using Python's wave module."""
with wave.open(str(filepath), "rb") as wav_file:
sample_rate = wav_file.getframerate()
"""Read audio file. WAV uses built-in wave module; other formats require soundfile."""
filepath = Path(filepath)
# Read all audio data as 16-bit signed integers
raw_data = wav_file.readframes(-1)
samples = np.frombuffer(raw_data, dtype=np.int16).astype(np.float32) / 32768.0
if filepath.suffix.lower() == ".wav":
# Use built-in wave module for WAV files
with wave.open(str(filepath), "rb") as wav_file:
sample_rate = wav_file.getframerate()
n_channels = wav_file.getnchannels()
raw_data = wav_file.readframes(-1)
samples = np.frombuffer(raw_data, dtype=np.int16).astype(np.float32) / 32768.0
if n_channels > 1:
samples = samples.reshape(-1, n_channels).mean(axis=1)
return torch.from_numpy(samples).unsqueeze(0), sample_rate
# Return as mono tensor (channels, samples)
wav = torch.from_numpy(samples.reshape(1, -1))
return wav, sample_rate
# For non-WAV formats, use soundfile (optional dependency)
try:
import soundfile as sf
except ImportError as e:
raise ImportError(
"soundfile is required to read non-WAV audio files. "
"Install with: `pip install soundfile` or `uvx --with soundfile`"
) from e
data, sample_rate = sf.read(str(filepath), dtype="float32")
if data.ndim == 1:
wav = torch.from_numpy(data).unsqueeze(0)
else:
wav = torch.from_numpy(data.mean(axis=1)).unsqueeze(0)
return wav, sample_rate
class StreamingWAVWriter: