ZONOS2 Technical Report

ZONOS2 8B is a high-performance text-to-speech (TTS) system designed to balance generation stability, naturalness, and inference efficiency. By utilizing a mixture-of-experts (MoE) transformer architecture, the authors scale the model to 8B total parameters (900M active) while maintaining the low latency required for real-time streaming. The model improves upon previous iterations by moving away from phoneme-based inputs to a byte-level tokenization approach, which enhances robustness across diverse languages and reduces errors caused by traditional grapheme-to-phoneme (G2P) pipelines.

The model uses a decoder-only transformer backbone that processes text and audio tokens autoregressively. To handle audio, it employs the Descript Audio Codec (DAC) with a novel delay pattern that allows for parallel generation of codebook tokens, significantly improving inference speed. Speaker cloning is achieved through zero-shot conditioning using ECAPA-TDNN embeddings, which are processed via a Linear Discriminant Analysis (LDA) transform to remove nuisance factors like background noise and recording duration. The training process involved a massive 6.2-million-hour corpus, curated through an ensemble of ASR systems to ensure high-quality transcript alignment.

The authors introduce ZTTS1-Eval, a new benchmark designed to address the limitations of existing TTS evaluations. Unlike previous benchmarks that focus primarily on English or Chinese, ZTTS1-Eval covers 9 read-speech languages and 17 in-the-wild languages, incorporating metrics for prosody, speaker similarity, and generation quality. ZONOS2 demonstrates competitive performance against current state-of-the-art systems, offering users fine-grained control over speaking rate and output quality through specialized conditioning tokens.