Fish Audio Drama 3 is a large-scale, high-performance text-to-speech (TTS) model designed specifically for multi-character, long-form audio generation. With a total parameter count of 35B and 3B active parameters per token, the model utilizes a Mixture-of-Experts (MoE) text backbone combined with a hybrid attention mechanism (blending full and linear attention) to manage complex narrative structures and long-context dependencies.

Core Methodology and Technical Architecture

The model’s architecture centers on the following technical components:

1. MoE and Attention Backbone: The model employs a MoE-based transformer architecture. By using 3B active parameters, it balances high reasoning capability for dramatic content with computational efficiency. The hybrid attention mechanism is designed to handle the variable-length nature of long-form audio by combining full attention for precise local dependencies with linear attention for global context, optimizing performance across extended sequences.
2. Codebook-MTP Audio Decoder: Drama 3 utilizes a Codebook-Multi-Token Prediction (MTP) decoder. Unlike standard autoregressive models that may struggle with multi-codebook synchronization, this decoder predicts 10 codebooks causally across the entire audio sequence. This ensures coherent synthesis by maintaining structural integrity throughout the generation process.
3. Data Representation and Quantization: The model supports mixed-precision inference, utilizing $BF16$ for base weights and $FP8$ ($E4M3$) for the MoE experts, effectively reducing the memory footprint while maintaining synthesis quality.
4. Multi-Speaker Control: The model supports up to 16 distinct speaker embeddings per input sequence, triggered via speaker tags (e.g., $<|speaker:0|>$ to $<|speaker:15|>$). This facilitates complex, multi-character dialogue scenes within a single inference pass.

Inference and Production Optimization

Drama 3 is built for production environments using the *SGLang-Omni* framework, which optimizes streaming and throughput.

• Streaming Performance: On an NVIDIA H200, the model achieves a Time to First Audio (TTFA) of approximately 88.9 ms (p50) at a concurrency level of 1.
• Vocoder Placement: The system allows for flexible deployment configurations. Moving the vocoder to a dedicated GPU (separate from the engine’s GPU) at high concurrency (256 streams) provides a significant performance gain, increasing throughput by approximately 72% and reducing median TTFA by 37%.
• Audio Editing: The model supports a continuation-based editing paradigm. By providing previous_takes (the VQ tokens from earlier generations) as context, the model can refine or modify specific segments of dialogue, effectively treating audio generation as a continuous language modeling task.

Performance Summary

The model is optimized for high-throughput serving, maintaining an Real-Time Factor (RTF) of 0.80 even at a concurrency of 64. Throughput, measured in acoustic frames per second, scales significantly with hardware utilization, making it suitable for real-time applications requiring long-form, multi-character narrative synthesis.