The paper introduces Xiaomi MiMo-V2.5-Pro-UltraSpeed, a collaboration between Xiaomi and TileRT, which achieves a groundbreaking generation speed of over 1000 tokens per second (TPS) on a 1-trillion-parameter (1T) model using commodity GPUs. This unprecedented speed aims to transform AI applications by enabling real-time decision-making, enhancing reasoning capabilities through parallel computation (e.g., Best-of-N, Tree Search), accelerating coding agent productivity, and facilitating critical applications such as high-frequency trading, anti-fraud, and surgical assistance.

The core methodology relies on an "extreme model-system codesign" approach, eschewing specialized hardware in favor of deep optimization on standard commodity GPUs. This codesign encompasses two primary algorithmic innovations on the model side and a revolutionary execution model on the system side.

1. FP4 Quantization: To address the prohibitive memory footprint and bandwidth limitations of 1T models, the MiMo team employs virtually lossless FP4 (MXFP4) quantization. Recognizing the Mixture-of-Experts (MoE) architecture of MiMo-V2.5-Pro, where Experts constitute the vast majority of parameters and exhibit high tolerance to quantization, only these MoE Experts are selectively quantized to FP4. Other modules maintain their original precision. This selective quantization, combined with Quantization-Aware Training (QAT), dramatically reduces model size and maximizes hardware bandwidth utilization while preserving the model's overall capability.

2. DFlash Speculative Decoding: Traditional speculative decoding is bottlenecked by the draft model's quality, which influences acceptance rates, and the serial nature of token generation. DFlash, an innovative block-level masked parallel prediction method, fundamentally overcomes this limitation. The draft model predicts and fills an entire block of masked positions in a single forward pass, eliminating autoregressive drafting constraints. Custom optimizations for trillion-scale MoE and long-context scenarios were implemented:
   • The use of Muon second-order optimizer and model self-distillation ensures compact mask blocks yield ideal acceptance rates while minimizing draft-stage overhead.
   • The draft model exclusively utilizes Sliding Window Attention (SWA), aligning with the MiMo-V2 series design and reducing per-prediction compute from $\mathcal{O}(L)$ (context-length-linear) to $\mathcal{O}(1)$ (constant), where $L$ is the context length.
   • Mask-signal sampling is pushed to GPU-local shards during training, allowing a single sequence to generate tens of thousands of independent training signals across diverse context positions in one step, thereby supporting long contexts and avoiding cross-device communication overhead.

This parallel prediction enables high acceptance lengths (e.g., 6.30 tokens in coding scenarios, out of a block size of 8), meaning the large model confirms more content per verification round, leading to significant inference throughput gains.

3. TileRT Ultra-Low-Latency Inference Kernels/System: TileRT serves as the foundational infrastructure that extracts maximum physical potential from commodity GPUs by addressing the microsecond-level "Execution Gaps" inherent in traditional inference systems. TileRT introduces a "paradigm-level execution model revolution":
   • Persistent Engine Kernel: This discards the conventional per-operator launch paradigm. The entire compute pipeline remains persistently resident and flowing within the GPU, enabling full-pipeline continuous prefetching and achieving extreme overlap between data movement and computation.
   • Warp Specialization (Heterogeneous Pipeline Collaboration): At the Tile level, communication, data movement, and tensor computation are physically decomposed with finer granularity. Different Warps (thread groups) and heterogeneous execution domains across the GPU operate independently yet in precise coordination, transforming the GPU into a continuously flowing, precisely orchestrated heterogeneous execution system.

TileRT's system is tightly coupled with MiMo's algorithmic characteristics (FP4 quantization and DFlash speculative decoding), providing custom-built compilation engines and compute kernels. This deep hardware-software co-creation, involving profound joint engineering tradeoffs based on hardware physics, is critical to achieving the 1000+ TPS performance.

The paper emphasizes that this breakthrough is not a sum of point optimizations but rather the inevitable result of world-class system infrastructure and extreme algorithmic models deeply converging and co-evolving. The MiMo-V2.5-Pro-FP4-DFlash checkpoint, including FP4 quantized weights and DFlash model parameters, has been open-sourced on HuggingFace.