Kimi K3 is a 2.8 trillion-parameter, open-weights model designed for frontier-level intelligence, specifically targeting long-horizon coding, knowledge work, and complex reasoning. It features a 1-million-token context window and represents the first open model in the 3T-class. While performance remains slightly behind proprietary benchmarks like Claude Fable 5 and GPT 5.6 Sol, K3 demonstrated competitive or superior results in various internal and third-party evaluations, particularly in agentic workflows.

Core Methodology and Architecture

The model’s efficiency—approximately 2.5× better than its predecessor, Kimi K2—is driven by architectural innovations designed to optimize sequence information flow and depth-based representation retrieval:

• Kimi Delta Attention (KDA) and Attention Residuals (AttnRes): These architectural foundations improve how information is managed across long sequences and model depth. AttnRes selectively retrieves representations rather than uniformly accumulating them.
• Stable LatentMoE: Kimi K3 employs a sparse Mixture of Experts (MoE) architecture, activating 16 out of 896 experts. To stabilize routing, the model utilizes Quantile Balancing, which derives expert allocation directly from router-score quantiles, removing the need for heuristic updates or sensitive balancing hyperparameters.
• Optimization and Activation: The model utilizes Per-Head Muon to optimize attention heads independently, enhancing adaptive learning. Nonlinear activation control is managed via Sigmoid Tanh Unit (SiTU), while Gated MLA improves attention selectivity.
• Training and Scaling: Training involves quantization-aware methods (MXFP4 weights and MXFP8 activations). To maintain high throughput at massive expert-parallel scales, the team implemented a fully balanced expert-parallel training method using static shapes to eliminate host synchronization on the critical path.

Capabilities and Benchmarks

Kimi K3 excels in "agentic" tasks where the model must navigate complex, multi-step environments:

• Coding and Compiler Design: K3 can autonomously develop compilers from scratch (e.g., "MiniTriton") and perform GPU kernel optimization, rivaling human-engineered stacks.
• Vision-in-the-Loop: With native multimodal capabilities, K3 iterates between code generation and live visual feedback (screenshots) to refine 3D graphics, game development, and CAD workflows.
• Scientific Research: The model autonomously integrates literature review, numerical validation, and code implementation, as demonstrated in its reproduction of complex astrophysical universal relations.
• Agentic Knowledge Work: Through the "Widgets" and "Dashboard" interfaces, the model maintains persistent state and can execute recursive, long-running research tasks involving thousands of web/terminal fetches.

Deployment and Limitations

Kimi K3 is designed for deployment on supernode configurations with 64 or more accelerators to maximize high-bandwidth communication. It is natively integrated into Kimi Code, Kimi Work, and the Kimi API.

Current technical limitations include:

1. Sensitivity to Thinking History: K3 is trained in "preserved thinking history mode"; failure to pass historical context correctly (or switching models mid-session) leads to unstable generation.
2. Proactiveness: Due to its focus on long-horizon autonomy, K3 may exhibit excessive improvisation; users are advised to apply strict behavioral constraints in system prompts if task boundaries are rigid.
3. UX Gap: Despite its technical performance, the developers acknowledge a perceptible gap in the overall user experience compared to the leading proprietary models.