A.X K2 is a large-scale, sparse Mixture-of-Experts (MoE) foundation model designed for agentic, multilingual, and long-context applications. With 688 billion total parameters and 33 billion active parameters per token, it utilizes a 256-expert routing architecture (plus one shared expert) with an 8-expert activation pattern. The model was developed as part of a Sovereign AI initiative, with a primary focus on deep Korean language understanding alongside English, Chinese, Japanese, and Spanish.

Core Methodology and Architectural Innovations

1. Hybrid Reasoning Control (Think-Fusion):
A.X K2 employs a "Think-Fusion" training recipe that unifies reasoning and non-reasoning modes within a single model. This is achieved via a paired supervised fine-tuning (SFT) dataset where each prompt includes both a multi-step reasoning trace—encapsulated by $<think>...</think>$ tokens—and a concise, low-latency alternative. This avoids category-specific artifacts, allowing users to toggle reasoning depth per request to balance accuracy and inference latency.

2. Sparse Gated Attention (SGA):
To optimize performance at the 256K context limit, A.X K2 integrates SGA with Multi-head Latent Attention (MLA).

• Sparse Indexing: The model uses a lightweight indexer to score and rank key-value (KV) candidates, selecting only the top-$k$ (where $k=2048$) tokens per query.
• Gated Attention: A head-specific output gate is applied to modulate attention outputs, suppressing attention sinks and improving signal calibration. This significantly reduces compute requirements for long sequences while maintaining retrieval accuracy.

3. Gated Norm (GN):
To address training instability and low-precision serving challenges, the model implements Gated Norm. A learned, input-dependent gate is applied after RMSNorm to suppress massive activations and hidden-state outliers. This single-path normalization simplifies the architecture compared to dual-normalization designs while producing activation distributions more amenable to aggressive quantization (FP8/NVFP4).

4. Native FP8 Training and Deployment:
A.X K2 was trained from scratch natively in MXFP8 (E4M3 format) for both forward and backward passes. The released checkpoints are distributed in block-scaled FP8, utilizing $128 \times 128$ weight blocks with dynamic activation scaling. This eliminates the need for post-hoc quantization, enabling the model to be served with half the memory footprint of BF16 without degrading performance.

Training and Performance

The pre-training process involved 8.2 trillion tokens arranged in a three-stage difficulty-increasing curriculum (General Knowledge $\rightarrow$ Reasoning $\rightarrow$ Long-Context). The model achieved state-of-the-art results on benchmarks such as AIME26 (97.1%), KMMLU-Pro (80.5%), and RULER, while maintaining perfect retrieval scores on the "needle-in-a-haystack" test even at a 256K context window. Post-training involved two stages of SFT followed by multi-task reinforcement learning focused on safety and agentic tool use, specifically utilizing an Evolved Safety Specification Alignment (ESSA) approach.