Solar Open 2 (250B-A15B) is an advanced, open-weight Large Language Model (LLM) developed by Upstage AI, architected as a Hybrid-Attention Mixture-of-Experts (MoE) system specifically engineered for agentic workflows. It incorporates 250 billion total parameters while maintaining a highly efficient inference profile, activating only 15 billion parameters per token.

Core Methodology and Architecture

The model’s efficiency is derived from its unique hybrid-attention stack, which replaces standard dense attention layers with a pattern that interleaves three linear-attention layers with one softmax-attention layer, repeating this structure 12 times across 48 total layers.

1. Linear-Attention and NoPE: By utilizing linear-attention mechanisms, the model eliminates the need for Rotary Positional Embeddings (RoPE). The linear-attention layers intrinsically encode token sequence order within their recurrent states, effectively removing the fixed RoPE extrapolation limit and enabling a context window of 1 million tokens.
2. Memory Management: The hybrid architecture significantly optimizes memory usage; only 12 of the 48 layers utilize a Key-Value (KV) cache. This reduction allows the model to manage long-context memory at approximately one-quarter of the memory footprint of a standard softmax-only model of equivalent size.
3. Mixture-of-Experts (MoE): The model employs a MoE design featuring 321 experts (320 routed, 1 shared). During inference, the top-8 routed experts plus the single shared expert are activated per token, ensuring high-capacity performance with significantly lower computational overhead.
4. Selective Weight Transfer: To accelerate convergence, Solar Open 2 was initialized using a selective weight transfer from its predecessor, Solar Open 1 (102B). Approximately 2.3% of the weights were carried over from the original architecture, while the remaining parameters were randomly initialized, optimizing the starting point for training at the 250B scale.

Technical Performance and Capabilities

Solar Open 2 is optimized for complex, multi-step reasoning and tool-calling, supported by a specialized chat template designed to preserve reasoning traces. Key performance benchmarks indicate state-of-the-art capability in agentic tasks, particularly in environments like SWE-Bench Verified (70.4%) and complex mathematical reasoning (AIME2026: 95.7%).

The model supports high-efficiency deployment via vLLM with expert-parallelism, using specific kernels (fla-core) for linear-attention optimization. It follows the OpenAI function-calling interface for tool use, facilitating integration into agentic frameworks like Claude Code and Hermes Agent through Model Context Protocol (MCP) compatibility.

Inference Parameters

The model supports distinct reasoning modes controlled by the reasoning_effort parameter:

• None: Provides a direct response.
• High: Activates a structured reasoning process, where the reasoning trace is stored in the reasoning_content field (or similar, depending on the implementation), with a reasoning block limit of 131,072 tokens within a maximum total token window of 256,000.