Ripple-65B Preview is a 65B parameter dense Transformer model developed by Ricursive Intelligence, designed to achieve state-of-the-art agentic and coding performance while maintaining high infrastructure efficiency.

Core Architecture and Methodology

Unlike the prevalent trend of using Mixture-of-Experts (MoE) architectures, Ricursive opted for a modernized, dense Transformer design. The architecture is characterized by a "thin-and-deep" aspect ratio, utilizing a narrow $d_{model}$ of 5120 across 80 layers. The design transitions from an Olmo 3 (LLaMA-like) framework to a Gemma 3N-like architecture to ensure training stability at scale.

Key technical enhancements include:

• Attention Optimization: The model overprovisions attention heads while adjusting the ratio of Sliding Window Attention (SWA) to global layers, alongside a reduced SWA window size to offset computational costs.
• Structural Refinements: The architecture incorporates gated attention, global NoPE (No Position Embedding), value residuals, and FFN activation clipping.
• Training Efficiency: The development team achieved a 40% Model Flops Utilization (MFU) through simple sharding without the need for custom kernels or quantization, resulting in a compute efficiency improvement of $16\times$ (up to $55\times$ at full scale) relative to the Olmo 3 ladder.

Training Pipeline

The model was trained on a 12T token corpus (predominantly English and code). The post-training stack involves a multi-stage approach:

1. Midtraining: 200B tokens (64K sequence length).
2. Long Context: 6.5B tokens (2M sequence length).
3. SFT: 183B tokens (128K sequence length).
4. LC SFT: 13.5B tokens (2M sequence length).
5. RL: Mixed-domain Reinforcement Learning.

Performance and Benchmarking

Ripple-65B Preview is positioned as an agentic-focused model. It achieved a score of 80.52 on TerminalBench 2.1, outperforming significantly larger frontier models, including the Reflection Beam 501B and Meta Muse Spark Open 816B. The model supports context lengths up to 1Mi (1,048,576 tokens).

The base model is characterized as a pretrain-only checkpoint that avoids synthetic data targeting in favor of high-quality natural data during pretraining, opting instead to utilize synthetic data during the midtraining phase. While traditional pretraining benchmarks show competitive results with models like DeepSeek V3.1, the model’s primary strength lies in its agentic reasoning capabilities, demonstrated by its strong performance in SWE-bench Pro v2.0 Hard (79.1) and its ability to fit on consumer-grade hardware like the NVIDIA DGX Spark.