Ornith-1.5: From Self-Scaffolding to Self-Improvement
Key Points
- 1Ornith-1.5 introduces an advanced end-to-end self-improvement framework where models autonomously generate tasks, construct execution scaffolds, and refine solution rollouts via reinforcement learning.
- 2By utilizing a multi-factor reward system that balances task validity, frontier difficulty, and novelty, the model creates a continuous curriculum that evolves alongside its own increasing capabilities.
- 3The Ornith-1.5 model family demonstrates state-of-the-art performance across reasoning and agentic coding benchmarks, with the flagship 397B model rivaling Claude Opus and the compact 9B variant outperforming significantly larger existing models.
Ornith-1.5 is a foundation model series that implements a fully integrated, end-to-end self-improvement loop, advancing the self-scaffolding framework from Ornith-1.0. The architecture encompasses three distinct scales—397B MoE, 35B MoE, and a 9B dense model—designed to enhance reasoning, agentic, and coding capabilities. By autonomously proposing tasks, designing task-specific scaffolds, and executing solution rollouts, the model facilitates continuous learning through reinforcement learning.
Core Methodology: The Self-Improvement Loop
The Ornith-1.5 training cycle operates by optimizing the joint pipeline of task generation, harness construction, and solution generation. Unlike static datasets, the system employs a reward-driven approach to expand its curriculum based on capability gaps.#### Task Reward Function
The reward for a task is defined to ensure it is actionable and effectively pushes the model’s learning frontier. For a question , scaffold , and a set of rollouts , the task reward is:
- Validity (): A hard gate ensuring the scaffold is verifiable and functional. If , the task is discarded.
- Frontier Difficulty (): Rewards tasks where the empirical success rate is close to a target threshold , defined as:
- Novelty (): Promotes diversity by calculating the cosine similarity against a buffer of previously encountered tasks:
#### Harness and Rollout Optimization
The system evaluates harnesses via , focusing on task alignment (), reward fidelity (), and hack resistance (). Solutions are scored based on task success . The entire three-stage process is optimized using Group Relative Policy Optimization (GRPO).