The paper introduces Recursive Synthetic Terminal Tasks (RST), a framework designed to automate the large-scale generation of verified, long-horizon terminal-agent tasks. Addressing the high costs and scalability issues of human-authored datasets—which can cost thousands of dollars per task—the authors propose a recursive, solution-first approach that ensures task consistency through verified execution.

Core Methodology

The RST framework initiates from a set of 639 verified seed tasks and iterates through fifteen recursive rounds. Each round follows a structured pipeline:

1. Target Selection: The system selects a rewrite operator from a taxonomy of 40 primitives categorized into five families (e.g., Configuration and Control, Filesystem and Resources, Runtime and Diagnostics).
2. Rewrite: The generator extends the reference solution (solve.sh), updates the environment (Dockerfile), and aligns the verifier and public instruction to match the new, more complex workflow.
3. Validation: The framework employs a two-stage process:
   • Local Checks: Static analysis ensures the task is not a near-duplicate, lacks malicious leakage, and maintains metadata integrity.
   • Sandbox Oracle: The candidate task is executed in a fresh sandbox. A task is "oracle-valid" only if the reference solution passes the private verifier.
4. Reseeding: Validated tasks enter an accepted pool, which serves as both a source for the next synthesis round and a training set for supervised fine-tuning (SFT) and reinforcement learning (PPO).

The framework enforces Contract Validity, ensuring that all requirements checked by the private verifier are explicitly stated or inferable from the public instruction and workspace, preventing hidden requirements that would otherwise lead to agent failure.

Key Insights and Findings

• Scalability and Cost: The pipeline produced 37,484 verified tasks at a marginal cost of approximately $0.05 per task. The yield remained stable throughout 15 rounds, with the candidate pass rate oscillating between 74.5% and 81.5%, demonstrating no evidence of synthesis collapse.
• Increasing Difficulty: The complexity of the synthesized tasks grows significantly over time. From R1 to R15, the median solution length increases by $5.6\times$ (67 to 374 lines), and command usage increases by $6.1\times$ (40 to 244 commands). Conversely, the instruction length increases by only $1.4\times$, suggesting the tasks become richer in "hidden" complexity.
• Downstream Performance: The synthesized trajectories improve Qwen3.5-27B and 122B-A10B models on benchmarks such as Terminal-Bench 2 and Long-Horizon Terminal Bench. Supervised fine-tuning provides gains of up to 10 points, while PPO further enhances model performance (e.g., a 21.9% relative gain on Long-Horizon Terminal Bench).
• Diversity: Domain diversity is preserved throughout the 15 rounds. The effective number of domains remains stable (dropping only from 11.22 to 11.09), and domain concentration metrics show no signs of the system collapsing into a narrow subset of task types.
• Evidence of Long-Horizon Competence: The authors verify that the synthesized tasks are distinct from standard benchmarks using lexical distance metrics (Unigram JSD) and sliding-window overlap checks, confirming the model improvements arise from generalized capability development rather than data contamination.

In conclusion, RST provides a robust, low-cost mechanism for creating verified agent training data, showing that recursive synthesis can continue to scale in both quantity and difficulty without hitting an immediate performance ceiling.