View original
Evo$^2$: Evolving the Evolver through Weight-Harness Co-Evolution
Paper

Evo$^2$: Evolving the Evolver through Weight-Harness Co-Evolution

Chejian Xu
2026.10.15
·Arxiv·by Homin.Lee
#Agent#Co-evolution#LLM#Reinforcement Learning#Self-evolution

Key Points

  • 1Evo2 introduces a weight-harness co-evolution framework where a single model acts as both a Proposer and a Reflector to iteratively improve an agent's harness through feedback-driven modifications.
  • 2The system utilizes EvoTIDE, a training pipeline that leverages trajectory recovery, supervised fine-tuning, and interleaved reinforcement learning to train the model on the outcomes of its own harness evolution attempts.
  • 3Empirical results demonstrate that Evo2 significantly outperforms existing self-evolution methods on coding and memory benchmarks, while also showing superior transferability and effectiveness when applied to fixed proprietary models.

Evo2 is a framework for self-evolving agent systems that achieves weight-harness co-evolution. Existing approaches typically treat model weight updates (training on self-generated experience) and agent harness modifications (rewriting prompts, tools, and execution flows) as separate processes. Evo2 bridges these domains by training a unified policy to act as both a Proposer and a Reflector, where the outcomes of harness modifications provide a continuous stream of feedback to optimize the model’s weights.

The core methodology revolves around a tree-based harness search managed by a shared policy πθ\pi_\theta acting in two modes. The Proposer component generates Type A edits (independent, autonomous modifications) and Type B edits (directed interventions based on feedback). The Reflector component analyzes execution outcomes from the task executor MexecM_{exec} to provide revision, crossover, or retention decisions. Crossover searching allows the framework to integrate successful strategies across different branches of the harness tree by using a library S={sj}S = \{s_j\} of extracted mechanisms.

Weight evolution is facilitated by the EvoTIDE (Tree-guided Interleaved Dual-role Evolution) pipeline. To handle partially successful or invalid rollouts during data collection, the framework uses edit extraction and prefix continuation, restoring intermediate harnesses to resume generation. Training involves an initial supervised fine-tuning (SFT) phase followed by interleaved Group Relative Policy Optimization (GRPO). The training employs a dynamic parent pool PrP_r that maintains harnesses at various performance levels, sampled using probabilities:

pr(h)={ρ,h=hroot(1−ρ)w(h)∑u∈Nrw(u),h∈Nrp_r(h) = \begin{cases} \rho, & h = h_{root} \\ (1 - \rho) \frac{w(h)}{\sum_{u \in N_r} w(u)}, & h \in N_r \end{cases}

where w(h)=ϵp+∣V^(h)−1/2∣w(h) = \epsilon_p + | \hat{V}(h) - 1/2 | balances the exposure of both high-performing and underperforming candidates. Rewards for the Proposer (rexecPr^P_{exec}) and Reflector (rexecRr^R_{exec}) are defined by the change in task performance (V^\hat{V}) resulting from their respective modifications.

Experiments across coding benchmarks (Polyglot, SWE-bench Verified, LiveCodeBench) demonstrate that Evo2 significantly outperforms existing frameworks. For instance, on Polyglot, Evo2 achieved 82.1% pass@1 compared to the 62.6% of the strongest baseline, GEA. Furthermore, Evo2 shows strong transferability, with harnesses evolved on one benchmark performing robustly on others. The framework also demonstrates generalizability to non-coding tasks (e.g., LoCoMo and LongMemEval) and the ability to improve proprietary models with fixed weights, nearly doubling the performance of a GPT-5-mini agent from 24.4% to 48.3%.