Apodex Discovery: Reality Benchmarks and Environments for Evaluating and Building Discoverative Artificial Intelligence
Paper

Apodex Discovery: Reality Benchmarks and Environments for Evaluating and Building Discoverative Artificial Intelligence

Felix Liu
2026.09.02
·Arxiv·by JunhaRyu
#AI#Benchmark#Discoverative AI#Evaluation#Framework

Key Points

  • 1The authors introduce Apodex Discovery, a framework and reality-based benchmark (TRACES) designed to evaluate "heavy-duty solvers" that perform extended, verifiable, and stateful investigations on open-ended, real-world problems.
  • 2The framework employs an HDS6 process-verification metric to assess solvers across six critical capabilities—Tools, Repair, Alternatives, Coherence, Evidence, and Scope—independently of final success outcomes.
  • 3Experimental results demonstrate that Apodex Discovery outperforms state-of-the-art benchmarks in domains such as AAV capsid design and drug repurposing by enabling an iterative loop of hypothesis generation, tool usage, and verification-driven repair.

*Apodex Discovery* introduces a comprehensive framework and benchmark, named TRACES, for evaluating and building "discoverative AI"—AI systems capable of conducting sustained, stateful, and verifiable scientific or industrial investigations. The authors argue that current AI evaluation, typically focused on static question-answering benchmarks, fails to capture the multi-step, iterative process required for consequential real-world problem-solving.

Core Methodology: The Heavy-Duty Solver (HDS)

The framework defines a Heavy-Duty Solver (HDS) as an integrated system consisting of a foundation model, a task-specific harness, specialized tools, and a control policy. The methodology is built upon four pillars:

  1. Problem Manifests: Instead of static prompts, problems are defined by manifests that codify objectives, constraints, task decomposition, and success criteria. These allow complex ambitions (e.g., AAV capsid design) to be broken into sub-tasks with load-bearing intermediate artifacts.
  2. Reality-Based Environments: These are stateful, interactive substrates where the solver operates. They provide tools, data, and simulation interfaces that return fresh observations (e.g., experimental outcomes or execution errors) in response to solver actions, mirroring real-world research workflows.
  3. Verification Mechanisms:
    • Outcome Verification: Evaluates final submissions and intermediate results against hidden ground truth or high-fidelity proxies, employing anti-gaming gates to prevent data leakage.
    • Process Verification (HDS6): A novel metric evaluating the *how* of the investigation. HDS6 measures performance across six capabilities: Tools, Repair, Alternatives, Coherence, Evidence, and Scope. This allows for rigorous evaluation even when definitive ground truth is delayed or unavailable.
  4. Repair Loops: The framework utilizes a "measure-diagnose-improve" loop. Verification feedback (excluding ground truth) is returned to the solver, enabling it to localize failures, revise hypotheses, and refine its trajectory.

TRACES Benchmark and Empirical Results

The authors developed TRACES by scouting 561 industries to identify 423 high-value problems, 20 of which form the initial benchmark release. The evaluation approach is strictly modular; by holding the environment interface constant, the researchers can perform controlled ablations to isolate the contributions of specific solver components (e.g., the model vs. the harness).

Key findings include:

  • AAV Capsid Design: The Apodex solver surpassed the published state of the art by 7% across viability, tropism, structure, and generative design tasks.
  • Drug Repurposing: Integrating a domain-specific biomedical environment significantly boosted the performance of foundation models (e.g., GPT-5.5 and GPT-5.6-sol), improving normalized prediction scores by 2.5 and 7.6 points, respectively.
  • Process Efficiency: The use of structured, verification-driven repair loops demonstrated that systems could effectively improve their investigations upon receiving diagnostic feedback, providing a clear path to iterative refinement.

In summary, *Apodex Discovery* shifts the AI paradigm from solving pre-defined test questions to executing complex, grounded investigations, providing the infrastructure necessary for AI to act as a genuine agent of scientific and technical discovery.