Code as Agent Harness
Paper

Code as Agent Harness

Tianxin Wei
2026.09.07
·Arxiv·by igor
#Agent Harness#Agentic AI#Code Generation#LLM#Software Engineering

Key Points

  • 1The paper introduces "code as agent harness," a conceptual framework that redefines code as an executable, inspectable, and stateful substrate for LLM-based agent systems rather than just a final output.
  • 2The authors organize this perspective into three layers: harness interfaces for reasoning and environment interaction, mechanisms for planning and tool use, and multi-agent scaling through shared code artifacts.
  • 3This unified roadmap addresses critical challenges in agent autonomy, including systematic verification, reliable long-horizon execution, and collaborative coordination in software-centric AI environments.

"Code as Agent Harness: Toward Executable, Verifiable, and Stateful Agent Systems" formalizes a paradigm shift in agentic AI, where code evolves from being a mere output artifact to serving as the operational substrate of the agent system. The paper introduces the concept of an "agent harness"—a software-defined layer that wraps LLMs with tools, sandboxes, and execution loops—to transform stateless models into reliable, long-horizon, and verifiable autonomous agents.

Core Conceptual Framework

The authors define the "code as agent harness" view through three interconnected layers that govern how code interacts with the agentic loop:

  1. Harness Interface (§2): This layer acts as the bridge between model reasoning and the task environment. It moves beyond traditional natural language chain-of-thought by externalizing computation into executable programs. This provides:
    • Executability: Translating internal logic into programs that external interpreters or symbolic solvers can run.
    • Inspectability: Exposing intermediate execution traces as structured, readable feedback.
    • Statefulness: Using persistent program variables and repository states to maintain context over long-horizon tasks.
  1. Harness Mechanisms (§3): This layer provides the infrastructure for reliability. Once code is integrated, the harness manages:
    • Planning: Utilizing structural decomposition, trajectory search, and workflow orchestration to manage complex software tasks.
    • Memory & Tool Use: Maintaining working, semantic, and experiential memory, while connecting agents to APIs, sandboxes, and repositories.
    • Feedback-Driven Control: Implementing closed-loop systems where runtime errors, static analysis, and test feedback drive iterative code repair.
  1. Scaling the Harness (§4): This layer extends the framework to multi-agent ecosystems. It defines the harness as a shared artifact—often centered around a common repository or structured execution environment—that enables coordination, review, testing, and red-teaming. Agents assume specialized roles (manager, coder, reviewer, tester) and synchronize progress through shared code states.

Core Methodology and Technical Mechanisms

The methodology focuses on "agent-initiated code artifacts," which are dynamically created and refined during the task execution loop. The core mechanism involves a recursive Plan-Execute-Verify cycle:

  • Program-Delegated Reasoning: Agents replace monolithic textual reasoning with modular code generation. By offloading logic to an interpreter, the system creates a "verification bridge" where, if an execution result rr from program PP violates a predicate V(r)V(r), the harness triggers an automated repair mechanism.
  • Harness Optimization: The survey highlights the transition toward "evolution agents." These agents treat the harness itself as a target for optimization, using deep telemetry—logs of previous failures, execution time, and state transitions—to mutate and improve their own tool-use policies or workflow logic.
  • Shared-Harness Synchronization: In multi-agent settings, the harness facilitates communication via shared memory objects and code repositories. Coordination is governed by formal protocols where agents propose patches or tests that are validated against a shared test suite TT. The system state is defined by a collection of persistent artifacts, and agents synchronize their internal representations by querying this shared state through the harness interface.

Open Challenges and Future Outlook

The paper identifies several critical hurdles for the field:
  • Oracle Adequacy: Moving beyond simple final task success to evaluating the quality of intermediate code artifacts.
  • Regression-Free Improvement: Ensuring that self-evolving agents do not degrade system stability as they modify their own code/harness logic.
  • Semantic Verification: Extending verification beyond syntax and execution errors to ensure that agent-generated programs align with complex human intent and safety-critical constraints.
  • Multimodal Integration: Bridging the gap between code-based symbolic control and raw perceptual input (e.g., in robotic or GUI/OS automation environments).

In summary, the paper argues that the future of agentic AI lies in "harness engineering," where the design of the execution environment is as critical as the LLM itself, providing a unified roadmap for creating robust, inspectable, and autonomous systems.