If you are curious about harness engineering, stop the prompt and write a specification.
Key Points
- 1Uroboros introduces a "harness engineering" philosophy, advocating for detailed specification over iterative prompting to guide AI towards deterministic, precise outputs and prevent wasteful token generation.
- 2The system employs a Socratic interview process where the AI actively queries users to extract and clarify tacit knowledge, forming a detailed "seed" that directs subsequent task decomposition and iterative refinement.
- 3By prioritizing clear user intent, Uroboros aims to enhance AI productivity, leveraging concepts like Recursive Language Models (RLM) for contextual understanding and envisioning an "Agent OS" for efficient multi-agent workflow orchestration.
Ouroboros is a methodology and framework addressing the non-deterministic nature of Large Language Models (LLMs) by promoting "Harness Engineering" and "Specification-Driven Development" (Specifying) over traditional prompting. Its core philosophy is "Stop Prompting, Start Specifying," aiming to translate ambiguous human intent into precise, deterministic inputs for LLMs, thereby maximizing productivity and minimizing wasted tokens from iterative prompting.
The central problem identified is that despite LLMs' advanced capabilities, human-AI interactions often lead to undesired outputs due to the ambiguity inherent in human thought and language. Humans implicitly assume LLMs understand their "blueprints," leading to "drift" or "divergence" in outcomes and excessive token consumption. Previous Specification-Driven Development (SDD) approaches failed because LLMs, without proper "Harness Engineering," suffered from "lost in the middle" phenomena and context forgetting (primacy/recency bias), leading to over-engineering.
Ouroboros's core methodology revolves around a Socratic Interview process facilitated by a Master Control Program (MCP) agent. This process aims to:
- Uncover Tacit Knowledge and "Unknown Unknowns": The MCP, acting as a "pure" questioner (devoid of tools to prevent self-diversion), continuously asks probing questions to the user. This iterative questioning helps formalize the user's implicit knowledge, goals, and constraints, eliciting details the user might not initially realize are relevant.
- Generate a "Seed": The output of the Socratic Interview is a "seed" specification, which is a formalized document containing the clear
goal,constraints,acceptance_criteria(ACs), and identifiedambiguities. This seed is then "frozen" for execution. - Hierarchical Task Decomposition: The
seedis then fed to the Ouroboros engine, which decomposes the specification into a tree-like structure of atomic tasks. This process recursively applies the MECE (Mutually Exclusive, Collectively Exhaustive) principle to break down complex problems into manageable units (Depth-First Search - DFS tree). This atomic decomposition mitigates "lost in the middle" issues and biases, ensuring each task is solvable by an individual agent. - Execution and Evolution Loop (Life-loop):
- Execution (
run): Individual leaf-node tasks are executed by agents. Upon completion, results are reported back to parent nodes, triggering a bottom-up evaluation. In cases of conflict or failure, a coordinator agent intervenes to resolve issues or add further ACs. - Evaluation (
evaluate): A three-phase evaluation process is employed:- Deterministic Checks: Lints, builds, and automated tests are run to ensure basic functional correctness.
- Semantic Verification: An agent evaluates the artifacts against the initial
acceptance_criteriaand task summaries. - Consensus-Based Review: Multiple models collaboratively review the output, flagging any omissions or failures against the criteria. Failures can lead to re-attempts (QA mode) or trigger an evolution.
- Evolution (
evolve/reflect): If the generated outcome deviates from the intended goal (e.g., based on comparison between the initial seed and the output), a Socratic agent initiates a "wonder" phase, questioning what was missed. This triggers a reflection process to refine theontology(the shared understanding/knowledge base) and theseed. This reflection leads to a new "generation" or iteration of the life-loop. The loop continues until the similarity between successive generations' ontologies (e.g., >95% similarity) indicates convergence, or a maximum number of cycles (e.g., 30) is reached.
- Execution (
Technical Architecture & Concepts:
- MCP (Master Control Program): Functions as the primary orchestrator and Socratic interviewer. It can communicate with various LLM backends (e.g., Codex, Claude) through configurations (
ouroboros.config). This allows for dynamic backend selection. - Sub-Agents/Workers: These are specialized agents assigned to atomic tasks decomposed from the
seed. - Runtime Abstraction: Ouroboros abstracts the LLM runtime, allowing different models to be used interchangeably for different roles (e.g., Claude for main session, Codex for MCP calls). This also enables parallel sessions for MCP calls, where multiple sub-processes can operate without explicitly consuming the main session's context, maintaining a clean dashboard-like main session.
- MCP Mesh: Future development aims for MCPs to call other MCPs recursively, creating a mesh-like communication network for complex problem-solving.
- RLM (Recursive Language Model): A concept explored by Ouroboros where a large prompt is recursively broken down into smaller, manageable chunks for the model, improving accuracy and interpretability. This is likened to an OS-like approach where the LLM manages its memory and sub-tasks.
- Agent OS: The vision extends to an "Agent OS," an operating system where various "harnesses" (like Ouroboros, OMX, Hermes) can be "mounted" or utilized to handle specific actions or objectives, communicating via MCPs to achieve complex workflows.
- Nine Agents: Ouroboros employs nine distinct types of agents (e.g., Socratic Interviewer, Ontologist, Seed Architect, Evaluator, Contrarian, Hacker, Simplifier, Researcher, Architect) that perform specialized roles within the workflow.
The philosophy emphasizes that human intellectual effort is crucial in the initial specification phase, leading to more targeted and efficient LLM utilization, preventing aimless "slot machine" pulling for desired outputs, and fostering a collaborative, precise AI-human development loop.