When AI builds itself
Key Points
- 1Anthropic reports significant internal progress in AI development, with AI systems now authoring over 80% of their codebase and substantially accelerating engineer productivity and research tasks.
- 2This trend suggests a future where AI development becomes largely automated, potentially leading to full recursive self-improvement where AI systems autonomously design and build their own successors.
- 3The authors highlight that this trajectory could bring immense benefits but also risks of humans losing control over AI, advocating for a globally coordinated and verifiable slowdown or pause in frontier AI development to allow for better societal adaptation and safety research.
The paper, "When AI builds itself: Our progress toward recursive self-improvement, and its implications," presents evidence that AI systems are already significantly accelerating the development of AI, leading towards a future where AI could autonomously design and develop its own successors, a concept termed recursive self-improvement (RSI). While RSI is not yet a reality and not inevitable, the paper suggests current trends indicate it could arrive sooner than anticipated, bringing both immense benefits and substantial risks related to human control.
The authors detail a progression in AI's role within Anthropic's development cycle: from early chatbot assistance (2023-2025) to coding agents writing and editing full files (2025-2026), and currently to autonomous agents that can run code and delegate tasks. The predicted future state (20XX?) involves agents capable of building and training models themselves, leading to continuous self-improvement of AI systems.
Evidence supporting this accelerating trend is drawn from both public benchmarks and internal Anthropic data. Publicly, the length of tasks AI models can reliably complete has been doubling roughly every four months. For example, Claude Opus 3 handled 4-minute tasks in March 2024, Claude Sonnet 3.7 handled 1.5-hour tasks a year later, and Claude Opus 4.6 managed 12-hour tasks by March 2026. If this trend holds, AI could handle tasks taking weeks by 2027. Benchmarks like SWE-bench (software engineering bug fixing) and CORE-Bench (reproducing research results) show rapid saturation, with models achieving near 100% performance in two years or fifteen months, respectively, from low starting points. METR also found Claude Mythos Preview could work for at least 16 hours on long-duration tasks.
Internally at Anthropic, the impact of AI on AI development is more direct. In engineering, Claude now handles underspecified problems, with humans providing the goal rather than the method. As of May 2026, over 80% of code merged into Anthropic’s codebase was authored by Claude, a significant increase from low single digits before February 2025. This has led to a remarkable increase in engineer productivity: in Q2 2026, the typical engineer merged 8 times as much code per day as in 2024. While "lines of code" is an imperfect metric, subjective impressions and the ability to tackle previously intractable cleanup tasks (e.g., 800 API error fixes in April 2026 that would have taken a human four years) corroborate substantial productivity gains. The quality of Claude-written code is also improving rapidly; its success rate on open-ended tasks reached 76% in May 2026 (up 50 percentage points in six months), and code quality, once inferior to human code in late 2025, is now roughly at parity and expected to surpass human quality within a year. Automated Claude code reviewers are also now catching about a third of the bugs that led to past incidents on claude.ai.
In research, Claude has become "superhuman" at optimizing steps within clearly defined experiments. For instance, in an experimental research loop to speed up small AI model training code, Claude Mythos Preview achieved a ~52x speedup by April 2026, compared to a ~3x speedup by Claude Opus 4 in May 2025. This far exceeds the ~4x speedup a skilled human researcher could achieve in 4-8 hours. More critically, Claude is showing nascent capabilities in proposing its own experiments and steering research sessions. In an April 2026 demonstration, Claude-powered agents designed and executed an entire open-ended AI safety research project, recovering 97% of a performance gap that human researchers recovered only 23% of over a similar timeframe. Additionally, in analyzing real Anthropic research sessions, the latest model (Mythos Preview, April 2026) made better "next step" decisions than human researchers 64% of the time, up from 51% in November 2025. This indicates an improving research judgment capacity in AI.
The paper posits that the human role in AI development is narrowing. Humans' comparative advantage is currently in "research taste" and judgment, such as choosing which problems to work on or which results to trust. However, the early evidence on Claude’s improving judgment suggests this too might become an AI capability. The "perspiration" aspect of research is becoming increasingly automated, accelerating progress.
The authors outline three possible future scenarios:
- The trend stalls, but today’s AI capabilities are widely diffused: This "S-curve" scenario suggests current capabilities might hit a bottleneck (e.g., architectural limitations, compute supply, exogenous shocks). Even so, today's AI would still bring major societal changes (e.g., cyber defense, 100-person companies doing work of 1,000). The paper considers this unlikely, given observed exponential growth across all measured capabilities.
- AI labs continue to see compounding efficiency gains: AI development becomes substantially automated, but humans retain control over research directions and judgment. Organizations using AI would become vastly more efficient (100-person companies acting like 10,000-person ones). This scenario is deemed likely, but raises concerns about bottlenecks shifting (e.g., human code review, managing an explosion of AI-generated ideas). Amdahl's Law is cited, where the overall pace is limited by the slowest component.
- AI systems themselves become capable of full recursive self-improvement: If current trends persist and AI develops transformative human-level ingenuity, AI could fully design and refine itself. Human roles would diminish, shifting to oversight and verification of an expanding "virtual lab" run by AI. This could revolutionize science and lead to rapid advancements, potentially bringing unprecedented benefits (e.g., in healthcare, robotics). However, it also presents the greatest risks regarding alignment and control, as compounded misalignment could lead to loss of control, and it's difficult to predict the societal and economic implications if human labor becomes uncompetitive.
Finally, the paper discusses "What should we do?" The authors believe a coordinated slowdown or temporary pause in frontier AI development would be beneficial to allow societal structures and alignment research to catch up. They propose the need for verifiable global coordination mechanisms, similar to arms control treaties, to ensure all parties comply and prevent "bad actors" from secretly advancing. A unilateral pause by one lab is deemed ineffective, as it would only shift leadership. The Anthropic Institute plans to research and build systems to enable such verifiable slowdowns or pauses, if other major developers agree.