The paper introduces Recursive Language Models (RLMs), a novel inference paradigm designed to enable Large Language Models (LLMs) to process arbitrarily long prompts, extending far beyond their inherent context window limitations. The core problem addressed is the "context rot" phenomenon, where LLM performance degrades significantly with increasing prompt length, even within the context window, and the inability of current LLMs to handle inputs requiring millions of tokens. Existing solutions like context compaction often lose crucial details, while conventional coding agents struggle with programmatically recursive calls and managing vast amounts of input data.

The central methodology of RLMs involves treating the long user prompt not as a direct input to the LLM's context window, but as an external object within a persistent environment. Specifically, an RLM operates around a base neural language model $M$ with maximum context size $K$. Given an arbitrary prompt $P \in \Sigma^\star$, the RLM initializes a Read-Eval-Print Loop (REPL) programming environment, which is the persistent external environment $E$. Inside this REPL, $P$ is loaded as the value of a variable, providing a symbolic handle to the prompt.

The operational flow of an RLM is detailed in Algorithm 1:

1. Initialization: The REPL environment state is initialized using $InitREPL(prompt=P)$. A function sub_RLMM for invoking sub-RLMs (or sub-LMs) is added to this state. The initial hist (history for the LLM) is populated with Metadata(state), which provides the LLM with constant-size information about the prompt (e.g., length, a short prefix, access methods).
2. Iterative Loop: The process enters a loop where the LLM (LLMM) is invoked with the current hist to generate code. This code is then executed within the REPL environment ((state, stdout) ← REPL(state, code)). The state variable tracks intermediate values and the overall environment. Any stdout from the code execution is collected, and only its Metadata (e.g., short prefix, length) is appended to hist along with the executed code. This mechanism is crucial as it prevents the LLM's hist from overflowing with large outputs from the REPL, forcing the LLM to manage large strings symbolically within REPL variables.
3. Recursion and Symbolic Manipulation: The LLM is prompted (or fine-tuned) to generate code that manipulates the prompt $P$ and other variables in the REPL. Crucially, it can programmatically invoke sub_RLMM on slices or transformations of $P$, allowing for arbitrary recursion depth. This enables the LLM to perform $\Omega(|P|)$ or even $\Omega(|P|^2)$ semantic work by iterating through and processing parts of $P$ using sub-calls within loops, storing intermediate results symbolically in REPL variables.
4. Termination: The loop continues until the LLM sets a specific variable named Final within the REPL environment. The value of state[Final] is then returned as the final response $Y \in \Sigma^\star$.

The paper contrasts this robust design with a "deceptively similar" flawed Algorithm 2, highlighting three key design choices that make RLMs superior:

• Prompt Handling: Unlike Algorithm 2, which naively puts the entire prompt $P$ into the LLM's context window (hist), thereby inheriting its $K$ token limit, RLM uses a symbolic handle to $P$ in the REPL.
• Output Generation: Algorithm 2 requires the LLM to directly generate the final output via a Finish action, limiting output length to the context window size. RLM accumulates the response in a REPL variable (state[Final]), allowing for unbounded output length.
• Symbolic Recursion: Algorithm 2 might offer Exec (code execution) and sub_LLMM actions, but without programmatic control over sub-calls within loops or the ability to manipulate P symbolically, it cannot achieve the arbitrary recursion and work complexity of an RLM. RLMs allow the LLM to write short programs that loop over slices of the prompt and launch $\Omega(|P|)$ or $\Omega(|P|^2)$ processes.

The evaluation demonstrates RLM's efficacy using state-of-the-art models like GPT-5 and Qwen3-Coder-480B-A35B across diverse long-context tasks: S-NIAH (constant complexity), BrowseComp-Plus (constant docs, multi-hop QA), OOLONG (linear complexity, semantic aggregation), OOLONG-Pairs (quadratic complexity, pairwise aggregation), LongBench-v2 CodeQA (code understanding), and LongCoT-mini (compositional reasoning).

Key findings include:

• RLMs successfully scale to 10M+ token inputs, outperforming base models and conventional scaffolds (compaction, CodeAct, OpenCode) often by double-digit percentage points, while maintaining comparable or lower average costs.
• RLMs exhibit significantly less performance degradation with increasing input length and task complexity compared to vanilla LLMs, especially on information-dense tasks like OOLONG and OOLONG-Pairs, where they achieve substantial F1 score improvements (e.g., GPT-5 RLM achieves 58.0% on OOLONG-Pairs vs. 0.1% for base GPT-5).
• The REPL environment for externalizing context is crucial, and recursive sub-calling capability is vital for information-dense tasks. While RLM (recursion depth=0) and some coding agents can handle large inputs by offloading context, true recursion is necessary for complex semantic work.
• RLMs enable advanced reasoning capabilities beyond just long contexts. On LongCoT-mini, RLM(GPT-5.2) significantly outperforms the base model, especially with decomposition hints, demonstrating its ability to programmatically traverse complex reasoning graphs via sub-calls.
• Fine-tuning small models (e.g., RLM-Qwen3-8B) on RLM trajectories from larger models can drastically improve their performance and efficiency as RLMs, exhibiting length generalization even when trained on unrelated domains.