Summary: Retrieval as Reasoning (LLM Wiki)

Overview
The paper "Retrieval as Reasoning" challenges the prevailing Retrieval-Augmented Generation (RAG) paradigm, which relies on fragmenting documents into isolated chunks based on semantic similarity. The authors argue that this "chunk-and-retrieve" approach suffers from a fundamental limitation: the destruction of relational context, which hinders performance in complex, multi-hop reasoning tasks. Instead, the paper proposes "LLM Wiki," a system where knowledge is stored as a network of interlinked markdown pages, and retrieval is performed by an agent capable of traversing these links—effectively treating retrieval as a multi-step reasoning process.

Core Methodology
The LLM Wiki framework moves away from static, flat document chunks toward a dynamic, graph-based knowledge repository. The core mechanics are categorized into three pillars:

1. Structural Knowledge Representation: Rather than arbitrary chunking, documents are ingested and decomposed into entities and concepts represented as individual, interconnected pages. Each page serves as a semantically coherent node, with bidirectional links (hyperlinks) explicitly defining the relationships between concepts.
2. Agentic Traversal (Reasoning-based Retrieval): Unlike standard RAG that performs a one-shot retrieval, the LLM Wiki agent treats the retrieval process as an iterative navigation task. The agent utilizes a set of three fundamental tools:
   • Search: Identifies the entry point (the most relevant page(s)) using hybrid search (BM25 + vector similarity).
   • Read: Extracts information from the current page.
   • Link: Navigates to connected nodes to gather supporting evidence.

This iterative "traversal" allows the model to perform multi-hop reasoning by moving across the information graph until sufficient evidence for the query is synthesized.

3. System Maintenance and Evolution:
   • Error Book: An autonomous self-correction mechanism that identifies and resolves structural or semantic inconsistencies within the wiki.
   • Self-Evolution: The system improves its internal knowledge structure through repeated interaction, refining the graph as it is utilized over time.

Technical Findings and Ablation Study
The research validates this approach against standard retrieval models across benchmarks like HotpotQA, MuSiQue, and 2Wiki. The empirical results demonstrate significant performance gains:

• Performance: The LLM Wiki model consistently outperformed baselines by 2.0–8.1 in F1-score across all multi-hop tasks.
• Ablation Analysis: To determine the contribution of each component, the authors systematically removed modules, revealing the hierarchy of importance in maintaining performance:
  • Traversal: Removing the agent's ability to navigate links caused the most severe decline (F1 loss: $-11.7$ to $-13.8$ depending on the dataset).
  • Structure: Removing the pre-defined wiki structure led to a moderate decline (F1 loss: $-6.1$ to $-7.0$).
  • Error Book: Removing the self-correction mechanism caused the smallest decline (F1 loss: $-3.4$ to $-4.0$).

The conclusion emphasizes that "traversal"—the ability to actively navigate and synthesize information across multiple nodes—is approximately twice as critical to reasoning performance as the static structure of the knowledge base itself.

Implementation Perspective
While the original research defines the agent’s navigation during the "answer synthesis" phase, practical implementations (such as the llm-wiki-newsroom project) have introduced a "GROUND Ladder" (Guided Retrieval Over Unstructured Networked Data). This involves five distinct levels of reading depth—from referencing dependencies and indices to deep content crawling. This ladder allows the agent to intelligently decide whether to stay on a page or branch out into the network based on the "information sufficiency" signal detected during the reasoning loop, effectively moving the complexity from the retrieval phase into the knowledge ingestion and navigation logic.