LLM Wiki is an open-source system designed for automated knowledge base compilation and maintenance, leveraging Large Language Models (LLMs) to structure and synthesize information from raw sources into a coherent, cross-referenced wiki. Its primary goal is to offload the "bookkeeping" burden of knowledge management from human users to an LLM, ensuring consistency, discoverability, and up-to-dateness of information.

The system operates on a three-tiered architecture:

1. Raw Sources: This immutable layer comprises the foundational data, such as articles, papers, notes, and transcripts. These sources serve as the ultimate ground truth, which the LLM reads from but never modifies. They are typically stored in a designated raw/ directory.
2. The Wiki: This is the LLM-generated knowledge base, consisting of markdown pages that include summaries, dedicated entity pages, and extensive cross-references. The LLM is the sole author of this layer, writing and updating its content based on the raw sources and defined schema.
3. The Schema: This configuration file acts as the instructional blueprint for the LLM. It defines the desired structure of the wiki, specifies content conventions (e.g., formatting, entity definitions), and dictates the workflows the LLM should execute upon data ingestion or other operations.

The core methodology of LLM Wiki is structured around three primary processes:

4. Ingest: When a new raw source is introduced into the system (e.g., by dropping a file into raw/), the LLM automatically initiates a comprehensive processing workflow. This involves:
   • Reading and Comprehension: The LLM reads and understands the content of the new source.
   • Summarization: It generates a concise summary of the new source.
   • Knowledge Integration: The LLM identifies relevant concepts and entities within the new source and updates corresponding pages across the wiki. This may involve modifying or creating multiple wiki pages (e.g., 10-15 pages for a single source) to ensure all related information is consolidated.
   • Contradiction Flagging: A critical function is the LLM's ability to identify and flag potential contradictions or inconsistencies between the newly ingested information and existing knowledge within the wiki. This proactive detection helps maintain the integrity and accuracy of the knowledge base.

5. Query: This process allows users to ask complex questions directly against the pre-compiled and synthesized wiki. Unlike systems that perform real-time retrieval from raw chunks, LLM Wiki's knowledge is already integrated and synthesized into a structured format. This pre-computation significantly enhances query performance and the quality of responses by drawing from a coherent, interconnected knowledge graph rather than disjointed raw data. Furthermore, insightful answers derived from query interactions can be subsequently filed back as new wiki pages, enabling a compounding effect on knowledge accumulation.

6. Lint: The Lint process involves periodic "health checks" of the entire wiki. During this phase, the LLM systematically reviews the wiki's content and structure to:
   • Identify Inconsistencies: Detect data points that conflict with each other within the wiki.
   • Flag Stale Claims: Identify information that may be outdated or no longer accurate based on the latest understanding or new sources.
   • Locate Orphan Pages: Find wiki pages that are not linked or referenced by any other pages, indicating potential isolation or irrelevance.
   • Check Missing Cross-references: Ensure proper interlinking between related concepts and entities, improving navigability and completeness.
   • Suggest Improvements: Based on the linting results, the LLM can suggest new questions for further exploration, highlight areas where more research might be needed, or recommend searching for additional raw sources to enrich the wiki.

The fundamental advantage of LLM Wiki, as highlighted by Andrej Karpathy, lies in its ability to automate the tedious and detail-oriented aspects of maintaining a knowledge base. LLMs excel at tasks requiring attention to detail, consistency, and cross-referencing across numerous documents, which are often error-prone and time-consuming for humans. By entrusting these "bookkeeping" operations to an LLM, the system transforms knowledge management into a more efficient, scalable, and less laborious process.