KGGen is a novel text-to-knowledge-graph (KG) generator that addresses the scarcity and incompleteness of high-quality KGs by leveraging language models (LMs) and a sophisticated entity and edge resolution approach. Existing methods like OpenIE and Microsoft's GraphRAG often produce sparse and disconnected KGs due to a lack of effective entity resolution and relation normalization. KGGen aims to create dense, high-quality KGs by clustering and de-duplicating related entities and normalizing relation types. The paper also introduces MINE (Measure of Information in Nodes and Edges), the first benchmark specifically designed to evaluate KG extractors from plain text.

The core methodology of KGGen is a multi-stage approach:

1. Entity and Relation Extraction:

This initial stage processes unstructured text to produce a raw set of subject-predicate-object triples. KGGen utilizes a large language model, such as Google's Gemini 2.0 Flash (or other state-of-the-art LLMs like Claude Sonnet 3.5 or GPT-4o), to perform the extraction. The process is structured into two distinct steps, guided by DSPy signatures to ensure structured output and consistency:

• Step 1 (Entity Extraction): Given the source text, the LLM is prompted to extract a list of relevant entities.
• Step 2 (Relation Extraction): Using the source text and the previously extracted list of entities, the LLM is prompted to output a list of subject-predicate-object relations, ensuring that subjects and objects are consistent with the extracted entities.

Strong prompting constraints are imposed to prevent the LLM from conflating similar but distinct entities or relations (e.g., "Type 1 diabetes" vs. "Type 2 diabetes").

2. Aggregation:

After triples are extracted from individual source texts, all unique entities and edges across all generated knowledge graphs are collected and combined into a single, aggregated graph. All entities and edges are normalized to lowercase to reduce superficial variations. This step primarily reduces redundancy and does not involve an LLM.

3. Entity and Edge Resolution:

This is KGGen's key innovation, designed to merge duplicate or synonymous entities and consolidate redundant edges, thereby preventing the formation of sparse KGs and improving graph density. This stage employs an iterative, two-stage approach combining embedding-based clustering with LLM-based de-duplication, applied separately to both entities and edges:

• Semantic Embedding: All items (entities or edges) in the aggregated graph are first converted into semantic embeddings. The paper specifically mentions using all-MiniLM-L6-v2 from SentenceTransformers for this purpose.
• Clustering: K-means clustering is applied to these semantic embeddings, grouping similar items into clusters (e.g., clusters of 128 items). This pre-processing step efficiently handles large KGs by partitioning the resolution task.
• Similar Item Retrieval: Within each cluster, for every item, the top-k (e.g., $k=16$) most semantically similar items are retrieved. This retrieval leverages a fused approach combining BM25 (for lexical similarity) and semantic embedding similarity. The combined similarity score $S$ for a query $Q$ and an item $I$ can be represented as a weighted sum:

$S(Q, I) = \alpha \cdot \text{BM25\_Score}(Q, I) + (1-\alpha) \cdot \text{Cosine\_Similarity}(\text{Emb}(Q), \text{Emb}(I))$
where $\alpha$ is typically 0.5 for equal weighting.

• LLM-based De-duplication: The retrieved set of similar items is then passed to an LLM. The LLM is prompted to identify *exact duplicates* from this set, specifically considering variations in tense, plurality, case, abbreviations, and shorthand forms (e.g., "Olympic Winter Games", "Winter Olympics", "winter Olympic games"). This step leverages the LLM's understanding of natural language nuances for precise matching.
• Canonical Representative Selection: For each identified set of duplicates, the LLM selects a single, canonical representative (similar to aliases in Wikidata) that best captures the shared meaning. Cluster maps are maintained to track which original entities or edges are mapped to which canonical alias.
• Iteration: The item and its identified duplicates are removed from the current cluster, and the process (retrieval, LLM de-duplication, canonical selection) repeats until no items remain in the cluster. This iterative refinement ensures comprehensive de-duplication.

KGGen's performance is evaluated using the novel MINE benchmark, which includes:

• MINE-1 (Knowledge Retention): Measures the ability of a KG extractor to capture information from short texts. It assesses if 15 pre-defined, ground-truth facts from an article can be recovered from the extracted KG via a semantic query process and LLM-based verification.
• MINE-2 (KG-Assisted RAG Description): Evaluates downstream RAG performance using KGs built from multi-million token datasets (WikiQA). It assesses how well the KG facilitates answer synthesis for given questions, using retrieved triples and an LLM to generate responses.

The paper demonstrates that KGGen significantly outperforms GraphRAG and OpenIE in terms of information retention on MINE-1, achieves comparable RAG performance on MINE-2, and critically, produces KGs with more concise, generalizable entities and relations. Qualitatively, KGGen generates KGs that are more informative and coherent, avoiding the hyperspecific or generic nodes often found in outputs from other tools. Furthermore, KGGen exhibits superior scaling properties, with the average re-usability of relation types increasing with corpus size, unlike GraphRAG, which generates nearly as many relation types as edges even for large corpora. Efficiency analysis shows KGGen to be significantly faster and more cost-effective for large corpora compared to GraphRAG, primarily due to its effective de-duplication reducing the number of LLM calls in the later stages.