GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
Key Points
- 1GPTZero uncovered 100 confirmed hallucinated citations spanning over 51 accepted NeurIPS 2025 papers, despite NeurIPS having a policy against such content.
- 2This widespread issue is attributed to a massive 220% increase in submissions to top AI conferences, overwhelming the peer review pipeline and leading to oversight and negligence.
- 3The report highlights a critical vulnerability in academic peer review, revealing that many published papers contain fabricated references, some also showing signs of AI-generated text.
This report from GPTZero exposes a significant vulnerability in the peer review process of top-tier machine learning conferences, specifically focusing on hallucinated citations in accepted papers. Following an initial discovery of 50 such instances in ICLR 2026 papers, GPTZero scanned 4841 accepted papers from NeurIPS (Conference on Neural Information Processing Systems) and uncovered hundreds of hallucinated citations, explicitly confirming 100 across 51 NeurIPS papers that had already been accepted, presented, and effectively published.
The core problem identified is a "submission tsunami" driven by generative AI, paper mills, and intense publication pressure, which has strained conference review pipelines. NeurIPS submissions, for example, increased over 220% from 9,467 in 2020 to 21,575 in 2025. This surge necessitates recruiting a large volume of reviewers, leading to issues of oversight, misaligned expertise, negligence, and potential fraud. The report emphasizes that the system leaves academic reviewers, editors, and conference organizers "outnumbered and outgunned," facing challenges the traditional peer review model was not designed to withstand.
The methodology involved using GPTZero's "Hallucination Check" tool to scan accepted papers for "Sources" (hallucinated citations) and "AI" (AI-generated text), noting "* " for likely mixed content and "**" for likely fully AI-generated content. These findings are particularly concerning given that NeurIPS's LLM policy, similar to ICLR's, considers hallucinated citations grounds for a paper's rejection or revocation. Despite a 24.52% acceptance rate for NeurIPS 2025 main track papers, these papers containing hallucinations successfully passed the review process.
Examples of confirmed hallucinations demonstrate various forms of fabrication or inaccuracies:
- Fabricated Authors/Titles: Citations with non-existent authors or titles (e.g., "John Doe and Jane Smith" or titles that don't match any publication).
- Fake URLs/DOIs: Provided links or Digital Object Identifiers that are non-functional, lead to unrelated content, or are entirely fabricated.
- Incomplete Identifiers: ArXiv preprints cited with incomplete IDs (e.g., "arXiv:2305.XXXX").
- Mismatched Details: Authors and titles might exist, but publication details such as year, volume, issue, page numbers, or even the publisher are incorrect or do not correspond to the cited work (e.g., an article whose authors match, but the title, publisher, and page numbers are wrong; or a title matches, but the authors, date, and publisher are incorrect).
- ArXiv ID Discrepancies: An ArXiv ID is provided, but it links to a completely different article than the one described by the title and authors.
- Partial Matches: Some authors or parts of the title might vaguely resemble an existing paper, but a precise match is absent, or key identifying information is incorrect.
- Author Omissions/Additions: The cited paper exists, but the author list in the citation is incomplete (some authors omitted) or includes fabricated authors not on the original paper.
- Non-existent in Publication: The citation details (authors, title, venue) are provided, but no such paper exists within the specified conference proceedings, journal volume, or issue.