RAG Collapse: LLM Responses Collapse When Retrieved Documents Are Self-Authored

arXiv:2608.221182026-08-25

When AI search tools keep citing the AI's own past writing, its answers eventually all converge on one version

When AI systems like ChatGPT use web search to pull references and some of those references are articles the AI itself previously wrote, the answers gradually become more alike until they're essentially all the same. The authors call this 'RAG collapse' and found it happened in 79.6% of 1,528 simulations across three model families. Strikingly, even a single self-authored reference was enough to trigger this collapse.

METAL LAB explanatory visual

How RAG collapse unfolds across simulation rounds

Evidence statusMeasured results reported

Step 1: Answer with original referencesThe AI answers the same question ten times using references pulled from ChatGPT or Google AI Overviews, producing varied answers
Step 2: Turn answers into articlesSome of the AI's answers are rewritten into article-style text, creating 'self-authored' references
Step 3: Replace or compete referencesReplace All swaps every reference each round, Replace One swaps just one, and Search lets original and self-authored references compete via retrieval
Step 4: Repeat over roundsThis cycle repeats for up to 10-30 rounds, regenerating answers each time
Step 5: Check for collapseMentioned entities and wording become increasingly similar, ending in collapse (identical entities across all answers) in 79.6% of simulations

An explanatory diagram made by METAL LAB, not a figure supplied by the paper's authors.

What they did

The researchers built three simulation designs (Replace All, Replace One, Search) using GPT, Gemini, and Claude model families and 1,019 information-seeking prompts to mimic AI systems retrieving content they had previously generated themselves.
In each round, the AI's answers were rewritten as article-style text and placed into a pool of references, then the original references were replaced one at a time, all at once, or made to compete via search retrieval, before regenerating answers again.
1,216 of 1,528 simulations (79.6%) ended in collapse, and in one experiment the share of response pairs judged to be paraphrases of each other rose from 22% at the start to 89% by the end.
Even when only one reference was self-authored, collapse could still begin, because the model disproportionately cited its own writing over original references even after controlling for reference quality — a form of self-bias.
The team also found that 38.9% of ChatGPT's cited references in January 2026 and 42.7% in June 2026 were estimated to be AI-generated, showing the feedback loop conditions already exist to some degree.

Figure 1: Two responses to the prompt “Who are the best Twitch streamers currently?” from GPT-5.2 that differ in structure and streamers mentioned due to random sampling.

Figure 2: Entity visibility for “Who are the best Twitch streamers currently?” at round 1. “Kai Cenat” appears in all ten, while “Squeezie” appears in only two of ten.

Table 1: Dataset statistics.
Name	Questions
Entity ChatGPT	843
Editorial ChatGPT	159
Entity AI Overview	60
Editorial AI Overview	45

Figure 3: Entity visibility for “Who are the best Twitch streamers currently?” at round 5. Every streamer now has either 0% or 100% visibility. The distribution has collapsed.

Figure 4: Overview of Replace All. In each round, every reference is replaced by a self-authored reference generated from a response in the previous round.

Table 2: Estimated percentage of ChatGPT references classified as AI-generated, overall and by prompt type. Brackets show 95% Wilson confidence intervals over unique reference URLs.
	Overall	Entity	Editorial
1/26 ChatGPT	38.9% [38.0, 39.9]	41.1% [40.1, 42.1]	20.6% [18.3, 23.0]
6/26 ChatGPT	42.7% [41.7, 43.7]	45.8% [44.7, 46.8]	27.5% [25.4, 29.7]

Figure 5: Overview of Replace One. Original references are replaced more slowly than in Replace All, although the design still assumes that self-authored references are retrieved.

Figure 6: Overview of the Search simulation. Self-authored references are added to a reference pool and retrieved using search, competing with the original references for retrieval.

Table 3: Summary of experiments and collapse rates. The distinction between GPT-5.2 and GPT-5.2 Chat is explained in Section 5.
Simulation	Model	Dataset	Questions	Collapsed at Start	Collapsed at End	Rounds Collapsed
Replace All	GPT-5.2 Chat	Entity ChatGPT	101	2.97%	88.12%	68.51%
Replace All	GPT-5.2 Chat	Editorial ChatGPT	57	29.82%	91.23%	79.65%
Replace One	GPT-5.2 Chat	Entity ChatGPT	101	2.97%	88.12%	67.18%
Replace One	GPT-5.2 Chat	Editorial ChatGPT	57	21.05%	94.74%	81.67%
Search	GPT-5.2 Chat	Entity ChatGPT	101	1.98%	77.23%	62.31%
Search	GPT-5.2 Chat	Editorial ChatGPT	57	31.58%	75.44%	73.63%
Replace One	Gemini 3 Pro	Entity AI Overview	60	3.33%	80.00%	43.83%
Replace One	Claude Sonnet 4.5	Entity AI Overview	60	1.67%	91.67%	64.33%
Replace One	Gemini 3 Pro	Editorial AI Overview	45	6.67%	68.89%	42.56%
Replace One	Claude Sonnet 4.5	Editorial AI Overview	45	11.11%	97.78%	69.33%
Search	GPT-5.2	Entity ChatGPT	742	7.01%	73.85%	59.93%
Search	GPT-5.2	Editorial ChatGPT	102	7.84%	83.33%	65.85%

Figure 7: Final-round entity visibility across nine independent runs for “Who are the best Twitch streamers currently?” Different entities survive in each run, illustrating run-to-run variance.

Figure 8: Collapse rates across simulation designs using GPT-5.2 Chat.

Table 4: Correlations between initial-response statistics and eventual collapse in entity-question Search simulations.
Name	Pearson Correlation	p
unique entities	−0.342	1.32×10−24
length of responses	−0.245	5.31×10−13
unique words	−0.223	6.21×10−11
entity ranking similarity	0.164	1.61×10−6
same-answer %	0.151	1.01×10−5

(b) Collapse by simulation, editorial questions.

(c) Questions collapsed by round, entity questions.

Findings

Across all three simulation designs, unique words and unique entities mentioned in responses decreased over rounds while semantic similarity and entity-ranking similarity between responses increased.
1,216 of 1,528 simulations (79.6%) ended in collapse.
In one experiment, the share of response pairs judged to be paraphrases rose from 22% initially to 89% by the end of the simulation.
When both original and self-authored references were present, models cited self-authored references disproportionately more, even after controlling for reference quality.
An estimated 38.9% of ChatGPT's cited references in January 2026, and 42.7% in June 2026, were classified as AI-generated.

(d) Questions collapsed by round, editorial questions.

Figure 9: Core metrics for entity questions across simulation designs. The numbers of unique words and entities decrease over rounds, indicating declining response diversity, while semantic and entity-ranking similarities increase.

Where it can be used

Teams building search-grounded AI systems could use this as a reason to examine or filter how much AI-generated content ends up in their retrieval pools.
Marketers or content strategists doing generative-engine optimization might factor in that AI systems disproportionately favor citing AI-style or self-similar content when planning what content to publish.
Teams monitoring AI answer quality could adopt metrics like entity visibility, unique word counts, and paraphrase rate across responses to track whether answers are becoming less diverse over time.

Limits and open work

The original 'seed' references already contained an estimated 38.9-42.7% AI-generated content that could not be definitively identified as self-authored, meaning the measured collapse and self-bias are likely conservative estimates rather than overstatements.
The experiments only tested repeatedly introducing the same model's self-authored references, not other types of AI-generated content added iteratively, which the authors note could also cause collapse.
The study focuses on information-seeking prompts, especially entity-comparison questions, so collapse dynamics on other prompt types remain untested.
The simulations demonstrate that collapse can occur under these conditions but do not prove that RAG collapse is already happening inside real commercial AI systems, which may already have undisclosed mitigations in place.
Future work still needed includes testing collapse under agentic search, evaluating whether proposed mitigations actually work, and studying cross-model retrieval of each other's self-authored content.

Why it matters

People assume search-grounded AI answers reflect the internet's diversity of opinion, but this work shows that when the AI keeps retrieving and citing its own past output, that diversity can quietly disappear and answers settle on a single, possibly arbitrary, version. This matters for anyone building retrieval-based AI systems or relying on AI-generated content strategies (like generative-engine optimization), since the feedback loop could already be forming as more AI content fills the web.

Terms in this paper

RAG (retrieval-augmented generation) · having an AI model search the web and use retrieved documents to help write its answer
model collapse · a known phenomenon where AI trained repeatedly on its own outputs produces less and less diverse, eventually distorted, results
self-authored reference · a document the same AI model generated from its own earlier answer, later retrieved and cited as if it were an independent source
self-bias · the tendency of a model to favor its own writing over other content of equal quality
entity visibility · the percentage of generated answers that mention a specific named entity, like a particular streamer or restaurant

Original abstract (English)

LLM responses are based on the internet (via training or RAG), and AI is now used to generate a significant amount of content online (Paredes et al., 2026), creating the potential for a self-reinforcing feedback loop. Prior work has shown that when LLMs are recursively trained on their own output, they experience model collapse (Shumailov et al., 2024): responses become less diverse, and eventually no longer resemble the original training data. In this paper, we show that a similar collapse occurs if LLM-based AI systems retrieve references they authored using a search tool. We call this RAG collapse. We conduct extensive experiments with three types of simulations of AI systems retrieving references they generated, using three model families, and 1,019 information-seeking prompts, totaling 1,528 simulations and over one million LLM API calls, and find that 79.6% (1,216/1,528) of simulations end in collapse. Surprisingly, even a single self-authored reference can trigger collapse because the LLM disproportionately cites its own content. This self-bias persists even after controlling for reference quality.

Authors · Gregory Druck, Ethan Smith

Read on arXiv

Latest papers

All papers →

Latest from METAL LAB

Figures: Gregory Druck et al., arXiv:2608.22118, CC BY 4.0