工作日早上 7 点读 AI，周日早上 8 点读周报订阅邮件

METAL LAB

RAG Collapse: LLM Responses Collapse When Retrieved Documents Are Self-Authored

arXiv:2608.221182026-08-25

AI如果不断检索并引用自己以前写的内容,答案最终会趋同成一个版本

当像ChatGPT这样的AI系统通过网络搜索获取参考资料来回答问题时,如果这些参考资料中混入了AI自己以前生成的文章,答案会逐渐变得越来越相似,最终几乎完全一致。研究者将这种现象称为RAG崩溃,在涉及三个模型家族的1,528次模拟中,79.6%最终发生了崩溃。令人意外的是,哪怕只有一条自撰参考资料,也足以引发崩溃。

METAL LAB 解读图

RAG崩溃如何在多轮循环中逐步发生

证据状态已报告实测结果

  1. 第一步:用原始参考资料生成回答AI基于从ChatGPT或谷歌AI Overview获取的原始资料,对同一问题连续回答十次,得到多样的答案
  2. 第二步:把回答改写成文章把部分AI回答改写成在线文章的形式,制成自撰参考资料
  3. 第三步:替换或竞争参考资料Replace All每轮整体替换,Replace One每轮替换一条,Search让原始资料与自撰资料在检索中相互竞争
  4. 第四步:反复多轮循环这一过程重复最多10到30轮,每轮都重新生成答案
  5. 第五步:检验是否崩溃回答中提到的实体和措辞越来越相似,最终在79.6%的模拟中所有回答都只提到相同的实体,判定为崩溃
这是 METAL LAB 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. 研究团队用GPT、Gemini、Claude三个模型家族和1,019个信息检索类问题,设计了三种模拟方式(Replace All、Replace One、Search),模拟AI系统检索到自己以前生成内容的情形。
  2. 每一轮中,研究者把AI的回答改写成文章形式放入参考资料池,再逐一替换、整体替换或让其与原始资料在检索中竞争,之后重新生成答案,如此反复多轮。
  3. 1,528次模拟中有1,216次(79.6%)最终发生崩溃,其中一项实验里,被判定为互为改写(说的是同一件事)的回答对比例,从最初的22%升到模拟结束时的89%。
  4. 即便只有一条参考资料是AI自己写的,也可能引发崩溃,原因是即使控制了资料质量,模型仍会不成比例地更多引用自己撰写的内容,这是一种自我偏好(self-bias)。
  5. 研究还发现,截至2026年1月ChatGPT引用的参考资料中约38.9%、到2026年6月约42.7%被判定为AI生成,说明这种反馈循环的条件在现实中已部分具备。
Figure 1: Two responses to the prompt “Who are the best Twitch streamers currently?” from GPT-5.2 that differ in structure and streamers mentioned due to random sampling.
Figure 1: Two responses to the prompt “Who are the best Twitch streamers currently?” from GPT-5.2 that differ in structure and streamers mentioned due to random sampling.
Figure 2: Entity visibility for “Who are the best Twitch streamers currently?” at round 1. “Kai Cenat” appears in all ten, while “Squeezie” appears in only two of ten.
Figure 2: Entity visibility for “Who are the best Twitch streamers currently?” at round 1. “Kai Cenat” appears in all ten, while “Squeezie” appears in only two of ten.
Table 1: Dataset statistics.
NameQuestions
Entity ChatGPT843
Editorial ChatGPT159
Entity AI Overview60
Editorial AI Overview45
Figure 3: Entity visibility for “Who are the best Twitch streamers currently?” at round 5. Every streamer now has either 0% or 100% visibility. The distribution has collapsed.
Figure 3: Entity visibility for “Who are the best Twitch streamers currently?” at round 5. Every streamer now has either 0% or 100% visibility. The distribution has collapsed.
Figure 4: Overview of Replace All. In each round, every reference is replaced by a self-authored reference generated from a response in the previous round.
Figure 4: Overview of Replace All. In each round, every reference is replaced by a self-authored reference generated from a response in the previous round.
Table 2: Estimated percentage of ChatGPT references classified as AI-generated, overall and by prompt type. Brackets show 95% Wilson confidence intervals over unique reference URLs.
OverallEntityEditorial
1/26 ChatGPT38.9% [38.0, 39.9]41.1% [40.1, 42.1]20.6% [18.3, 23.0]
6/26 ChatGPT42.7% [41.7, 43.7]45.8% [44.7, 46.8]27.5% [25.4, 29.7]
Figure 5: Overview of Replace One. Original references are replaced more slowly than in Replace All, although the design still assumes that self-authored references are retrieved.
Figure 5: Overview of Replace One. Original references are replaced more slowly than in Replace All, although the design still assumes that self-authored references are retrieved.
Figure 6: Overview of the Search simulation. Self-authored references are added to a reference pool and retrieved using search, competing with the original references for retrieval.
Figure 6: Overview of the Search simulation. Self-authored references are added to a reference pool and retrieved using search, competing with the original references for retrieval.
Table 3: Summary of experiments and collapse rates. The distinction between GPT-5.2 and GPT-5.2 Chat is explained in Section 5.
SimulationModelDatasetQuestionsCollapsed at StartCollapsed at EndRounds Collapsed
Replace AllGPT-5.2 ChatEntity ChatGPT1012.97%88.12%68.51%
Replace AllGPT-5.2 ChatEditorial ChatGPT5729.82%91.23%79.65%
Replace OneGPT-5.2 ChatEntity ChatGPT1012.97%88.12%67.18%
Replace OneGPT-5.2 ChatEditorial ChatGPT5721.05%94.74%81.67%
SearchGPT-5.2 ChatEntity ChatGPT1011.98%77.23%62.31%
SearchGPT-5.2 ChatEditorial ChatGPT5731.58%75.44%73.63%
Replace OneGemini 3 ProEntity AI Overview603.33%80.00%43.83%
Replace OneClaude Sonnet 4.5Entity AI Overview601.67%91.67%64.33%
Replace OneGemini 3 ProEditorial AI Overview456.67%68.89%42.56%
Replace OneClaude Sonnet 4.5Editorial AI Overview4511.11%97.78%69.33%
SearchGPT-5.2Entity ChatGPT7427.01%73.85%59.93%
SearchGPT-5.2Editorial ChatGPT1027.84%83.33%65.85%
Figure 7: Final-round entity visibility across nine independent runs for “Who are the best Twitch streamers currently?” Different entities survive in each run, illustrating run-to-run variance.
Figure 7: Final-round entity visibility across nine independent runs for “Who are the best Twitch streamers currently?” Different entities survive in each run, illustrating run-to-run variance.
Figure 8: Collapse rates across simulation designs using GPT-5.2 Chat.
Figure 8: Collapse rates across simulation designs using GPT-5.2 Chat.
Table 4: Correlations between initial-response statistics and eventual collapse in entity-question Search simulations.
NamePearson Correlationp
unique entities−0.3421.32×10−24
length of responses−0.2455.31×10−13
unique words−0.2236.21×10−11
entity ranking similarity0.1641.61×10−6
same-answer %0.1511.01×10−5
(b) Collapse by simulation, editorial questions.
(b) Collapse by simulation, editorial questions.
(c) Questions collapsed by round, entity questions.
(c) Questions collapsed by round, entity questions.

研究结果

  • 在三种模拟设计中,随着轮次增加,回答中出现的独特词汇数和独特实体数都在减少,而回答之间的语义相似度和实体排序相似度都在上升。
  • 1,528次模拟中有1,216次(79.6%)最终以崩溃收尾。
  • 在一项实验中,被判定为互为改写的回答对比例从最初的22%上升到模拟结束时的89%。
  • 当原始资料和自撰资料同时存在时,即便控制了资料质量,模型仍会不成比例地更多引用自撰资料。
  • 截至2026年1月,ChatGPT引用的参考资料中约38.9%被判定为AI生成,到2026年6月这一比例升至约42.7%。
(d) Questions collapsed by round, editorial questions.
(d) Questions collapsed by round, editorial questions.
Figure 9: Core metrics for entity questions across simulation designs. The numbers of unique words and entities decrease over rounds, indicating declining response diversity, while semantic and entity-ranking similarities increase.
Figure 9: Core metrics for entity questions across simulation designs. The numbers of unique words and entities decrease over rounds, indicating declining response diversity, while semantic and entity-ranking similarities increase.

可应用场景

  • 搭建基于检索的AI系统的团队,可以据此考虑对检索池中AI生成内容的比例进行监测或过滤。
  • 从事生成式引擎优化(GEO)或希望自己内容被AI答案引用的内容策划者,可以考虑到AI系统会不成比例地偏好引用与自身风格相似的内容这一点来调整发布策略。
  • 希望监测AI答案多样性的团队,可以借鉴实体可见度、独特词汇数、改写比例等指标,来追踪答案是否正在趋同崩溃。

局限与待验证事项

  • 实验起始所用的原始参考资料中本身就含有约38.9%至42.7%的AI生成内容,且无法精确判定其中哪些是真正的自撰内容,这意味着实际测得的崩溃程度和自我偏好可能被低估而非高估。
  • 实验只测试了反复引入同一模型自撰内容的情形,没有测试反复加入其他类型AI生成内容是否也会导致崩溃,作者认为这也有可能发生。
  • 研究聚焦于信息检索类问题,尤其是实体比较类问题,其他类型问题下的崩溃表现尚未验证。
  • 模拟结果表明在这些条件下会发生崩溃,但并不能直接证明现实中的商业AI系统已经发生了RAG崩溃,这些系统可能已经存在未公开的缓解机制。
  • 涉及代理式搜索下的崩溃情况、缓解措施是否真正有效、以及一个模型检索另一个模型自撰内容的情形,都留作未来工作。

为什么重要

很多人以为依靠搜索的AI答案能反映互联网上多元的观点,但这项研究表明,如果AI不断检索并引用自己过去写的内容,这种多样性会悄悄消失,答案会收敛到某一个未必更好的版本。对于搭建检索增强型AI系统的团队,以及依赖AI生成内容做营销推广(如生成式引擎优化)的人来说,随着网上AI生成内容越来越多,这种反馈循环可能已经在形成,值得提前关注。

本文术语

  • RAG(检索增强生成) · AI在回答问题时先用搜索工具查找相关文档,再据此撰写答案的方式
  • 模型崩溃(model collapse) · AI如果反复用自己生成的内容来训练自己,输出会逐渐失去多样性,最终偏离原始数据分布的现象
  • 自撰参考资料(self-authored reference) · 由同一个AI模型根据自己之前的回答生成、之后又被该AI检索并引用的文章
  • 自我偏好(self-bias) · AI在同等质量下更倾向于引用自己写的内容而非其他来源
  • 实体可见度(entity visibility) · 在多次生成的回答中,某个具体对象(例如某位主播的名字)被提到的回答所占的比例

论文原文摘要（英文）

LLM responses are based on the internet (via training or RAG), and AI is now used to generate a significant amount of content online (Paredes et al., 2026), creating the potential for a self-reinforcing feedback loop. Prior work has shown that when LLMs are recursively trained on their own output, they experience model collapse (Shumailov et al., 2024): responses become less diverse, and eventually no longer resemble the original training data. In this paper, we show that a similar collapse occurs if LLM-based AI systems retrieve references they authored using a search tool. We call this RAG collapse. We conduct extensive experiments with three types of simulations of AI systems retrieving references they generated, using three model families, and 1,019 information-seeking prompts, totaling 1,528 simulations and over one million LLM API calls, and find that 79.6% (1,216/1,528) of simulations end in collapse. Surprisingly, even a single self-authored reference can trigger collapse because the LLM disproportionately cites its own content. This self-bias persists even after controlling for reference quality.

作者 · Gregory Druck, Ethan Smith

在 arXiv 阅读

最新论文

全部论文 →

METAL LAB 最新报道

图片来源： Gregory Druck et al., arXiv:2608.22118, CC BY 4.0