← Back to Blog
For: AI Engineers, ML Engineers, Platform Engineers, AI Systems Architects

The AI Ouroboros Moved: Model Collapse Is a RAG Problem

The labs that train frontier models solved training on synthetic data. Your retrieval-augmented generation corpus is where the loop is still open, and every metric in your eval stack is structurally unable to see it.

ai-engineeringretrieval-augmented-generationgenai
#model-collapse#synthetic-data#rag#retrieval#provenance#data-quality-ai-bias#evaluation#llm-infrastructure#c2pa#synthid

In 2024, a team at Renmin University ran a simple experiment. They took four open-domain question answering datasets, let an LLM write answers into the retrieval corpus, retrieved again, and repeated the cycle ten times. By the tenth iteration, human-written text had fallen from 77.7% of the corpus to 5.8%.

Retrieval accuracy dropped with it. Acc@5 fell an average of 21.4% on Natural Questions and 19.4% on PopQA.

The answer quality metric did not move.

That is the sentence worth sitting with. Their paper states it plainly: "Contrary to expectations, the QA performance does not mirror the retrieval accuracy's decrease." Exact match held roughly flat across all ten iterations while the evidence base underneath it was replaced almost entirely by machine output. Every dashboard stayed green over a corpus that had been hollowed out. That is model collapse arriving through the retrieval layer of a RAG system rather than through a training run, and it is the version almost nobody is watching for.

The thesis: model collapse closed at the labs and reopened in your RAG corpus

In the popular version of the AI ouroboros, models train on their own output, degrade, and drag the next generation down with them. That story was accurate about one specific experimental condition and has been wrong about production practice for roughly two years.

Here is the claim this article owns. Model collapse is largely a managed problem for the organizations that train frontier models, and an unmanaged, unmeasured one for everyone who retrieves and evaluates. That failure moved downstream into your vector store, where it is invisible to your eval suite for a structural reason: correctness and provenance are independent properties, and every metric you run measures the first one.

Worse, the standard fix is not available. You cannot detect your way out, and I will show why the arithmetic forbids it.

What the Nature paper actually tested

Shumailov and colleagues published "AI models collapse when trained on recursively generated data" in Nature in July 2024. Their result is real, and their setup deserves more attention than it got.

In the language model experiment they fine-tuned OPT-125m on wikitext2 at a sequence length of 64 tokens. A model fine-tuned on real wikitext2 reaches about 34 mean perplexity, down from a zero-shot baseline of 115. Each generation then trained on text the previous generation produced.

In the preprint, the headline condition reads like this: the model "is trained for 5 epochs on the original dataset and no original data." No original data. Under that condition perplexity climbs steadily across generations and the tails of the distribution disappear first, a progression the authors split into early collapse and late collapse. A second condition in the same paper changes one thing. There the model "is trained for 10 epochs on the original dataset and every new generation of training, a random 10% of the original data points are sampled." Their own summary of that run: "preservation of the original data allows for better model fine-tuning and leads to only minor degradation."

Ten percent retention. Minor degradation. It is in the paper that launched the collapse narrative, and it is the condition almost nobody quotes.

Accumulate, do not replace: what actually prevents model collapse

Retention versus replacement turned out to be the whole ballgame, and Gerstgrasser and colleagues at Stanford tested it directly in "Is Model Collapse Inevitable?", framing the problem this way: prior work "largely assumed that new data replace old data over time, where an arguably more realistic assumption is that data accumulate over time."

They proved a bound. For linear regression with isotropic features and TT samples per iteration where T≥d+2T \ge d+2, the accumulate case satisfies

EtestAccum(w^n)  ≤  σ2dT−d−1⋅π26E_{\text{test}}^{\text{Accum}}(\hat{w}_n) \;\le\; \frac{\sigma^2 d}{T-d-1} \cdot \frac{\pi^2}{6}

while replacing the data each generation gives

EtestReplace(w^n)  =  σ2dT−d−1⋅nE_{\text{test}}^{\text{Replace}}(\hat{w}_n) \;=\; \frac{\sigma^2 d}{T-d-1} \cdot n

The first bound has no nn in it. That is the whole result: under accumulation the error ceiling is a constant no matter how many generations you run, while under replacement it grows linearly in the generation count nn. Their conclusion: "if data instead accumulate, the test error has a finite upper bound independent of the number of iterations, meaning model collapse no longer occurs."

Empirically they trained GPT-2 and Llama2 variants on TinyStories for up to ten generations. At iteration 4, Llama2-125M under accumulation reached about 1.59 cross-entropy, better than its 1.71 starting point. Under replacement it degraded to about 2.23. Diffusion models on GEOM-Drugs and VAEs on CelebA showed the same split: replacement produced complete mode collapse and accumulation preserved the major axes of variation.

A 2025 position paper from Schaeffer, Kazdan, Arulandu and Koyejo went further. They catalogued eight distinct and partly conflicting definitions of model collapse in the literature, then argued that "certain predicted claims of model collapse rely on assumptions and conditions that poorly match real-world conditions, and in fact several prominent collapse scenarios are readily avoidable." Their verdict on the discourse is blunt: the idea has been "warped from a nuanced multifaceted consideration into an oversimplified threat."

How much synthetic data frontier labs actually use

If synthetic data were the poison the popular story describes, the organizations with the most to lose would avoid it. They do the opposite, and they publish the numbers.

Microsoft's Phi-4 technical report gives its pretraining mix in Table 5: 40% synthetic, 15% web, 15% web rewrites, 20% code, 10% acquired sources. They generated roughly 400 billion unweighted tokens across 50 broad types of synthetic dataset. Their 14-billion-parameter result scores 56.1 on GPQA against GPT-4o's 50.6, and 80.4 on MATH against 74.6. It loses on MMLU and HumanEval, so this is not a clean sweep. It is also not collapse. Phi-4's authors argue synthetic data is better for some purposes, not merely cheaper: "each token generated by a language model is by definition predicted by the preceding tokens, making it easier for a model to follow the resulting reasoning patterns."

Meta's Llama 3 paper is more striking. They generated over 2.7 million synthetic examples for code supervised fine-tuning (SFT) and roughly 650,000 for tool use. They ran six rounds of post-training, using earlier Llama 3 checkpoints to generate data and to rate it. In the multilingual slice of their SFT mix, Table 7 reports 2.4% human annotations. That is one slice rather than the whole corpus and should be quoted as such, but it is still a shipped frontier model whose multilingual instruction data was 97.6% machine-produced.

How much synthetic data is right? A Meta and Virginia Tech study presented at EMNLP 2025 trained over 1,000 LLMs across more than 100,000 GPU-hours to find out. Their optimum converged near 30% rephrased synthetic data against 70% natural web text, with speedups of 5x to 10x at larger budgets. Rephrased synthetic data alone did not beat natural web text. Textbook-style synthetic data alone produced patterns the authors describe as consistent with model collapse.

Be careful what these results prove, because it is less than the headline suggests and the smaller claim is the one I need. Phi-4 and Llama 3 are single-generation pipelines, not ten-generation recursive runs, and no lab publishes the mix for the run that degraded, so survivorship bias is unavoidable here. They show that synthetic data works when it is curated. They do not show that recursion is harmless.

That distinction is the argument rather than a caveat on it. Look at what sits inside these pipelines: executed unit tests for code, rejection sampling against a reward model, a held-out real corpus, human annotation on the slices that matter, deduplication. Every one of those injects a signal from outside the model into the loop. That is what a curation gate actually contains, and naming it matters, because a reader who takes away "the answer is dosing" has taken the least important of the three.

So the recipe is not abstinence. It is a dose, a gate with a non-model signal in it, and retention of the real corpus. Keeping real data while mixing deliberately is exactly the condition the Nature paper found produced only minor degradation. Hold on to that list, because your ingest pipeline has none of these, and it is why the loop closes there and stays open here.

None of this means collapse is solved. Dohmatob, Feng and colleagues showed in "Strong Model Collapse" at ICLR 2025 that even 1% synthetic data can induce collapse under their assumptions, though those assumptions are linear regression with isotropic Gaussian features, and the authors state that the theory "does not apply directly" to fully trained networks. Managed in practice is the honest phrasing. Solved is not.

Why AI-generated content wins your retrieval ranking

Before going further I should be precise about what carries over from the last four sections, because a metaphor is not an argument and the ouroboros is a metaphor.

Model collapse is a claim about parameters. Variance shrinks, tails vanish, a model fits its own output across generations. Nothing in your retrieval stack trains anything, so the tail-loss result from Shumailov and the regression bounds from Gerstgrasser do not transfer to a corpus, and I am not claiming they do. What transfers is narrower and it is structural: recursive self-consumption in which replacement is not offset by retention of some signal from outside the loop. That is the precondition both systems share. The lab satisfies it deliberately by keeping real data and injecting an external check. Your corpus satisfies none of it by default. Where the collapse literature is genuinely useful downstream is as a source of the failure's shape, not its mathematics, and one specific finding does transfer cleanly, which I come back to later: tails go first.

So: not the same mechanism, the same precondition. With that said, consider your retrieval corpus. You did not curate it the way Meta curated Llama 3's pretraining mix. You pointed a crawler at a docs site, a Confluence space, a support archive, or the open web, and you scheduled a refresh.

Three findings, published across three years and three venues, describe what happens next.

Retrievers prefer machine text. Dai and colleagues at KDD 2024 measured that "neural retrieval models tend to rank LLM-generated documents higher," named it source bias, and found that it extends to second-stage neural rerankers. Their explanation is that LLM-generated text "exhibits more focused semantics with less noise," which makes semantic matching easier.

Perplexity is the mechanism, and it is causal. At ICLR 2025, the "Perplexity Trap" paper found that retrievers assign higher relevance to LLM-generated documents "even when their semantic quality is comparable to human-written ones," because pretrained language model (PLM) based retrievers learn perplexity as a feature for relevance estimation. Gradients from the language modelling objective and the retrieval objective are positively correlated. Low perplexity reads as relevance.

Contamination amplifies through the ranking stage. Yu, Kim and Kim's "Retrieval Collapses When AI Pollutes the Web" at The Web Conference 2026 ran 20 rounds of cumulative contamination over 1,000 MS MARCO query-answer pairs and a 10,000-document pool. They separate three rates: pool contamination, exposure contamination in the top-10, and citation contamination in what the answerer actually cites. At 67% pool contamination, exposure exceeded 80% while answer accuracy moved from about 68% to about 70%.

Two independent teams, two years apart, different corpora, different retrievers, different models. Both measured the evidence base turning synthetic while the answer metric held flat.

Provenance Blindness: the failure every RAG eval metric is built to miss

I call this Provenance Blindness: the condition where a retrieval system's measurement surface reports on the correctness of answers and nothing in it reports on the origin of evidence, so corpus composition can invert without any metric that reliably signals it.

Be careful with "reliably", because it is the load-bearing word and my own opening looks like a counter-example. Acc@5 fell 21.4% in the Renmin run, and Acc@5 is a retrieval recall metric. So a metric did move. The problem is that it carries no fixed sign. In that run retrieval accuracy fell; in the Yu, Kim and Kim run answer accuracy rose, from about 68% to about 70%, while exposure contamination passed 80%. Same underlying phenomenon, opposite readings. A signal that can move either way, or not at all, is not a signal you can alert on, and a retrieval-metric drop is at best a symptom you notice afterwards. Worse, catching it that way needs a labelled golden set, most teams do not keep one, and the labels themselves decay as the corpus drifts underneath them.

Provenance Blindness is not an oversight in any one tool. It is what every tool measures.

FrameworkMetricsProvenance metric
RagasFaithfulness, Response Relevancy, Context Precision, Context Recall, Noise Sensitivity, Topic AdherenceNone
DeepEval30+ across RAG, agentic, conversational, safety, multimodalNone
TruLensThe RAG Triad: Context Relevance, Groundedness, Answer RelevanceNone
Arize PhoenixRelevance, Q and A Correctness, Hallucination, ndcg@k, precision@kNone

Context precision asks whether the retrieved chunks were relevant. Context recall asks whether the relevant chunks were retrieved. Faithfulness asks whether the answer follows from the chunks. A synthetic document that summarises three real sources scores well on all three, because it is relevant, retrievable, and faithfully summarised. Provenance is orthogonal to every axis being measured.

By the definition I used in the Evals Blind Spot, the gap between what you measure and what can actually fail, this is a third dimension of that same blind spot rather than a separate species. I would rather say that plainly than claim a distinction I did not earn. What makes it worth its own name is that it is the dimension that survives the fix: adding retrieval-layer metrics closes the first two and does nothing at all for origin. The Staleness Gap is likewise orthogonal, because it runs on the time axis, asking whether a document is current, while this runs on the origin axis, asking who produced it. A document can be fresh, relevant, and entirely machine-written.

One more term needs separating, since it is mine and it starts with the same word. Provenance debt, which I wrote about in the context of vendor watermarking, is the unresolved attribution obligation you accrue in training data. Provenance blindness is the inability to measure where retrieved evidence came from at query time. Different layer, different owner, different remedy.

The interaction between staleness and provenance is where it gets unpleasant, though it holds under narrower conditions than the neat version of the sentence suggests. Re-ingesting more often is the standard remedy for the Staleness Gap. When the refresh is discovery-driven and replacing, meaning it finds new URLs and lets them displace old ones, a faster cycle pulls in a larger share of recently published material, and recent material is where synthetic content concentrates. Under those two conditions, fixing your freshness problem accelerates your provenance problem. Re-fetch a fixed URL set on a faster cadence and this does not apply; you are changing document versions, not corpus composition. Nor does it apply if your index accumulates rather than replaces, which is the same lesson the labs learned, arriving at your corpus.

Why AI text detection cannot clean a RAG corpus

An obvious response is to classify documents as human or machine at ingest and filter, and three separate results say that fails.

Writing in Patterns in 2023, Liang and colleagues tested seven then-current GPT detectors against 91 TOEFL essays written by non-native English speakers. Average false positive rate: 61.3%. Eighteen of the 91 essays were flagged by all seven detectors unanimously. Eighty-nine of 91 were flagged by at least one. Those same detectors correctly classified over 90% of US eighth-grade essays as human. Perplexity is the mechanism again: detectors flag low-perplexity text, and a competent non-native writer using conventional phrasing produces low-perplexity text. Then this, from the same study. Prompting ChatGPT to elevate the essays' vocabulary dropped the average false positive rate from 61.3% to 11.6%. A detection filter is trivially evaded by anyone who bothers, and reliably misfires on people who did nothing wrong.

That specific rate is three years old and a 2026 detector will post a different one, so do not treat 61.3% as a current constant. Treat the failure mode as the durable part. Flagging low-perplexity text and evaporating under a paraphrase prompt are structural properties of likelihood-based detection, and nobody has shown them fixed.

Now put the two perplexity results side by side, carefully, because the sloppy version of this comparison is wrong. Retrievers rank low-perplexity documents higher. Detectors flag low-perplexity documents as machine-written. These are not literally the same number: the detectors compute likelihood explicitly under a reference model, while the retrievers have absorbed a correlate of it as a learned feature, and no paper establishes that the two track each other document by document. What they share is the underlying property, which is how predictable the text is under a language model. That property is what the retriever rewards and what the detector punishes. Build the filter on it and you are not removing synthetic content so much as penalising fluent, conventional writing, which in a global corpus means penalising your non-native authors. Provenance is not a property of text. It is a property of history, and history is not recoverable from the tokens.

Watermarking does not close the gap either, and Google says so directly. SynthID-Text is real, published in Nature, and validated across roughly 20 million Gemini responses. DeepMind's own documentation states its confidence scores "can be greatly reduced when an AI-generated text is thoroughly rewritten or translated to another language," and that it is "less effective on responses to factual prompts because there are fewer opportunities to adjust the token distribution." Short factual text is most of a RAG corpus. I covered how the scheme works in How Google's SynthID Actually Works.

C2PA is the right standard and has Adobe, Google, Meta, Microsoft, OpenAI, BBC, Sony and TikTok on its steering committee. It publishes no adoption statistics, and that absence is the answer for planning purposes: you cannot build a coverage-dependent control on a standard whose coverage is unpublished.

Where this could be wrong: live-web exposure, derivative accuracy, and poisoning

A skeptical staff engineer should push back here, and the pushback is good enough that the article owes it a section.

Your exposure numbers come from simulations that deleted the defenses. Correct, and the authors agree. Yu, Kim and Kim write that their metrics are "descriptive decompositions intended to characterize dynamics rather than novel theoretical measures" and call for validation "in live, large-scale web environments." In the live web the number is much smaller. A study of 31,493 keywords found 86% of Google-ranking articles were human-written, and only 7% of position-one results were AI-generated.

Two things need saying about that figure and only one of them helps me. It is a vendor study by an SEO firm whose business benefits from the conclusion. It was also produced by running an AI-text classifier across ranking articles, which by the argument I just spent a section making is an unreliable instrument in both directions. So I am conceding a number that my own analysis says nobody should fully trust, including me, and I would rather concede it on those terms than quietly drop it.

Where I think it still cuts against me is narrower than it looks, and it is about which system you run. That 14% is what twenty years of adversarial spam defense buys on Google's search results. Your Confluence space has no spam team. An enterprise corpus is an undefended append-only surface, so 14% is the best case that heavy defense achieves, not the baseline you inherit.

The synthetic documents were better. This is the sharpest objection and it is empirically true. In Yu, Kim and Kim's SEO scenario, the synthetic pool scored 66.79% correctness against the original web pool's 51.69%. Machine-written summaries displaced human pages and were more likely to support the right answer. A fifteen-point correctness gain is large, and "I will take that and monitor" is a rational response.

I want to be honest about what I can and cannot show here. No published run has gone far enough to observe correctness turn back down: Yu ran 20 rounds and accuracy rose, Xie ran 10 iterations and exact match stayed flat. Predicting an eventual downturn would be extrapolating past the measured range, which is precisely what I criticised the popular collapse narrative for doing. So I will not. Instead, here are two harms that a correctness number is structurally unable to price, both of which are visible inside the measured range.

The first is independence collapse, and it is the one that matters. Groundedness, faithfulness, and every consensus or self-consistency check in your stack quietly assume that retrieved documents are independent pieces of evidence. Derivative synthetic documents are not independent. When four of your top-5 are summaries drawn from the same underlying sources, you no longer hold five pieces of evidence agreeing; you hold one piece of evidence with five votes. Accuracy on a benchmark cannot see this, because the answer stays right. What degrades is calibration: the system becomes confident for a reason that is no longer valid, and confidence is what you route escalation and human review on.

The second is head and tail asymmetry. That 66.79% was measured over MS MARCO query-answer pairs, which are head questions with short checkable answers. Summarisation preserves exactly that and loses the rare case. This is the one place where the Nature finding transfers cleanly rather than metaphorically: tails disappear first, and a head-weighted benchmark is the last place you would ever see it happen.

There is no fix, so this is not an indictment. Partly right, and it forces the recommendation to change. I am not proposing detection. What follows never classifies text.

The real risk is deliberate poisoning. Fair, and I would rather concede the priority than defend a ranking. PoisonedRAG achieved a 90% attack success rate by injecting five malicious texts per target question into a database of millions, which is a sharper and more immediate threat than gradual drift. It is also a security problem with an active literature and known mitigations, where this is neither. What the two share is a root cause: the corpus is an unmonitored append-only surface. The instrumentation below was designed for drift, and it happens to give an incident responder the one thing they otherwise lack, which is a record of what entered the corpus, when, and through which path.

The wrong way: ingest by relevance alone

Most pipelines I have reviewed look roughly like this. I am deliberately showing a reasonably mature one rather than a toy, because the toy version lets you off the hook.

code
# ingest.py - a competent version, with real metadatadef refresh_corpus(urls, store, embed, chunk_text, fetch, extract_text):    for url in urls:        doc = fetch(url)        text = extract_text(doc)        for offset, chunk in enumerate(chunk_text(text, size=512)):            store.upsert(                id=f"{url}#{offset}",                vector=embed(chunk),                payload={                    "doc_id": doc.id,                    "title": doc.title,                    "section": doc.section_for(offset),                    "text": chunk,                    "url": url,                    "updated_at": doc.updated_at,                    "tenant": doc.tenant,                },            )

Seven fields, access control, section anchors for citation, a timestamp. The chunking strategy is considered and the embedding model is domain-matched. This is not a strawman and it passes review at most companies. It still has no provenance, and updated_at is the field that shows why: it describes the content lifecycle of the document, when its text last changed. It says nothing about how the document entered your corpus or where it came from before that. Run this quarterly against a web source and corpus composition shifts underneath you with no record of the shift, because nothing here writes down when a document first appeared in your index, how it got in, or whether it duplicates something already there. There is no artifact to compare against last quarter.

What makes it hard to notice is that the evaluation you run over this pipeline passes. Take a query whose top-3 hits are all machine-written summaries of the same three human sources, and score them the usual way. Treat this as pseudocode; the real frameworks wrap these in metric objects.

code
# All three hits are synthetic summaries of real pages. Score them anyway.hits = [h1, h2, h3]                        # retrieved chunks, top-3print(context_precision(query, hits))      # 1.00 - every chunk is on topicprint(context_recall(query, hits, truth))  # 1.00 - the answer is in thereprint(faithfulness(answer, hits))          # 0.98 - the answer follows the chunks

Every number is honest. Each chunk really is relevant, the answer really is supported, and a reviewer spot-checking the output would sign it off. Not one of these three functions takes an argument that could carry origin, which is the part worth pausing on: the interface has nowhere to put the information even if you had it.

The right way: record lineage at ingest, measure composition at retrieval

Instead of asking "was this written by a machine", record what you already know at crawl time. None of these fields requires a classifier.

code
# lineage.pyfrom dataclasses import dataclass, asdictfrom datetime import datefrom typing import Literal, Optionalfrom urllib.parse import urlparseIngestPath = Literal["vetted_feed", "partner_api", "open_crawl", "user_upload"]# ChatGPT's public release. Publishing before this date rules out# general-purpose LLM chat as the source. It does NOT mean "human":# machine translation and template generation predate it by years.PRE_LLM_CUTOFF = date(2022, 11, 30)def domain_of(url: str) -> str:    return (urlparse(url).hostname or "").lower().removeprefix("www.")@dataclass(frozen=True)class Lineage:    source_domain: str    ingest_path: IngestPath           # you control this    first_seen: date                  # you control this    published_at: Optional[date]      # SOURCE-CONTROLLED. Do not trust it.    c2pa_manifest: bool    near_duplicate_of: Optional[str]  # a chunk in a DIFFERENT document    def date_class(self) -> str:        """Three outcomes, not two. An absent date is its own finding."""        if self.published_at is None:            return "unknown"        return "pre_llm" if self.published_at < PRE_LLM_CUTOFF else "post_llm"    def is_vetted(self) -> bool:        """Trust in the channel, not verification of origin:        a partner API can happily serve you LLM-generated text."""        return self.ingest_path in ("vetted_feed", "partner_api")    def to_payload(self) -> dict:        """Flat, JSON-safe scalars so the store can filter on them."""        d = asdict(self)        d["first_seen"] = self.first_seen.isoformat()        d["published_at"] = (            self.published_at.isoformat() if self.published_at else None        )        return d    @classmethod    def from_payload(cls, p: dict) -> "Lineage":        return cls(            source_domain=p["source_domain"],            ingest_path=p["ingest_path"],            first_seen=date.fromisoformat(p["first_seen"]),            published_at=(                date.fromisoformat(p["published_at"]) if p["published_at"] else None            ),            c2pa_manifest=p["c2pa_manifest"],            near_duplicate_of=p["near_duplicate_of"],        )

Three things in there are deliberate and worth calling out, because each one is a place I got it wrong first.

published_at is the only field in the record that the source controls, which makes it the only one an adversary can set. Backdating a meta tag on a page you already described as sitting on an undefended append-only surface is a one-line change. ingest_path and first_seen you assign yourself, so they are the fields worth building alerts on.

date_class returns three values rather than a boolean. A large share of any real crawl has no usable publication date at all, because meta tags are missing, the CMS emits last-modified as published, or the document is a PDF. Collapsing those into "not pre-LLM" builds a metadata-coverage alarm and labels it a contamination alarm, which then fires permanently and gets muted in week one. A high unknown_date_share is a real and separately actionable finding: it means you cannot answer this question for that slice of the corpus at all.

near_duplicate_of points at a chunk in a different document, and the exclusion matters more than it looks. Match within the same document and every unchanged chunk near-duplicates its own copy from the previous crawl, so the metric pins at 1.0 from run two onward. The signal you want is cross-document derivation.

Ingest fills it in from the crawl itself:

code
# ingest.py - lineage-awaredef refresh_corpus(sources, store, embed, chunk_text, fetch, extract_text,                   today, find_near_duplicate):    for url, ingest_path in sources:        doc = fetch(url)        text = extract_text(doc)        # Key the age to the DOCUMENT, never the chunk. See below.        doc_first_seen = store.first_seen(url) or today        for offset, chunk in enumerate(chunk_text(text, size=512)):            lineage = Lineage(                source_domain=domain_of(url),                ingest_path=ingest_path,                first_seen=doc_first_seen,                published_at=doc.published_at,                c2pa_manifest=doc.has_c2pa_manifest,                near_duplicate_of=find_near_duplicate(chunk, exclude_url=url),            )            store.upsert(                id=f"{url}#{offset}",                vector=embed(chunk),                payload={"url": url, "text": chunk, **lineage.to_payload()},            )

store.first_seen(url) returns the recorded date for a document already in the index and None for a new one, so re-ingesting does not reset its age. Key that lookup to the URL rather than the chunk id, which is the single most breakable thing in this design. Chunk ids embed an offset. Edit one paragraph near the top of a page and every offset below it shifts, so every chunk looks new, and the document's entire history resets to today. Change your chunk size or upgrade your chunker, which is a routine Tuesday, and the whole corpus resets at once. Nothing alerts when this happens, because a corpus where every document was first seen today is indistinguishable from a fresh install. You would lose the field without ever being told. If you want chunk-level granularity, hash the chunk content instead of numbering it, and fall back to the document date.

Then measure the composition of what actually reaches the model, which is the top-k, not the pool:

code
# composition.pyfrom collections import Counterfrom dataclasses import dataclassfrom typing import Optionalfrom lineage import Lineage@dataclass(frozen=True)class Composition:    n: int    pre_llm_share: float        # of hits with a KNOWN date    post_llm_share: float       # of hits with a KNOWN date    unknown_date_share: float   # of all hits    vetted_share: float    duplicate_share: float    domain_concentration: float  # share held by the single largest domain    median_age_days: int         # median days since first_seendef composition_of(hits, today) -> Optional[Composition]:    """hits: the top-k actually placed in the context window.    Returns None on an empty result set. That is a retrieval failure,    not a contamination signal, and it should not page anybody.    """    n = len(hits)    if n == 0:        return None    recs = [Lineage.from_payload(h.payload) for h in hits]    classes = Counter(r.date_class() for r in recs)    known = classes["pre_llm"] + classes["post_llm"]    domains = Counter(r.source_domain for r in recs)    ages = sorted((today - r.first_seen).days for r in recs)    return Composition(        n=n,        pre_llm_share=classes["pre_llm"] / known if known else 0.0,        post_llm_share=classes["post_llm"] / known if known else 0.0,        unknown_date_share=classes["unknown"] / n,        vetted_share=sum(r.is_vetted() for r in recs) / n,        duplicate_share=sum(r.near_duplicate_of is not None for r in recs) / n,        domain_concentration=domains.most_common(1)[0][1] / n,        median_age_days=ages[n // 2],    )# Movement against your own baseline, not absolute levels. There is no# universal correct value for any of these - the right starting point is# whatever your corpus reads today, captured once and kept.FALLING = ("pre_llm_share", "vetted_share")RISING = ("duplicate_share", "domain_concentration")def trend_breaches(current: Composition, baseline: Composition, limit=0.05):    fired = []    for field in FALLING:        delta = getattr(baseline, field) - getattr(current, field)        if delta >= limit:            fired.append(f"{field} fell {delta:.3f} against baseline")    for field in RISING:        delta = getattr(current, field) - getattr(baseline, field)        if delta >= limit:            fired.append(f"{field} rose {delta:.3f} against baseline")    return fired

Log a Composition per query and aggregate it daily against a stored baseline. You are measuring drift in where evidence comes from, which is a logging problem, and it sidesteps the false positive rate from the previous section entirely because no text is ever classified.

Two of those fields look like ordinary hygiene metrics and are not. duplicate_share and domain_concentration are the fingerprint of independence collapse from the objections section. Rising near-duplication and rising domain concentration in the top-k are how "five sources agree" quietly becomes "one source, counted five times", and they are measurable without knowing whether any individual document is synthetic. median_age_days is the cheapest signal in the whole record: it needs a field you already store, the source cannot forge it, and it keeps working on corpora built entirely after 2023, where pre_llm_share is dead on arrival.

Does this actually catch anything?

Test it against the one published trajectory where the ground truth is known. Xie and colleagues report two anchor values for Natural Questions: human-written text holds 77.7% of the corpus at the first iteration and 5.8% at the tenth, while exact match stays roughly flat throughout. Those are pool shares rather than top-k shares, which matters and works in the instrument's favour, for reasons I get to below. I do not have their per-iteration table, so the curve between the anchors is a straight-line interpolation included to show when an alert would fire, not to report a measurement.

This is also where the difference between alerting on a level and alerting on a trend stops being a style preference.

code
# Anchors are published: Xie et al., ACL 2024 (NQ), human share 0.777 -> 0.058# across 10 iterations, with exact match roughly flat. Intermediate points are# INTERPOLATED for illustration, not measured values from the paper.start, end, steps = 0.777, 0.058, 10human_share = [start + (end - start) * i / (steps - 1) for i in range(steps)]LEVEL_FLOOR, TREND_LIMIT = 0.20, 0.05prev = Nonefor i, share in enumerate(human_share, start=1):    level = "FIRED" if share < LEVEL_FLOOR else "-"    drop = 0.0 if prev is None else prev - share    trend = "FIRED" if drop >= TREND_LIMIT else "-"    print(f"iter {i:>2}  human={share:.3f}  em=flat  "          f"level:{level:<5}  trend:{trend}")    prev = share
code
iter  1  human=0.777  em=flat  level:-      trend:-iter  2  human=0.697  em=flat  level:-      trend:FIREDiter  3  human=0.617  em=flat  level:-      trend:FIREDiter  4  human=0.537  em=flat  level:-      trend:FIREDiter  5  human=0.457  em=flat  level:-      trend:FIREDiter  6  human=0.378  em=flat  level:-      trend:FIREDiter  7  human=0.298  em=flat  level:-      trend:FIREDiter  8  human=0.218  em=flat  level:-      trend:FIREDiter  9  human=0.138  em=flat  level:FIRED  trend:FIREDiter 10  human=0.058  em=flat  level:FIRED  trend:FIRED

Look at the two alert columns rather than the numbers. The level floor fires at iteration 9, by which point 86% of the human corpus is already gone, and as a leading indicator that is close to worthless. The trend alert fires at iteration 2, on the first movement, with the corpus still 70% human. Same data, same instrument, same field written at ingest. The only difference is whether you alert on where the number sits or on where it is going, which is why the recommendation below says trend and why shipping Composition without trend_breaches would have given you a lagging indicator wearing a leading indicator's clothes.

The answer-quality column never fires at all, because across their ten iterations it never moved. That gap is Provenance Blindness, and closing it needed no classifier and no new research.

Three caveats on this demonstration. A straight line is the most conservative shape available between the published anchors: if the real decline is steeper early, as contamination dynamics usually are, both alerts fire sooner. The anchors are pool shares, and by the exposure argument above the top-k curve is steeper than the pool curve, so a top-k alert would fire earlier still. And pre_llm_share only proxies human authorship while your corpus still holds pre-2023 material, which is why median_age_days, vetted_share and duplicate_share exist alongside it.

How the two loops differ

Both loops below start from the same model output. One passes through a curation gate and returns to training. The other has no gate, and every metric you run probes it from the side that stays green.

mermaid
flowchart TD
    SRC["Model output<br/>Generation N"]

    subgraph LAB["The closed loop: gated at the lab"]
        direction TB
        GATE{"Curation gate<br/>accumulate real data<br/>filter, weight, dose"}
        CORP["Training corpus<br/>real data retained"]
        NEXT["Generation N+1<br/>measurably better"]
    end

    subgraph PROD["The open loop: no gate anywhere"]
        direction TB
        WEB["Published to the open web"]
        CRAWL["Scheduled re-ingest<br/>the staleness fix"]
        NOGATE["No provenance gate<br/>relevance is the only filter"]
        STORE["Vector store"]
        EVID["Top-k retrieved as evidence"]
        ANS["Answer: fluent, cited, grounded"]
    end

    SRC --> GATE
    GATE --> CORP
    CORP --> NEXT
    NEXT --> SRC

    SRC --> WEB
    WEB --> CRAWL
    CRAWL --> NOGATE
    NOGATE --> STORE
    STORE --> EVID
    EVID --> ANS
    ANS --> WEB

    subgraph MEAS["Your eval stack: all green"]
        direction LR
        M1["Context recall"]
        M2["Context precision"]
        M3["Faithfulness"]
        M4["Answer accuracy"]
    end

    EVID -.-> M1
    EVID -.-> M2
    ANS -.-> M3
    ANS -.-> M4

    COMP["Retrieval-set composition<br/>the only probe that moves"]
    EVID -.-> COMP

    style SRC fill:#4A90E2,color:#FFFFFF
    style GATE fill:#6BCF7F,color:#2C2C2A
    style CORP fill:#98D8C8,color:#2C2C2A
    style NEXT fill:#98D8C8,color:#2C2C2A
    style WEB fill:#FFA07A,color:#2C2C2A
    style CRAWL fill:#FFA07A,color:#2C2C2A
    style NOGATE fill:#E74C3C,color:#FFFFFF
    style STORE fill:#FFA07A,color:#2C2C2A
    style EVID fill:#FFA07A,color:#2C2C2A
    style ANS fill:#FFA07A,color:#2C2C2A
    style M1 fill:#6BCF7F,color:#2C2C2A
    style M2 fill:#6BCF7F,color:#2C2C2A
    style M3 fill:#6BCF7F,color:#2C2C2A
    style M4 fill:#6BCF7F,color:#2C2C2A
    style COMP fill:#C2185B,color:#FFFFFF

One node is the whole structural difference: the lab has a gate and you do not.

Fixing Provenance Blindness: measure first, then intervene

Five things to measure:

  1. Add a lineage record to your ingest payload this sprint. Source domain, ingest path, first-seen date, published-at date, C2PA manifest present, near-duplicate id. Six fields, all available at crawl time, none requiring a classifier. The C2PA boolean will read False for essentially every text document you hold today; record it anyway, because it costs one bit and that field is where coverage shows up first if the standard ever lands.
  2. Backfill first_seen from your crawl logs before you need it, and key it to documents. If you keep no crawl logs, start today and accept that your history begins now. Watch for the silent reset described above whenever you re-chunk.
  3. Log retrieval-set composition per query, not pool composition. Yu, Kim and Kim measured exposure running well ahead of pool contamination, 80% against 67%. Pool numbers will reassure you while the top-k is already gone.
  4. Alert on the trend against a stored baseline, not on the level. The demonstration above is the whole argument: the level floor fired seven iterations after the trend alert did. Capture today's numbers as the baseline and alert on movement.
  5. Hold a frozen evaluation slice. This sits alongside, not instead of, measuring RAG quality in production. Snapshot a corpus subset now, keep it immutable, and re-run your golden queries against it quarterly. When live and frozen results diverge, the corpus moved rather than the model.

Measurement on its own is a thermometer, so three things to actually do when it moves:

  1. Accumulate, do not replace, in your corpus too. This is the direct transfer of the lab lesson and the cheapest intervention here. Never expire or deprecate pre-cutoff and vetted documents out of the index just because fresher material outranks them. Retaining a real base is what turned catastrophic collapse into minor degradation in the Nature experiment, and in your index it costs storage and nothing else.
  2. Constrain top-k composition at retrieval time. Require that every context window contains at least one vetted-path or pre-cutoff document, backfilling from a reserved slot when the ranker did not supply one. It is deterministic, it classifies no text, and it is the only intervention on this list that changes what the model actually sees.
  3. Tighten the ingest path before tightening any filter. Moving a source from open_crawl to vetted_feed does more for composition than any detector and has no false positive rate. Treat user_upload as your highest-risk path, since it is where people paste model output directly into your corpus.

And one thing not to do:

  1. Do not deploy an AI-text detector as an ingest gate without measuring its false positive rate on your own content first. Specifically on your non-native English sources and your templated documents. The published rate on non-native writing was catastrophic in 2023, the failure mode is structural to likelihood-based detection, and a vendor's headline accuracy number tells you nothing about your corpus.

Decision guide: what to log for your corpus type

Your situationDo this
Internal documents only, no web crawlLog ingest_path and duplicate_share. Skip domain_concentration, which is a constant 1.0 when everything is one host; substitute space or author concentration if you want a diversity signal
Internal docs mixed with scraped public sourcesFull lineage record, trend alerting on vetted_share
Primarily open-web crawlFull lineage plus a frozen eval slice, and the top-k composition constraint from step 7
Corpus built entirely after 2023pre_llm_share is uninformative. Alert on median_age_days, vetted_share and duplicate_share
Publication dates mostly missing or untrustworthyIgnore the date fields entirely and alert on unknown_date_share plus the fields you control: ingest_path and first_seen
You are asked to prove the corpus is cleanYou cannot. Report composition trend and ingest-path mix, and refuse to report an authorship percentage

What this means for the corpus you own

The snake is still eating itself. It moved to a part of the stack where nobody installed a mirror.

Organizations training frontier models read the collapse literature, noticed that the catastrophic condition was total replacement, and built pipelines that accumulate and curate instead. Phi-4 runs 40% synthetic on purpose. Llama 3's multilingual instruction data was 2.4% human-annotated. Roughly 30% synthetic is where the controlled study puts the optimum. They kept the real data and they dose deliberately, which is precisely the condition the Nature paper found produced only minor degradation.

You inherited the other condition. Your corpus refreshes from a web where Brooks and colleagues found over 5% of newly created English Wikipedia articles carrying detector flags by August 2024, and where Thompson and colleagues at AWS found machine-translated content dominating the total web text of lower-resource languages entirely, with low-quality English material preferentially translated outward. Your retriever actively prefers that material, because low perplexity reads as relevance. Your measurement surface reports on whether answers are correct rather than on where evidence came from. That is Provenance Blindness, and it persists for a reason that is not negligence: provenance is not a property of text, so the classifier everyone reaches for cannot be built.

What can exist is a record. Write down where each document came from, on the day it arrives, and watch the mix. It will not tell you what is synthetic. It will tell you when the ground under your system has shifted, which is the question your dashboard is currently answering with a confident, well-cited, entirely green no.

References

AI Engineering

Retrieval Augmented Generation

Follow for more technical deep dives on AI/ML systems, production engineering, and building real-world applications:

Get the next article by email

One email when a new piece goes up. No digest, no drip sequence.

One email per new article. Unsubscribe in one click.

Books by Ranjan Kumar

The 7 GenAI Architectures cover

The 7 GenAI Architectures

Explore the Book
Building Real-World Agentic AI Systems with LangGraph cover

Building Real-World Agentic AI Systems

Explore the Book
The ChatML Handbook cover

The ChatML Handbook

Explore the Book
The Chat Templates Handbook cover

The Chat Templates Handbook

Explore the Book

Comments