infini-gram

Infini-gram pre-indexes an enormous collection of text and lets you instantly find out how many times a given sentence or phrase appears in it.

Think of it like scanning an entire library and typing a sentence into a search box to instantly learn which book, which page, and how many times that exact phrase shows up. Except here the "library" isn't just printed books—it's a massive collection of text scattered across the internet—and it can search for anything from a short word to a very long sentence, with no length limit.

The Allen Institute for AI (Ai2) built it and has used it to check whether the outputs of the language models (AI programs that generate text) it releases are directly copying text from their training data. More recently, it's been used in research tracking how much AI-written books overlap with rare phrases from existing published works.

Articles use it in phrasing like: "The research team used Ai2's infini-gram engine to examine overlap between AI-generated text and rare phrases from existing publications." One important caveat: infini-gram itself is not an AI detector. Determining whether a given piece of text was written by AI is left to separate detection tools—infini-gram's role is to trace where a flagged phrase actually originated from.

1. Go to infini-gram's public web demo page.
2. Paste the sentence or phrase you're curious about into the search box.
3. Check the results to see how many times, and where, that phrase appears in the reference corpus (the indexed body of text). You'll get more interesting results searching for an unusual phrase—like "fermented sour tang"—rather than a short, common word.