Pangram's AI Detection Scores Rattle Publishing, but Trust Questions Mount

Pangram is a service that estimates, as a probability, how much AI involvement went into a piece of writing. Feed it a text and it returns a percentage score for how likely the content is to be AI-generated — though ironically, AI itself is used in producing that judgment. Founders Max Spero, 30, and Bradley Emi met as students at Stanford. Spero previously worked on FLoC, Google's attempt to replace third-party cookies, a project scrapped in 2022 over privacy concerns; he later worked at self-driving car company Nuro. Emi's background includes stints at Tesla and AI biotech firm Absci. After ChatGPT launched in 2022, the two founded a company called Checkfor.ai in 2023, renaming it Pangram the following year. The market already had at least a dozen competitors — Originality.ai, GPTZero, Turnitin among them — but Pangram rose to the front of the pack after performing well in early independent testing.

The calls that shook publishing

Pangram's name became widely known in January over the self-published novel Shy Girl. After allegations that author Mia Ballard had used AI spread across Reddit and YouTube, Spero ran the manuscript through Pangram and posted on X that it came back 78% AI-generated. Ballard denied the claim, but Hachette canceled the book's publication anyway. Spero has argued his role in the episode was exaggerated, but reporting shows a Pangram salesperson had passed the story to a publishing industry analyst, who in turn brought it to the New York Times, where it became a full story.

Similar accusations kept coming. A New York Times Modern Love column was flagged over AI-writing suspicions and scored 100% on Pangram; a Commonwealth Short Story Prize winner also scored 100%; the novel Daggermouth scored 60%; and the thriller Call Me, I'll Hide the Body, which sold for a $2.4 million deal, scored 97%. In late July, Substack announced it was integrating Pangram into its platform so readers could quickly check how much AI involvement was behind a given piece of writing.

To unpack that a bit: two stories Metal Lab covered back in August both connect to this one. The research tracing plagiarism traces in an AI-written bestseller came from a team led by Stony Brook University's Tuhin Chakrabarty — who now reappears in this story as someone close to Pangram's founder and a recipient of ongoing API credits from the company. And Bradley Emi, who previously explained why ChatGPT's writing style sounds distinctly AI-like — attributing it to post-training safety guardrails — is Pangram's co-founder and CTO.

Work	Pangram score	Aftermath
Shy Girl (Mia Ballard)	78%	Hachette canceled publishing deal
NYT Modern Love column	100%	-
Commonwealth Short Story Prize winner	100%	Full review of winners since 2012 flagged 3 more works
Daggermouth	60%	-
Call Me, I'll Hide the Body ($2.4M deal)	97%	-

Unverified evidence, a pirated manuscript

The investigative project The Drey Dossier reported that the copy of Ballard's manuscript Spero used had come from a piracy site. In an interview with Wired, Spero admitted he had run the manuscript through Pangram without reading it in full. Pangram has since courted more controversy: after a Commonwealth Short Story Prize winner scored high, the company ran every prize winner since 2012 through its system and flagged three more works as suspected AI writing.

How the detection works, and the data behind it

Pangram trains its system by having LLMs imitate human-written text, teaching the model how AI tends to write. The company calls this "synthetic mirroring." It also runs a process it calls hard-negative mining, where false-positive cases are identified, run back through synthetic mirroring, and added to the training set. Spero has emphasized that Pangram's dataset is properly licensed, which is also part of why the product can operate on far less data than something like ChatGPT or Claude. Pangram sells its service into education, legal, and hiring markets too, but creative writing makes up the largest share of its training data.

Reliability concerns and conflicts of interest

Critics raise two main objections: harm from false positives, and bias baked into the machine-learning model itself. Publishing consultant Jane Friedman said there's "a lot of animosity and anger toward AI-detection software." Sam Illingworth, a professor at Edinburgh Napier University, has pointed to bias against particular groups in detection tools. One study found detectors more likely to misclassify writing by non-native English speakers as AI-generated, and neurodivergent writers have reportedly said their writing style gets flagged disproportionately as well. Notably, the three works at the center of this controversy — Shy Girl, Daggermouth, and Call Me, I'll Hide the Body — were all written by authors of color.

The research behind the Daggermouth accusation is also worth a closer look. Stony Brook's Tuhin Chakrabarty ran 14,419 self-published novels through Pangram for a working paper, finding that nearly 20% scored significantly for AI involvement. Chakrabarty is close to Spero and has continued receiving API credits from Pangram, and he is also in a relationship with Todd Shuster, co-CEO of literary agency Aevitas. Shuster himself consults for Pangram and uses it to vet manuscripts and book proposals.

Publishers are treading carefully

Of the Big Five publishers, Simon & Schuster and HarperCollins declined to comment, while Hachette and Macmillan did not respond. Penguin Random House said editors may use approved AI-detection tools as one of several signals for checking AI involvement, but not as a determinative one. In a survey of 1,481 authors last year by Gotham Ghostwriters, 61% said they use AI tools, and 7% said they had published AI-generated text. Regina Brooks, president of the Association of American Literary Agents, said the industry needs to pay close attention to fairness issues here.

Editor's take

Pangram has landed at the center of this fight not because the technology itself is unusual, but because of where it's being applied. A probability score is effectively functioning as the deciding piece of evidence in a decision — a publishing contract — that determines a single writer's livelihood and career. The fact that Spero fed Ballard's manuscript into the system without even verifying it first is the most damning detail here: it shows how loose the process behind a score can be, even as the consequences of that score land immediately and are hard to undo.

Anyone who has watched this pattern play out before will recognize it. When a new verification technology arrives, buzz tends to build trust faster than accuracy does. Something similar happened when Turnitin was first introduced for plagiarism checks, and again when the first spell-checkers appeared — both went through a period of overconfidence. Pangram is squarely in that period now, and it's made more precarious by how tangled the roles of verifier and researcher have become, as seen in its relationship with Chakrabarty.

The lesson for publishers and agencies elsewhere is straightforward: don't treat an AI-detection score as the sole basis for a contract decision — treat it as just one step in a broader manuscript review process. Penguin Random House's stated approach, that detection tools inform but don't determine editorial decisions, looks like the safest posture right now. For individual authors, meanwhile, keeping drafts, revision histories, and creative notes on hand may be the only real defense against being wrongly flagged.

More scandals like this one seem likely in the months ahead. The Substack integration has extended Pangram's reach into the world of individual creators, and Spero himself has said he'd pay a bounty to anyone who could prove a false positive — so far, no one has claimed it. Whether that silence reflects Pangram's accuracy, or simply reflects that writers don't trust they'd have any real avenue to push back, will likely become clearer the next time a case like this breaks.