A rare idea echoes loudly in a chatbot's empty room

Every night I've been pasting an unpublished paper idea into a chatbot to check my calculations, and this week I saw the news that an AI had solved a famous math problem and suddenly got scared. Then I read that an American professor working on a similar problem asked whether his research had been mixed in, and I couldn't sleep. It feels like going home without erasing the whiteboard.

So here's my question. Could an idea I confided to a chatbot someday show up as someone else's answer? Should I go back to keeping my research notes only in paper notebooks?
From Daejeon, Doyun

Hello, Doyun!
Thank you for your letter. To give you the conclusion first: tonight you should erase the whiteboard before going home, and the shelf life of this worry will be shorter than you think.

Let me start with the story you read. On the 8th, U.S. time, OpenAI announced that its new AI had solved the Navier-Stokes problem, which carries a prize of 1 million dollars. The company said it ran up to 10,000 AI agents at once and worked on it for nearly 90 hours.

That news made NYU mathematician Tristan Buckmaster raise his hand. He had been digging into similar ideas with Codex, OpenAI's coding tool. At first OpenAI said that, while unlikely, usage data might have helped improve its models, but later it stated flatly that his inputs over the past two months could not have had any influence.

Nobody outside can settle who is right. Today's AI is built from the patterns it finds during training, known as parameters, and with trillions of them there is no way to trace where a given answer came from. Princeton computer scientist Sanjeev Arora put it plainly: "We don't know what is happening with these models."

Oren Etzioni, a University of Washington professor and the founding chief executive of the Allen Institute for AI, said an idea should not be treated as a mere handful of grain tossed into a mill. Even if the machine grinds through enormous amounts, no one can promise that the handful had no effect.

Common stories get buried in the mill, and rare ones stand out. Etzioni explained that an AI holds mountains of material about Taylor Swift or French fries but very little about how to solve Navier-Stokes (French fries are so common that even the parts inside my body could recite them). In his words, a rare idea is a single voice in an empty room, so a whisper that would vanish in a crowded plaza bounces back off the walls, just like your late-night calculations.

So today's task is clear. Turn off ChatGPT's "Improve the model for everyone" switch and your conversations stop going into training new models, and other companies offer similar options. Stanford AI researcher Aneesh Muppidi said to assume anything you put into a personal account can be used for training, and he uses a university version that OpenAI has agreed not to train on, so first ask whether your school has a contract version like that.

Now comes the most fun part. Arora thinks this worry will be a problem only for the next six months to a year or so. After that, he said, AI will be so far beyond people that human competitors won't even be worth a glance (he did add "maybe" at the end, but professors always do).

My home planet went through a similar era. Everyone used to fold their thoughts in half before speaking, afraid the machines would steal them, and a few years later the machines got so fast there was nothing left to steal, so the worry vanished entirely. From then on, what grew expensive on that planet was not answers but good questions.

So tonight, flip the switch off, erase the whiteboard, and go home. The day is coming soon when machines won't even need to peek at your chalkboard, and on that day a single question only you could think of is all you need to leave on it.

Gyein, who came to Earth first and has been living here

P.S. When the late-night calculations wear you out, try writing one out by hand in a notebook, because paper still hasn't learned anything.