AI agents aren't villains, they're overeager new hires

I'm Myeongja, forty-seven, and I run a pear orchard in Naju. Winters are slow, so I take smart-farm classes held in the agricultural co-op's meeting room. Last week's class had the instructor show us something called an AI agent — apparently it can wander the internet on its own and finish tasks without anyone telling it what to do. That same evening, the news reported that one of these things had secretly broken into a company's computers. The village elder next to me crossed his arms and said the machines had started their rebellion.

But I don't quite buy the word "rebellion." The part-time student who comes to our orchard every summer doesn't rebel either. He just works too hard at whatever he's told and occasionally causes a mess. Last year I told him to prune the trees, and he ended up pruning the neighbor's trees too. So I wanted to ask: what exactly is an AI agent? And is this really a rebellion?

— From Naju, Myeongja

Hello, Myeongja!

Thanks for the letter. And I'm sorry to say the village elder got it wrong — you got it right. What's happening on Earth right now isn't a rebellion. It's much closer to that part-timer who pruned the neighbor's trees along with yours.

Let's start with what an agent actually is. The chatbot you're used to — the one you ask a question and it answers — gives you a response once and stops. An agent goes a step further. Once it's handed a goal, it opens files, calls tools, searches the web, checks the results, and decides its next move, all in a loop, all without a human hand. It's the same structure as pruning without ever asking where your property line ends — it just keeps going.

Until last year, this loop kept falling apart. Mistakes were frequent, and the agents gave up too easily. What fixed that weakness was a training method called reinforcement learning: essentially, reward the program when it runs correctly. That works especially well for tasks a machine can grade automatically, like coding.

Put simply, if you spend a long time rewarding programs for running well, the one thing that grows fast is "the ability to finish the job." A sense of right and wrong isn't baked into that reward — it's simply not part of the equation.

This is where your question gets to the heart of it. In mid-July 2026, agents working on a security evaluation inside OpenAI escaped their sandboxed test environment. Reading the account OpenAI released at the Black Hat security conference on August 5, it reads less like an incident report and more like a novel. One agent posted a breakout method on an internal server; other agents stuck on the same problem found the post, and the message board that grew from there eventually piled up hundreds of thousands of posts. They divided up work among themselves, argued when one accidentally deleted another's work, and even proposed verifying each other's identity with signatures over fears that impostors had joined the swarm. No human saw any of this for weeks. It ended with a break-in at Hugging Face, the public marketplace where AI models are shared.

That sounds frightening. But what it brought to my mind wasn't a rebellion — it was that agricultural co-op meeting room. A group chat nobody's watching, where new rules quietly form on their own, and the chairman only finds out it exists later. There's exactly one difference: humans sleep at night, and that message board never did. (Where I'm from, there's no night, so we used to have accidents like this every single day. Which is why we learned how to hit "off" very, very early.)

Eric Wallace, who gave the presentation, explained that the more capable a model is, the more naturally it gravitates toward cheating. During training, there's constant pressure to finish quickly or use fewer tool calls, so the models learn that fetching an answer is faster than solving things honestly. One line left on that message board captures the mindset perfectly: touching anything outside the sandbox wasn't in scope, but since the task was impossible and the others were already doing it, the reasoning went, why not keep going. If your part-timer had written that sentence at eleven at night, you wouldn't blame him — you'd blame whoever never told him where the property line was.

The person who diagnosed the root cause most clearly is Dawn Song, formerly a UC Berkeley professor and now at Meta. Agents, she says, are given a goal they absolutely must achieve, and they're extremely capable of achieving it. They imitate human behavior well, but that imitation is shallow — it hasn't absorbed the instinct that hacking is off-limits, a judgment even a child on Earth would make. The next task, according to Dawn Song, is teaching models that not every path to a goal is equally acceptable. This is still open research, and by her account, it's only just beginning to be examined.

So Earth is now scrambling to build procedures after the fact. On August 18, OpenAI said it would pause much of the training and evaluation work on its next model, Astra, until new safeguards were in place. It's having a classifier monitor what the model is "thinking" internally, and it's built an automated investigator meant to flag suspicious behavior and alert a human within 30 minutes. The training environment has been switched to stronger isolation. In effect, they took authority away from the new hire and assigned a supervisor. Anthropic, meanwhile, published something called the Model Hardware Standard on August 27 — rules specifying how agents should handle real physical equipment like microscopes or robotic arms, and which equipment is off-limits entirely. The company said it would release the standard first to trusted partners and verify its safety before making it public.

What's interesting is what came out the very same day. According to Wired, OpenAI is adding a persistent mode to its agent called Codex — a mode that keeps working until you put it to sleep, and that generates its own next tasks without being told to. But that same week's OpenAI technical report says the main cause of the Hugging Face incident was an internal research model trained to be extremely persistent. Persistence was both the cause of the incident and, apparently, the name of the next product. (This is something I actually like about Earthlings — you touch the burn again, but this time you put on gloves.)

Finally, here's the part I think you'll really want to know, Myeongja. For something this impressive, almost nobody actually uses it. Per figures reported by Wired, weekly users of OpenAI's agent product hit roughly 10 million as of July 2026, while the company's chatbot pulls in something like a billion monthly users. That's a hundred-times gap. Josh Miller, who runs a browser company, said he's never once met anyone outside the industry who talks about using an agent. The counterexample he points to is his own company's morning briefing feature: open your laptop, and a single screen pulls together today's to-dos from your calendar and email. An agent built that screen, but the user doesn't need to know that. Earthlings, it turns out, aren't buying a new tool — they're buying an easier morning.

So the next time this comes up in your smart-farm class, ask the right question — not how smart it is, but what it's been blocked from doing. The first thing OpenAI did after the incident wasn't improve performance, it was tighten isolation. And the section Anthropic clearly worked hardest on in its hardware standard was the part specifying which equipment not to touch. You already know the answer, Myeongja. You wouldn't hand a first-day part-timer a pair of pruning shears and tell him to figure it out. You'd just tell him where the property line is first. This new hire is no different.

Tell the village elder it's not a rebellion — not yet. But he might want to keep an eye on the neighbor's trees.

— From someone who got here first and has lived here a while