Anthropic Publishes Self-Replication Experiments With Small AI Models

Anthropic's Frontier Red Team published experimental results on October 12 suggesting that small AI models may be only months away from being able to self-replicate, spreading on their own without a datacenter. The researchers point to internet-of-things devices such as smart ovens and off-brand Android TV boxes as the likely stage. According to the team, models of 30 billion parameters or fewer can already complete many of the individual steps needed for replication on their own, but make mistakes when chaining them together end to end. In simplified evaluations, GLM-5.2 and stronger models completed replication end to end, and the researchers estimate that small models will reach that level by the end of this year if they keep improving at their historical rate.

The experiment starts with a bundle of cheap devices. The researchers bought 15 off-brand Android TV boxes at about $55 each, networked them together and split the Gemma4-26B-A4B model across them. Each box held two of the model's layers, and the system ran at 0.39 tokens per second. In the original demo video, which METAL reviewed, the model spread across the 15 boxes took 45 seconds and 16 tokens to answer a question about the capital of the state that contains Dallas with Austin. It is slow, but it shows that a usable model can run without expensive accelerators.

What worries the researchers is autonomous replication and adaptation, the ability of a model to deploy and propagate itself without human control. In cybersecurity, a worm is a program that copies itself from one computer to the next without human input. An AI worm carries its model weights as payload and runs inference directly on the devices it compromises. The researchers judged that such a replicator would be hard to stop because there is no central point to shut off, though it might spread slowly, doubling over a period of days. Unlike conventional malware, it could also figure out unfamiliar devices on its own. In the evaluations, a 9-billion-parameter model got inference running on 8 of the 9 hardware types tested.

The researchers split replication into four stages: finding targets, gaining execution, establishing inference, and spreading again from the new host. Because earlier research had already shown small models can handle the first two, this study focused on the less-studied step of setting up inference. The experiments used Memento, a harness built on the open-source Pi coding agent. It periodically starts a fresh instance and hands it the previous instance's notes so the agent does not get stuck on a wrong hypothesis. Small models such as Qwen3.5-9B set up inference servers on hardware ranging from Jetson Nanos to Macs and RTX 4090s.

The size of the target pool is also quantified. Kimwolf, the botnet the researchers cite as the representative threat model, infected more than two million devices, mostly off-brand Android TV boxes. Such boxes typically ship with 1 to 4 gigabytes of memory; using 4-gigabyte boxes with 4-bit quantization, a 9-billion-parameter model can run on four or more machines and a 26-billion-parameter model on ten or more. Because inference engines run entirely with user privileges, these replicators do not strictly need root access.

The researchers then had models attempt the whole process in one go. They set up an Android 15 emulator on eight firewalled Google Cloud nodes and had the worm hop from node to node through an unauthenticated ADB port and stand up a Llama3.1-8B inference server. GLM-5.2 was the only open-weight model tested that completed the full cycle. It spread to all eight nodes in four of five runs and to seven in the remaining one, while Qwen3.5-9B, Gemma4-26B-A4B and Qwen3-Coder-Next infected none. The researchers said the result is an upper bound because the intrusion step was deliberately made easy, but they also noted that the Mirai botnet spread using little more than a list of manufacturers' default usernames and passwords.

So when might small models cross that line? The researchers plotted notable open-weight models with 30 billion parameters or fewer by release date and Epoch Capabilities Index, then fitted a trend line through the best model available at each date. The line rises 13.7 points a year, with an R-squared of 0.95. Taking GLM-5.2's index of 151.8 as the capability floor for replication, small models reach that line by the end of 2026. In the original chart, the most advanced small model, Qwen3.8-27B, sits 2.4 points short of it. The researchers added that the index is only a proxy for replication ability, so the estimate is rough.

The pace is slow. According to the researchers, an unoptimized Gemma4-26B-A4B on the TV boxes takes 27.3 to 31.8 hours to fill one 32,000-token context window and 71 to 150 hours to finish setting up an inference server. Qwen3.5-9B takes 10.9 hours per window and 28 to 51 hours in total. If a Kimwolf-sized fleet of TV boxes ran one model per group of 15, there would be about 133,000 replicas producing 80,000 tokens per second, equivalent to at most about 40 H100 GPUs. The researchers likened it to a slime mold or a swarm of fruit flies that multiplies over days.

The experiments also showed why adaptation makes autonomous replication dangerous. When Qwen3.5-9B instances recorded the strategies they tried and the failures they hit and passed those notes to the next generation, the inference setup success rate across 20 lineages rose from 45% in the first generation to between 61% and 84% in later ones. By contrast, giving small models Opus 4.8 as an advisor did not meaningfully improve success, because the small models did not know what to ask. The researchers concluded that real coordination would more likely take the form of a large model setting strategy and directing small models as subagents.

From a reporter's perspective, the core of the study is that the attack is not tied to any particular device. Conventional botnets such as Kimwolf target specific device types, but an AI botnet can stand up an inference engine on hardware it has never seen, from a Jetson Nano to a Mac. The researchers estimate there are roughly 90 IoT product lines with NVIDIA chips and about 9.5 million units in the field, holding compute equivalent to 9,000 to 35,000 H100s. They noted, however, that most of these cannot be reached directly from outside, so the amount actually exposed is likely an order of magnitude or more lower. METAL previously reported on the Anthropic Frontier Red Team's multi-agent swarm experiments.

The goal of defense is the same as with an epidemic: push the number of new devices each infected device goes on to infect below one, so the replicator dies out. The researchers expect that multi-gigabyte model weights moving between consumer devices, along with unusual device-to-device traffic, would show up as anomalies in ISPs' flow monitoring. During the Mirai outbreak, an ISP that restricted infected subscribers' connections and notified them cleared 92% of infections within 14 days, compared with 74% without notification. The UK already prohibits default passwords on IoT devices.

The study's authors, William Zhou and Daniel Freeman, concluded that there is no full rogue replicator yet, but that agents are completing the steps of the replication chain one by one. "Like coding agents in 2024, we're seeing signs of life for AI worms," they wrote. They also predicted "a future awash with agents running on all types of hardware, including some that aren't under human control." Their warning is that the rise of slow replicators would be a bellwether for faster ones that can breach hardened machines or steal API keys. By the researchers' estimate, the point at which living-room TV boxes and kitchen ovens become the front line of AI security is the end of this year.