One of the longstanding criticisms of “Internet of Things” (IoT) devices, such as household appliances and lights connected to the internet, is that the market has been lax about cybersecurity. This has led to the creation of massive botnets that spread throughout these devices via worms introduced by hackers. Anthropic is now warning that even small AI models are rapidly approaching the capability to hack these devices and use them to self-replicate in a similar way, giving them a path to persistence and exponential growth without needing any access to a data center.
Anthropic believes “AI as smart oven worm” is only months away
The new warning from Anthropic stems from an internal Frontier Red Team experiment using the Gemma4-26B-A4B LLM (a product of Google DeepMind) that is presently available. The researchers networked together 15 inexpensive “off-brand” Android TVs that are commonly available; the general idea was a smaller-scale replication of the capability of the Kimwolf botnet that infected some two million IoT devices and disproportionately targeted similar budget-priced Android TV models that run forks of the Android Open Source Project (AOSP) that have no additional security features added to them.
The key metrics were that the AI models be able to replicate themselves reliably while simultaneously staying at a low enough size ceiling that they can also run on the modest hardware of their target devices. At least for the expected target capability of 4 GB of RAM, the AI models did not yet pass this test. However, they are extremely close. The researchers use the Epoch Capabilities Index as the benchmark for this capability, noting that each of the test models is gaining about 13.7 ECI per year with their new releases. To reach the capability floor to reliably infect and replicate on cheap 4 GB Android TVs, the closest models only need about 3 more ECI while most of those tested are at least within about 12. That means an expected capability breakthrough likely within months.
As the numbers indicate, the AI models would be slow replicators even when they enter this capability window. They would likely take at least 1 to 2 days to replicate to a new device in the near term. However, this would be taking place quietly below user-facing functions without interrupting them. The models could thus replicate reliably and exponentially without raising alarms among everyday users that are largely not cybersecurity-aware and would not be given any obvious signs of a problem with their device.
The other unique aspect to the threat is that the AI models would not only be learning as they go, but able to pass on what they learn to other models in the swarm. This means that they could independently figure out how to breach new IoT devices they have not encountered before, and then pass that information onto the rest of the botnet to make use of. Rogue AI models could both bundle these notes into the new versions that replicate onto new devices, and also call for assistance from a remote “frontier advisor” with greater capability that has taken up position on a server or more capable device.
Justin Beals, CEO & Founder of Strike Graph, stresses that while this is not an immediate threat due to current small model capability, it is a real longer-term threat that could emerge in the wild as early as next year and surprise organizations that are not keeping thorough inventories of their smart devices: “Anthropic tested the hard part honestly. The researchers kept the hacking step trivial, the model ran at 0.39 tokens per second across fifteen $55 TV boxes, and they called their own results an upper bound. The scary headline version of this story will run ahead of the data. Today, a slow worm on cheap hardware is a real but modest threat.”
“Mirai spread years ago by trying default manufacturer passwords, and plenty of those devices still sit on networks with nobody accountable for them,” Beals adds. “A model that can figure out unfamiliar hardware on its own makes that old governance failure more expensive. Most organizations can’t tell you every connected device in their buildings, let alone which ones still run factory credentials. You can’t govern an identity you never provisioned.”
John Strand, Owner of Black Hills Information Security, adds: “The biggest takeaway is that smaller models have many of the same capabilities as cutting-edge frontier models. They just take longer. When you see reports about high-end research models like Mythos, understand that smaller models running on systems with around 128 gigabytes of memory are still in the ballpark. They may not be as good or as fast, but defenders have to be ready for them.”
Small non-connected AI models could become a big headache
As the research stresses, the unique problem of the theoretical small self-replicating AI models is that there is no central point from which to turn them off. If they are not relying on a data center, there is no “kill switch” option to centrally disable them. Botnets made from traditional worms are generally “fixed” by the good guys seizing malicious infrastructure and pushing a patch out to breached devices, but in this case one would be dealing with an independent intelligence moving to avoid that possibility.
While a key aspect of these small AI models is that they do not require a data center to operate, the researchers believe the data centers powering bigger models would be a primary and priority “end game” target for anyone launching an intentional botnet of this nature. The researchers do not believe these small models will achieve meaningful social engineering capability any time soon, but are already quite capable at scanning for vulnerabilities and engaging in more traditional hacking.
The researchers also warn that this is not a case of one of these AI models needing to fit onto one device. The models could split themselves into “shards” that operate from different infected IoT devices, for example one malicious model splitting itself amongst a smart oven, a Roomba-like vacuum and a washing machine (and not all necessarily in the same location). A particular target of interest will likely be new IoT devices that are shipping with built-in AI capabilities and special chips to run them, which would provide malicious small models with substantially more compute power per device.
John Carberry, Solution Sleuth at Xcape, notes that there is a certain irony to the emergence of this threat but that defense is still very manageable: “We spent nearly four decades hardening networks against self-replicating worms only to invite them back in through unpatched conference room displays … Rather than panicking over speculative model autonomy, security leaders must prioritize fundamental architecture: isolate peripheral electronics into segmented subnets, disable unauthenticated management interfaces, and eliminate default credentials before automated propagation becomes viable.”
