A recent surge in phishing messages by profit-seeking criminal hackers has been tied to OpenAI tools, but a new report from Microsoft finds that state-sponsored hacking groups are making of use of these new AI abilities as well.
Microsoft’s report finds that the balance of AI use is currently in favor of the attackers, with the cybersecurity world struggling to leverage generative AI to keep up with these new capabilities. The report is something of an advertisement for Microsoft Threat Intelligence’s defensive platform and partnership with OpenAI, but it also provides a frank and useful assessment of how state-sponsored hacking groups have been making use of OpenAI tools since they became available to the general public in late 2022.
China, Russia, North Korea and Iran all observed leveraging OpenAI tools
Specific state-sponsored hacking groups from various nations, some of them very familiar faces, have been named as known users of OpenAI tools and other large language models (LLMs).
Heading up the list is “Fancy Bear,” or “Forest Blizzard” under Microsoft’s new naming schema state-sponsored hacking groups. The Russian group has made itself infamous for election interference in a number of different nations over the past decade, and is believed to presently be focusing on targets in Ukraine as the invasion continues on. Microsoft says that the group has used OpenAI tools to perform technical reconnaissance against satellite and radar targets, and to optimize basic scripting techniques to be deployed after it cracks into a target network.
Another of the state-sponsored hacking groups that has been spotted is a North Korean team that has been active for a similarly long period of time and that is known for specializing in “think tank” and academic targets. “Emerald Sleet” or “Kimsuky” has been known to mix confidence schemes with its hacking attempts, approaching foreign policy experts while posing as peers in the hopes of extracting private insights. The group has been observed using OpenAI tools both to scout out new contacts of this nature, and to polish communications with them. But it also makes use of it for hacking purposes, conducting research on publicly disclosed vulnerabilities and using it for post-breach scripting in a manner similar to Forest Blizzard.
Iran’s contribution to the list of state-sponsored hacking groups is Crimson Sandstorm, a threat actor that has been active since 2017 and that attacks a very broad array of targets (from international NGOs to prominent feminist activists). This group focuses on using OpenAI tools to polish phishing messages and generate pieces of code, in addition to the post-compromise scripting activities the other groups showed interest in.
China has at least two of its many state-sponsored hacking groups making use of OpenAI tools: Charcoal Typhoon and Salmon Typhoon. Charcoal Typhoon is also known as “CHROMIUM” or “RedHotel” and has been active since at least 2021 targeting US government entities as well as telecoms firms, media outlets and universities. Analysis published in mid-2023 found it to have become the most prolific and active of China’s attack groups, and it has been confirmed to have breached an unnamed US state legislature in 2022. Salmon Typhoon also targets government organizations as well as defense contractors, and has a particular interest in cryptographic technology. Both groups have used LLMs for reconnaissance, scripting and operational command techniques; Charcoal Typhoon additionally uses OpenAI to assist in translations of its phishing and social engineering communications.
State-sponsored hacking groups have accounts banned from OpenAI upon discovery
OpenAI monitors use of ChatGPT and other tools and puts ever-evolving guardrails on them, and the company has said that it has banned all of the accounts known to be associated with state-sponsored hacking groups. However, given the ease of access to new accounts that is likely only a short-term disruption to these attackers.
Though state-sponsored hacking groups seem to be ahead of the pack in use of OpenAI tools, the company nevertheless stated that the current capabilities with GPT-4 are “limited” and “incremental,” serving mostly to speed up or polish existing activities. However, Microsoft also said that these are not the only groups making use of LLMs for hacking purposes, just a representative sample of some of the biggest espionage-focused threats.
Roger Grimes, Data-Driven Defense Evangelist at KnowBe4, notes that the report contains nothing surprising but that cybersecurity teams nevertheless need to take note of how these capabilities develop: “I don’t think there is any shocking news here. In fact, an important related point is that, for sure, the U.S. is using advanced AI-driven tools to help with their offensive hacking capabilities. Every sufficiently-capable nation is using AI for offensive hacking. They would be remiss if they didn’t. The only questions are how has that increased the success of the various players and how much more successful are AI-driven tools in defending against every type of cyber attack including AI-driven attacks? The future of cybercrime is a competition of the bad actor’s bots against the good actor’s bots and the best algorithms win.”
Gerasim Hovhannisyan, CEO and co-founder of EasyDMARC, adds: “Microsoft’s discovery is a reminder of the evolving nature of cyber threats. The use of AI to automate and scale attacks allows for a level of persistence and sophistication that’s challenging to combat. However, it is important to remember that this doesn’t signal an irreversible advantage for cybercriminals. Instead, it underscores the importance of investing in advanced security measures. By harnessing new technology in the same way hackers do, companies can develop advanced systems capable of combatting threats before they can cause damage. It’s no longer sufficient to have static defence solutions – cybersecurity must be proactive and continuously adapting, complying with regulations and staying up-to-date with which evolving security solutions will prove most effective in combatting modern threats.”
Joseph Thacker, principal AI engineer and security researcher at AppOmni, believes that the key to the whole dynamic is how capable LLMs and similar tools end up being in developing custom malware: “Threat actors that are effective enough to be tracked by Microsoft are likely already proficient at writing software. Generative AI is amazing, but it’s mostly helping humans be more efficient rather than making breakthroughs. I believe those threat actors are using LLMs to write code (like malware) faster, but it’s not noticeably impactful because they already had malware. It’s possible they’re able to be more efficient, but at the end of the day, they aren’t doing anything new yet. However, if a threat actor found a novel attack use case, it could still be in stealth and not detected by these companies yet, so it’s not impossible. I have seen fully autonomous AI agents that can “hack” and find real vulnerabilities, so if any bad actors have developed something similar, that would be dangerous. And open source models like Mixtral are high quality and could be used at scale in novel ways.”
“Companies should remain vigilant. Keep doing the basics right. Bad actors will likely be able to deploy malware at a larger scale or on systems they previously didn’t have support for. LLMs are pretty good at translating code from one language or architecture to another. I can see them converting their malicious code into new languages they previously weren’t proficient in. Allowing AI systems to make decisions is convenient. That means many products will incorporate it without adequate security testing. We will see where this leads very soon,” added Thacker.
Mark Campbell, Sr. Director at Cigent, sees the biggest present-moment risk as being the improvements that OpenAI tools can make to phishing messages: “At the end of the day nothing really changes for security professionals. Phishing, whether human or AI generated, is still the leading cause of initial access. Cyber security professionals need to keep systems up to date and deploy advanced endpoint security solutions that include AI and behavior analysis, to more effectively detect and block malicious activities, including those initiated by AI generated phishing emails.”
