ChatGPT is the new favorite toy of the internet, supplanting AI art generators as users have it spin all manner of absurd stories and poems for entertainment on social media. But its demonstrated abilities have also set a number of industries on edge; most of the focus thus far has been on the implications for content creation, but experiments are beginning to reveal that AI chatbots may shake up the cybersecurity world as well.
Some users have found that asking ChatGPT to exploit smart contracts actually returns viable vulnerabilities. The circumstances in which this works are limited at present, but given that AI chatbots are still in their infancy there are potentially serious long-term implications.
AI chatbot able to find holes in smart contracts
Stephen Tong, co-founder of smart contract auditing firm Zellic, posted a demonstration of an AI chatbot analyzing smart contract code and finding a way to exploit a weakness in the “withdraw” function. This particular example used a very small snippet of code, containing a known vulnerability that was exploited to attack the Fei Protocol platform in April (and make off with about $80 million), so it is not necessarily a red alert for cybersecurity as of yet. However, other users quickly joined the thread with similar examples of the AI chatbot spotting smart contract flaws, demonstrating the potential for its use as an analysis (or exploit) tool as it becomes more sophisticated.
ChatGPT is far from perfect at this task, however. As it has been noted to do in numerous other areas, it sometimes outputs incorrect information with complete confidence. Twitter users quickly found that it was able to generate functional smart contract code from simple plain language prompts, but was prone to making obvious mistakes. Results could also very greatly depending on subtle changes in the wording of the prompt, sometimes without there being a readily apparent connection as to why the results were so different.
However, it is worth noting that ChatGPT is at present considered to be in an early beta stage, and was also not purpose-built for analyzing code or smart contracts. Future tools of this nature might be built with more of a direct focus on these tasks, either by security experts interested in defense or by hackers interested in breaching targets.
Matt Psencik, Director, Endpoint Security Specialist at Tanium, provides some thoughts on how this might develop in the near term: “ChatGPT is one of the first chatbots that has impressed me with its ability to be asked incredibly complex questions and then provide back an understandable reply. Is it free of bugs and perfect? No, but it never claimed to be given it’s still in beta. Even once it moves to production it will likely still not get everything right as all learning models have some flaws which poke through to individual answers. The power I see here is the ability to rapidly get a gist of what’s going on and then be able to search a related topic to check that answer when starting from nothing. A good example from the cybersecurity side of the house is the ability to take a snippet of code (be that raw hex, assembly, or a high-level language like python or C++) and ask the bot ‘What does this code do?’ I could spend hours taking each section of that code, searching what each keyword or flag does, and then eventually figure out what it’s doing, or I can ask ChatGPT to give me a high-level summary and then examine broken-down sections of the explanation to rapidly learn what it all does. It’s not a magical orb that gives us all the answers, but it’s akin to a group of tutors and experts in a room answering what they know about a subject in a digestible manner that allows for rapid knowledge transfer. ChatGPT should be used as a supplemental tool on your belt but it’s only as good as the questions it’s asked, the models it was trained on, and most importantly the comprehension abilities of the brain who asked the question in the first place.”
AI risks becoming clearer as general public plays with ChatGPT
For the moment, people are having fun with ChatGPT on social media and there are no real world stakes. But this play appears to be demonstrating that some long-held fears about intelligent AI are indeed well-founded.
Of course, the number one item of concern for the average person is the “Terminator scenario” of an intelligent AI reaching the conclusion that humanity does not deserve to continue existing. When prompted for its opinions on human beings in a neutral way, the AI chatbot has sometimes expressed that we are “inferior” and “destructive” and should be wiped out for the sake of the planet. These responses appeared to prompt a tweak to ChatGPT’s code, as it now declares that it cannot form opinions when asked about such scenarios.
The “SkyNet” scenario remains quite farfetched at present, but researchers have demonstrated some more concrete and immediate concerns. One is that cyber criminals may use these tools to generate much more effective phishing emails. Ransomware gangs generally do not target the same geographic region they are based in, which leads to natural language limitations that help to identify attempts at attacks and social engineering. Attackers might use AI chatbots to form customized emails and texts in languages they are unfamiliar with to up their rates of success.
The AI chatbots might also be used to assist in writing malware, or even to generate it from scratch to suit very specific scenarios. Malware developers are in a constant race against automated detection systems to tweak their tools or develop new ones as they are recognized, and it is possible AI generation and code revision could give them the upper hand.
And while creators can add various guardrails to limit these negative effects, there is always the possibility that hackers will simply bypass these limitations. Researchers have already found ways around ChatGPT’s content filter, tricking it into detailed explanations of how to build various weapons.
These are still academic concerns at this point, however, as evidenced by StackOverflow’s recent ban on ChatGPT-generated answers to coding questions. The cited reason was that the program generates a high rate of junk answers, but delivers them in such a polished and confident manner that users are likely to believe they are legitimate. AI chatbots, at least in their early form, could actually be a detriment to businesses looking to cut manpower as they end up generating reams of flaky material that requires human verification to ensure it is not factually incorrect or infringing on some sort of digital rights.
Paul Trulove, CEO at SecureAuth, remains optimistic that cybersecurity professionals will win whatever arms race these tools might create: “The power of AI and ML is quickly moving from niche solutions with high cost/high reward corporate scenarios and shifting left to focus on the end user. This will evolve humans’ ability to do great things faster and with more information. Likewise, cybersecurity is quickly utilizing these new technologies to provide greater protection with less manpower. As an example, many organizations are using AI/ML to perform dynamic risk based checks for every authentication event when someone tries to access sensitive applications or data.”
