In the past two years there have been two successful breaches of a United States municipal water supply, contributing to a general alarm about the state of security at the nation’s vital utility providers. Two new cyber attack attempts, one in the San Francisco Bay Area and another in Pennsylvania, will likely add fuel to that particular fire. But while these incidents serve as further demonstration of some glaring security holes in utility systems, sensational media headlines may also be stoking an unnecessary level of fear among the general public.
While it is sometimes worryingly easy for hackers to get into these systems, all of these incidents have demonstrated that bypassing mechanical failsafe measures once inside is a much tougher proposition for would-be hackers.
Cyber attacks demonstrate weak network security, strong physical failsafes
Concerns about the security of the country’s water supply began with a 2019 cyber attack on the Post Rock Rural Water District in Ellsworth, Kansas. A former employee responsible for after-hours security used their remote login credentials, which had apparently not been disabled when they resigned, to shut off the cleaning and disinfecting systems used to sanitize drinking water.
The case came to public attention when federal prosecutors filed a grand jury indictment against the former employee in early 2021, just as the city of Oldsmar in Florida suffered a similar cyber attack. An unknown hacker used old remote login credentials that had not been disabled to access the city’s water supply and attempt to raise the levels of cleaning agent in it to a dangerous degree.
In both cases, the attack attempts were thwarted by manual failsafe systems that detect levels of substances that shouldn’t be present in the water; these systems shut down operations and manually notify plant staff who must perform an in-person check.
Water supply security came to the front of the news cycle again as it was revealed that there was an additional attack in mid-January on an unspecified water treatment plant in the Bay Area. Once again, the culprit was an old remote login credential that no one had disabled. A former employee’s TeamViewer account was used to get into the treatment system network and delete programs that are used to clean the water. The facility discovered the hack the following day, changed passwords and reinstalled the software. It reported that none of the affected water made it out to the public.
The FBI also announced that it is investigating two recent attempted cyber attacks on two different water treatment facilities in Pennsylvania. This follows an email from the Pennsylvania Water Action Response Network to its members indicating that two facilities had fallen victim to cyber attacks in which the intruders installed a web shell used for remote control of local devices.
In addition to mechanical failsafes that provide a vital added layer of protection, the decentralized nature of the nation’s water supply also makes it difficult for a hacker to compromise more than one location at a time. But this also means that these plants are not standardized in any way, nor do they tend to be particularly well-monitored by either state or federal government. An NBC News report characterized a good deal of the 50,000 water supply facilities across the country as typically being run by “a couple of old guys who are plumbers.”
Water supply attacks are worrying, but an element of sensationalism is present
Chris Grove, Technology Evangelist at Nozomi Networks, points out that though these cyber attacks are a major cause for concern they are also not necessarily as drastic as some media reports make them sound: “While it’s important to keep an eye on major events, we should also avoid over sensationalized headlines intended to spread fear. Some headlines are taking the action of deleting code and jumping to attempted mass poisoning. Even the facility operator pointed this out. There was not an attempt at poisoning the water supply. That said, this is a stark reminder on how insecure our nation’s water facilities are.”
While it may be much more difficult for a hacker to actually poison a water supply than news stories sometimes make it sound, that does not excuse the sloppy first layer of security present at many water treatment plants. A recent survey by the Cybersecurity and Infrastructure Security Agency (CISA) found that, of the small amount of water supply facilities across the country that opt to receive cyber security assistance from it, 1 out of 10 had a critical vulnerability and over 80% of these were since-fixed software flaws that emerged prior to 2017. This indicates that many facilities are not taking the most elemental step of simply keeping remote access software patched and updated.
CISA and other cybersecurity experts familiar with the situation see the greatest cyber attack threat as being to the smaller water treatment facilities in more rural areas, which often do not have the budget to have a full-time IT professional monitoring them or to bring in consultants to shore up security. The Biden administration recently addressed similar security issues in the energy sector with an executive order, and has announced $111 billion in spending on water utilities as part of a $2 trillion proposed national infrastructure plan. $56 billion of that is earmarked for grants and low-cost loans meant for dated water facilities that need to modernize.
Sam Humphries, security strategist at Exabeam, shared some thoughts on how these facilities might make the best use of any federal funding received for cyber security upgrades: “The need to understand and baseline normal in terms of critical asset/system access is absolutely key in protecting critical infrastructure. Regardless of whether systems in operational technology (OT) environments are air-gapped or not, if there’s a digital route to the system, then it’s at risk. We’ve got to ensure we’re monitoring OT systems far more diligently by capturing all viable log data in terms of access control, system settings and maintenance. Any abnormality — regardless of how small — should be investigated, triaged and managed accordingly. Relying on users alone for the protection of our CNI systems does not (and will not) scale … Working smarter with automation technologies in managing large volumes of data streams, analyzing them for anomalies and reporting risk in real time, is the only way forward for CNI protection. This, in partnership with continued user education in being diligent and applying critical thinking analysis to system activity reports, is critical.”