News has recently emerged that the United States Census Bureau was compromised by a cyber attack in January 2020, just as the decennial census efforts were ramping up. The attack was traced back to a Citrix vulnerability that had just been publicly disclosed about a month prior.
The Census Bureau says that the attackers did not access any 2020 census information; census takers generally begin going out as the winter season calms down in April. The Citrix vulnerability opened a path for the attackers to remotely execute malicious code, but the government report indicates that they were only able to breach the internal network used to manage the agency’s remote workers before the backdoor was discovered and removed.
Previously undisclosed cyber attack capitalized on Citrix vulnerability disclosed in December 2019
The servers breached in the cyber attack were reportedly for facilitating remote access for the agency’s regular remote workers, and not for storing or handling census data. According to Census Bureau officials, these servers had no connections to the Decennial Census network used to record and process information from field workers. Additionally, census counting did not begin until early April.
The Office of the Inspector General (OIG) nevertheless took the Census Bureau to task for the incident in a harsh report, criticizing its failure to mitigate the known Citrix vulnerability and to maintain appropriate system logs that would have been helpful in the follow-up investigation. The report also noted that the Census Bureau was using out-of-date systems that were no longer supported by vendors.
The cyber attack occurred on January 11, 2020 and was not discovered by the security team until January 28 (though it appears that the automated firewall blocked communications with the attacker’s command and control servers on January 13). The initial public disclosure of the Citrix vulnerability (labeled as “CVE-2019-19781”) was on December 17 2019, at which time Citrix Systems issued mitigations to address the issue and alerted customers. Additionally, the National Institute of Standards and Technology (NIST) issued a warning about the Citrix vulnerability on December 31 and marked it as having a “critical” severity rating. The first proof-of-concept exploit code was published on Github just one day before the cyber attack on the Census Bureau servers.
The Cybersecurity and Infrastructure Security Agency (CISA) appears to have noticed the cyber attack before the Census Bureau team did, issuing a notification of a suspected breach and a request to investigate on January 16. However, it would take the bureau’s IT team nearly two weeks to run a script and confirm the breach. Had the firewall not limited the cyber attack to a compromise of several user accounts before remote access was cut off, the attackers would have had ample time to probe and escalate. The long investigation delay also led to key system logs being replaced or removed by the time staff got to them, and in some cases the OIG said that devices were not even keeping the appropriate logs to begin with. Some devices were also found to be attempting connections to a Security Information and Event Management (SIEM) platform that had been taken out of service a year prior.
The threat actors attempted to deploy ransomware during the cyber attack, with Sodinokibi and Ragnarok ransomware payloads found on the servers. It is not clear who the culprit was, but given that the attack came only a day after code for exploiting the vulnerability became available it was most likely an organized ransomware gang. The DoppelPaymer gang was observed exploiting the same Citrix vulnerability in February to penetrate a French telecommunications and hosting company.
Investigation revealed worrying issues
While the Bureau did ultimately stop the cyber attack without any known damage, even if it was done inadvertently by automated security tools, the follow-up investigation revealed a worrying amount of endemic weaknesses just waiting to be exploited by an opening such as the Citrix vulnerability.
While the firewall was able to stop the attack, it was not configured to perform real-time analysis of suspicious activity. A follow-up investigation in February found that the equipment that was past its end-of-life date was still in service. The Bureau said that it was depending on Citrix engineers to migrate to new servers, but that there was a backlog as these engineers were also performing similar work for numerous other federal departments. The follow-up investigation also found that the Bureau did not conduct a “lessons learned” meeting or have any kind of inter-department communication about the cyber attack after it was discovered as its response policy does not have any requirements of this nature.
Chris Clements, VP of Solutions Architecture for Cerberus Sentinel, points out that all of these elements would have been non-factors had a proper patching schedule been in place to remediate the Citrix vulnerability once it had been disclosed: “Patching for network appliances often lags significantly behind other common technology components. Most organizations have patching for Windows completed promptly, for third party software like PDF readers less often, and network appliances even less so. Many factors can contribute to this including lack of a regularly scheduled review of appliances, failure to assign a role in the organization as responsible for patching, and lack of visibility into such systems that may be affected by security vulnerabilities. Windows is infamous for pestering users to install the latest patches, but that’s rarely the case for network appliances. Detection is another area where organizations often struggle to identify that an attack is occurring … while services for outsourcing detection via SIEM or SOC as a service exist, they are often priced based on the amount of information processed and this can create a perverse incentive to not include certain systems into the monitoring to reduce costs but creating blind spots in an organization’s detection capabilities.”