Rear view shot of developer sitting in front of monitors showing Codecov supply chain attack

Codecov Supply Chain Attack Remained Undetected For Months and Potentially Affected Major Companies Including Google, IBM, HP, and Others

Hundreds of clients were compromised in the Codecov supply chain attack, investigators told Reuters on the condition of anonymity.

Investigators said that the attackers not only exploited the company but also used it as a launchpad for attacks against numerous customer networks.

Codecov offers an online software testing platform capable of generating code coverage reports and statistics preferred by professional software developers.

The company serves more than 29,000 enterprise clients, including IBM, Google, HP, the Washington Post, Atlassian, GoDaddy, Procter & Gamble, and the Royal Bank of Canada. The firm also works with several open-source communities and start-up companies.

Codecov supply chain attack manipulated Docker upload script

The attackers compromised Codecov’s environment without raising the obvious red flags, the company said. Codecov added that it experienced periodic unauthorized alterations to the Bash Uploader script going back to January 31, 2021.

A customer informed the company of a file fingerprint discrepancy between the Bash Uploader Script listed on the company’s website and that existing on Codecov’s GitHub.

The Codecov supply chain attack was discovered on April 1, 2021, and affected Bash Uploaders such as Codecov-actions uploader for GitHub, Codecov CircleCl Orb, and the Codecov Bitrise Step.

The threat actors exploited an error in Codecov’s Docker image creation process to steal credentials and manipulate its Bash Uploader script by introducing malicious code.

Supply chain attack leaked victims’ sensitive information

The investigators disclosed that the Codecov supply chain attack exposed information stored in the company’s continuous integration (CI) environments.

“The hackers put extra effort into using Codecov to get inside other makers of software development programs, as well as companies that themselves provide many customers with technology services, including IBM,” the investigators told Reuters.

The hackers replaced Codecov’s IP address with theirs allowing them to post customers’ credentials stored in the continuous integration environment to their servers.

They exfiltrated sensitive information such as environment variables such as the PATH variable, username, and the current working directory. They also exfiltrated various security credentials, such as security credentials, tokens, or keys used by various applications.

The attackers used automation to exfiltrate credentials from hundreds of clients’ systems, according to federal investigators.

The compromise could have allowed the attackers to compromise thousands of protected networks.

Codecov said that the attackers gain access to services, datastores, and application code that used the compromised keys and tokens. Similarly, URLs of origin utilizing Bash Uploaders were likely compromised.

The attackers also accessed “additional resources,” including data stored on the compromised networks from various software vendors.

IBM said it was investigating its development environment but didn’t discover any code modifications.

Mitigations implemented to address impact

Codecov said it fixed the issue and notified all Codecov customers via email by mid-April. Affected users should also change their credentials to avoid their platforms from becoming attack surfaces in the Codecov supply chain attack.

The software company also rotated internal credentials and involved a third-party cyber forensics firm to audit its environment. The firm also implemented other undisclosed mitigations to prevent a similar incident in the future.

Investigators compared the Codecov incident to SolarWinds attacks that compromised nine federal agencies and over 100 private entities.

Asaf Karas, Co-founder and CTO of Vdoo, said the Codecov supply chain attack highlighted the “need to verify and scan any 3rd party software artifacts introduced to enterprise networks or applications, especially as part of the build chain.”

He noted that shell scripts weren’t given enough attention making them perfect candidates for exploitation by malicious actors.

“While compromising human account credentials can be protected with multi-factor authentication, non-human account credentials such as the ones involved in the Codecov attack are usually unprotected and often unsupervised,” says Yaron Kassner, CTO of Silverfort. “It is critical to monitor the behavior of non-human accounts for anomalies and restrict their access to prevent these types of attacks.”