New AI data security guidelines approved by the Cybersecurity and Infrastructure Security Agency (CISA), National Security Agency’s Artificial Intelligence Security Center (NSA AISC), and Federal Bureau of Investigation (FBI) have been issued and are aimed at assisting organizations of all types that intend to (or already) handle AI training data.
The paper goes into detail on a variety of ways that training data may become compromised, including active targeting by attackers. This includes methods such as “poisoning” of scraped sources and targeting of weak points in infrastructure and authentication. It also tackles the concept of “data drift,” or inert data ingested by training models clashing with new input that the AI struggles to make sense of.
Broad AI data security guidance issued to address integrity and trustworthiness issues
The new guidance actually focuses on three main areas of AI data security: the aforementioned data drift and potentially poisoned data, and also risks in the data supply chain. The guidance builds upon the NSA’s existing Deploying AI Systems Securely publication, but adds much more detail specific to addressing potential vulnerabilities.
Some of the recommendations are sound in theory but not exactly practical for large models; for example, the section on mitigating risks in the data supply chain suggests that all ingested data is first screened for “malicious and inaccurate material,” something not really possible at the web-spanning scale that LLMs require to sustain themselves. It is practical for smaller, local and more single-purpose-focused AI deployments, however. Other AI data security tips for the supply chain include the use of metadata “content credentials” that better track and attribute sources of training data, storing in a digitally signed database that allows for access to remove old material when necessary, and documenting any non-temporary additions of new data. For those obtaining models from third party sources, the guidance also suggests only shopping with model providers that can produce a formal certification of data integrity.
The guidance also documents some of the known methods that malicious actors have been seen using to attempt to taint data stores. “Split-view poisoning” is one example and a simple matter of a threat actor obtaining access to a domain that the model trusts as an information source and introducing malicious elements, something that can run as low as the tens of dollars to acquire a domain name that has been left up for grabs. “Frontrunning poisoning” is another, a timed attack aimed at introducing malicious data during a brief window when major public data sources, such as Wikipedia, publish a “snapshot” available for general download. This is the sort of thing that is usually caught and corrected by moderators, but timing it right may introduce the malicious data to the snapshot before it is noticed.
In terms of addressing the “data drift” threat, the guidance advises that this can be differentiated from intentional poisoning attacks by a slow and gradual reduction in accuracy over time rather than sharp spikes of issues. This element is less a matter of AI data security as it is ensuring that models properly keep pace with upstream updates to their training information. However, it can have damaging consequences if not managed such as health care models outputting information that leads to negative patient impact. The guidance includes specific best practices for management of this inflow of data and also suggests regular testing with data quality assessment tools.
AI data security may be underlooked in rush to adopt new technologies
The market may be underlooking AI data security as a factor, under a general assumption that a well-made model will be able to filter out whatever bad elements it happens across. Improved understanding of exactly how these models work and how they must be managed and maintained for proper output is crucial as they become embedded in networks and core operations.
One of those elements of management is secure storage. The AI data security guidance suggests early adoption of a quantum-resistant method of encryption, and turning to NIST for advice on both that and its FIPS 140-3 standard covering certified data storage devices. Another suggested element is handling the intake and storage of personally identifiable information with a “data masking” method that replaces this sensitive (and highly regulated) data with depersonalized identifiers. Secure data deletion must also be considered, something that NIST can once again help out with via its NIST SP 800-88 Guidelines for Media Sanitization.
Any recommendations about AI data security must be taken with a grain of salt if AI was involved in their production, however. Kevin Kirkwood, CISO at Exabeam, points out at least one indication that the research team might have had room for improvement in following their own recipes: “The article guidance by CISA ends with a note outlining that AI was ‘carefully and responsibly use in the development’ of the best practices document. It leads me to question whether the data set that was used to outline the guidance was developed with the same integrity checks, freedom from poisoning and other threats. The guidance also suggests that AI should be training AI. This might be a path that increases the time for the AI model to become viable and available in the market, but also indicates another level of risk that should be considered. This element of the industry is moving incredibly fast and that also introduces opportunities that can be used to hurt businesses and end users.”
