California has passed the first state-level AI safety law in the United States, after seemingly making peace with former opponents of regulation such as Meta and OpenAI. The new law is pared down from the contents of a failed 2024 effort but nevertheless covers a broad range of areas, from forcing AI developers to be more transparent about risks and privacy issues to establishing new protections for whistleblowers.
Several of the AI safety law’s terms are even more stringent than comparable rules put into play in the European Union. But despite seemingly broad support there is still criticism of its expected negative impact on innovation, and from the privacy and security side some note that key regulations have yet to be placed on AI developers.
California AI safety law emerges after initial tech industry backlash
California first began attempting to pass an AI safety law last year, but quickly ran into major blowback from the big developers that call the state home. Both measures were headed up by San Francisco representative Scott Weiner, but Governor Gavin Newsom vetoed the 2024 version before signing this revised attempt.
Newsom called it a more “balanced” regulation, a sentiment that has been broadly echoed by the big AI players in the tech industry. Some of its elements make it surprising to see the likes of OpenAI and Anthropic embrace it (and Meta and others at least stand out of the way this time). Chief among these are transparency requirements that are among the world’s strongest to date, surpassing even the EU’s present standards under its AI Act. The California AI safety law requires that security protocols and plans be disclosed to the public, whereas in Europe it is only necessary to privately submit them to regulatory agencies. Requirements related to disclosure of crimes committed with the assistance of an AI model (such as cyber attacks or fraud must also be reported if human oversight was not present.
Weiner chalked up the success with the second effort to working more closely with the involved AI companies and Newsom’s office, basing revisions on a report issued by the latter earlier this year that collected recommendations from AI safety experts. However, the new AI safety law also saw some cuts that the tech companies liked, particularly as regards safety and oversight measures during the development process and the amount of potential fines for violations.
California AI safety bill could prompt either harmonization or state-by-state fragmentation
California’s AI safety law appears to provide a potential blueprint for the rest of the nation to follow, with the prospect of a federal-level AI bill seemingly as stalled as a data privacy bill at this point. But critics contend that it could also just as well prompt a patchwork of vastly differing state bills, as is currently happening with data privacy.
Some critics point to missing elements; one of those is large fines. The AI safety law does allow for “frontier” AI developers (those spending over $500 million on their models) to be penalized up to $1 million per violation, but the previous version of the bill lowered the spend level to $100 million and made the fines potentially just as high. That prompted much of the resistance from the AI developers, who saw the potential fine totals as an existential threat.
The AI safety law also leverages the existing California Consumer Privacy Act (CCPA), something most states do not have a comparable version of. It establishes that AI systems that create profiles about users and make predictions or inferences about them must treat that output as protected personal information under the CCPA terms. “Neural data” involving brain activity has a special classification as sensitive personal information, with enhanced protections. Californians also gain CCPA rights to access, review and delete this sort of AI-generated information.
There is some partisan political influence in this divide, though bipartisan support for some sort of federal AI safety law remains strong; an attempt by some Republicans in Congress to establish a bill blocking federal AI regulation was soundly defeated 99-1 last year. But in general the Trump administration has signaled it wants to take regulatory shackles off of AI in the name of innovation and out-sprinting China in development, and Newsom is widely expected to be a leading Democrat candidate for president in 2028. The California AI safety law could very well be something he intends to use as a campaign point. The bill has also prompted some Republicans in Congress, such as Ted Cruz, to signal more urgent support for a federal-level bill in the interest of not letting “blue” states and municipalities set and dictate the pace of regulation.
The leading tech industry lobbying group, the Chamber of Progress, also remains opposed to California’s AI safety law despite the watered-down terms. That group contains Amazon, Apple, Google, and Meta.
Mayank Kumar, Founding AI Engineer at DeepTempo, nevertheless sees the new law as a fundamentally positive development: “Responsible AI usually takes a back seat during the initial days of any disruptive technology. During these periods, everyone is focused on pushing for performance. The first time it moves from panel discussions to production is when regulations are enforced. California’s transparency in Frontier Artificial Intelligence Act (SB 53) is a welcome and necessary step, pioneering the push towards the responsible and secure use of AI. Standard bodies like NIST, ISO, etc., have already released frameworks for the secure development of AI systems, and I am confident that this will speed up the enforcement process. This law rightfully treats AI as a critical emerging technology that, for the sake of public safety, must be regulated. Its focus on mandatory incident reporting is particularly crucial, establishing a framework similar to cybersecurity protocols where transparency is key to managing systemic risks and building a culture of accountability. This legislation directly addresses the core “black box” problem that erodes public trust in AI. By requiring developers to be transparent about their safety practices and report failures, SB 53 empowers users and regulators with the knowledge needed to understand the technology’s real-world impact. This isn’t about stifling innovation; it’s about ensuring it proceeds securely. Ultimately, security is the bedrock of a society that can confidently integrate AI. If implemented correctly, this law will not only mitigate risks but also foster an environment where AI can be a more reliable and beneficial tool for everyone, moving us closer to a future where these powerful systems are used ethically and responsibly.”
