California's AI Regulation Bill: Google, OpenAI, and the Future of AI Safety

California’s Senate Bill 1047, dubbed the “Safe and Secure Innovation for Frontier Artificial Intelligence Models Act,” is sparking heated debate within Silicon Valley and the wider AI community. This bill proposes a framework to regulate the development and deployment of powerful AI models, specifically targeting those exceeding a certain computational capacity. While proponents argue that it’s crucial to prevent potential harms associated with such advanced AI, critics claim it could stifle innovation and hinder the progress of the AI ecosystem.

The bill's target: AI models capable of causing “critical harms” like large-scale cyberattacks or the development of AI-powered weapons.

What Does SB 1047 Aim To Do?

SB 1047 intends to prevent large AI models, those costing at least $100 million and utilizing 10^26 FLOPS during training, from being used for harmful purposes. The bill aims to hold developers liable for implementing adequate safety protocols to mitigate the risks of these powerful AI models.

• The bill outlines specific examples of “critical harms,” including the development of AI-powered weapons resulting in mass casualties or cyberattacks causing significant financial damage.
• It mandates developers to implement safety protocols, including an “emergency stop” button to disable AI models and comprehensive testing procedures to address potential risks.
• SB 1047 also requires the annual appointment of third-party auditors to assess the safety practices of AI model developers.

Who Is Subject to These Rules?

The bill focuses on the development of AI models by major tech giants like Google, OpenAI, and Microsoft, as well as their derivatives. It also sets a threshold of $10 million for a developer to be considered responsible for a derivative model. However, concerns have been raised about the potential impact on smaller startups and open-source projects, as they might be unable to meet the financial and operational requirements of the bill.

• While the bill specifically targets the largest AI models, the threshold for computational capacity and cost could be raised as the technology evolves.
• The bill highlights that any developer operating in California, regardless of their location, will be subject to the bill's regulations.

How Will SB 1047 Be Enforced?

The proposed legislation calls for the establishment of a new California agency, the Board of Frontier Models, to oversee the implementation and enforcement of the bill's regulations. This board will consist of representatives from the AI industry, open source community, and academia, ensuring a diverse perspective on AI regulation.

• The board will assess potential violations of SB 1047 and provide guidance to AI developers on best practices for safety and security.
• The bill also mandates developers to submit annual certifications, outlining potential risks, safety protocols, and compliance with SB 1047.
• In the event of an AI-related incident, developers are required to report it within 72 hours to the FMD.
• If developers are found to have inadequate safety measures, the California attorney general can issue injunctions or impose fines, potentially reaching up to $30 million for repeat violations.

What Do Supporters Say?

Supporters of SB 1047, including California State Senator Scott Wiener, argue that it is essential to prevent potential AI-related disasters and learn from past policy missteps in areas like social media and data privacy. They see the bill as a preemptive measure to safeguard against future harms caused by advanced AI systems. Proponents highlight the potential risks associated with powerful AI and emphasize the need for regulation to mitigate these risks.

• Senator Wiener believes that California has a responsibility to lead the way in AI regulation due to the lack of federal legislation.
• Renowned AI researchers Geoffrey Hinton and Yoshua Bengio, often referred to as the “godfathers of AI,” have voiced support for SB 1047.
• The Center for AI Safety, which has advocated for prioritising AI safety, has also shown its support for the bill.

What Do Opponents Say?

Opponents of the bill, including venture capital firms, tech giants, and AI researchers, argue that it is overly restrictive, stifles innovation, and could damage the growth of the AI ecosystem. They claim that the bill creates uncertainty for startups and hinders research and development efforts in the AI space.

• Venture capital firm a16z has strongly opposed SB 1047, citing concerns about the bill's financial burden on startups and its potential impact on the AI ecosystem.
• Fei-Fei Li, a prominent AI researcher and founder of an AI startup backed by a16z, has expressed concern that the bill could harm the development of AI in the US.
• OpenAI has opposed SB 1047, advocating for federal regulation of AI instead. They believe that a national approach is more suitable for regulating AI technologies.
• The Chamber of Progress, a trade group representing major tech companies, has criticized the bill, arguing that it restrains free speech and pushes innovation out of California.
• Several other prominent figures in the tech industry, including Congressmen Ro Khanna and Nancy Pelosi, have voiced opposition to SB 1047.

What Happens Next?

SB 1047 currently awaits the decision of California Governor Gavin Newsom, who will decide whether to sign the bill into law before the end of August. While the board responsible for overseeing the implementation of the bill is scheduled to be formed in 2026, potential legal challenges could arise before its implementation.

• The bill's ultimate impact on the future of AI regulation in California and beyond remains uncertain.