Catastrophic Risks in Artificial Intelligence Foundation Models

The Transparency in Frontier Artificial Intelligence Act (Bus. & Prof. Code, § 22757.10 et seq.) was enacted to increase transparency and safety regarding artificial intelligence foundation models. For example, large frontier developers must write, implement, and clearly and conspicuously publish on its website a frontier AI framework that applies to the developer’s frontier models and describes how the developer approaches, among other things:

  • Incorporating national standards, international standards, and industry-consensus best practices into its frontier AI framework
  • Defining and assessing thresholds used to identify and assess whether a frontier model has capabilities that could pose a catastrophic risk
  • Applying mitigations to address the potential for catastrophic risks
  • Identifying and responding to critical safety incidents
  • Assessing and managing catastrophic risk resulting from the internal use of its frontier models

Large frontier developers must also submit to the Office of Emergency Services a summary of any assessment of catastrophic risk resulting from internal use of its frontier models.

Employees responsible for assessing, managing, or addressing risk of critical safety incidents regarding AI foundation models (“covered employees”) may disclose information to the Attorney General, or to other specified entities, that the employee has reasonable cause to believe discloses either of the following:

  • The frontier developer’s activities pose a specific and substantial danger to the public health or safety resulting from a catastrophic risk
  • The frontier developer has violated the Transparency in Frontier Artificial Intelligence Act (Bus. & Prof. Code, § 22757.10 et seq.)

“Critical safety incident,” “foundation model,” “catastrophic risk,” and other terms are defined in Labor Code section 1107. Frontier developers cannot make or enforce rules, regulations, policies, or contracts that prevent covered employees from disclosing this information or that retaliate against them for doing so. (Labor Code, § 1107.1.)

If you are a covered employee and wish to make a report to the Attorney General regarding a frontier developer’s violation of the Transparency in Frontier Artificial Intelligence Act or a fronter developer’s activities that pose a specific and substantial danger to the public health or safety resulting from a catastrophic risk, please complete our online contact form and select the “Catastrophic risks in AI foundation models – employee reports (SB 53)” category.

The Attorney General is required to produce an annual report with anonymized and aggregated information about reports made to the Attorney General from covered employees. (Bus. & Prof. Code, § 22757.14, sub. (d).)