22603.
(a) Before beginning to initially train a covered model, the developer shall do all of the following:(1) Implement reasonable administrative, technical, and physical cybersecurity protections to prevent unauthorized access to, misuse of, or unsafe post-training modifications of, the covered model and all covered model derivatives controlled by the developer that are appropriate in light of the risks associated with the covered model, including from advanced persistent threats or other sophisticated actors.
(2) (A) Implement the capability to promptly enact a full shutdown.
(B) When enacting a full shutdown, the developer shall take into account, as appropriate, the risk that a shutdown of the covered model, or particular covered model derivatives, could cause disruptions to critical infrastructure.
(3) Implement a written and separate safety and security protocol that does all of the following:
(A) Specifies protections and procedures that, if successfully implemented, would successfully comply with the developer’s duty to take reasonable care to avoid producing a covered model or covered model derivative that poses an unreasonable risk of causing or materially enabling a critical harm.
(B) States compliance requirements in an objective manner and with sufficient detail and specificity to allow the developer or a third party to readily ascertain whether the requirements of the safety and security protocol have been followed.
(C) Identifies a testing procedure, which takes safeguards into account as appropriate, that takes reasonable care to evaluate if both of the following are true:
(i) A covered model poses an unreasonable risk of causing or enabling a critical harm.
(ii) Covered model derivatives pose an unreasonable risk of causing or enabling a critical harm.
(D) Describes in detail how the testing procedure assesses the risks associated with post-training modifications.
(E) Describes in detail how the testing procedure addresses the possibility that a covered model or covered model derivative can be used to make post-training modifications or create another covered model in a manner that may cause or materially enable a critical harm.
(F) Describes in detail how the developer will fulfill their obligations under this chapter.
(G) Describes in detail how the developer intends to implement the safeguards and requirements referenced in this section.
(H) Describes in detail the conditions under which a developer would enact a full shutdown.
(I) Describes in detail the procedure by which the safety and security protocol may be modified.
(4) Ensure that the safety and security protocol is implemented as written, including by designating senior personnel to be responsible for ensuring compliance by employees and contractors working on a covered model, or any covered model derivatives controlled by the developer, monitoring and reporting on implementation.
(5) Retain an unredacted copy of the safety and security protocol for as long as the covered model is made available for commercial, public, or foreseeably public use plus five years, including records and dates of any updates or revisions.
(6) Conduct an annual review of the safety and security protocol to account for any changes to the capabilities of the covered model and industry best practices and, if necessary, make modifications to the policy.
(7) (A) (i) Conspicuously publish a copy of the redacted safety and security protocol and transmit a copy of the redacted safety and security protocol to the Attorney General.
(ii) A redaction in the safety and security protocol may be made only if the redaction is reasonably necessary to protect any of the following:
(I) Public safety.
(II) Trade secrets, as defined in Section 3426.1 of the Civil Code.
(III) Confidential information pursuant to state and federal law.
(B) The developer shall grant to the Attorney General access to the unredacted safety and security protocol upon request.
(C) A safety and security protocol disclosed to the Attorney General pursuant to this paragraph is exempt from the California Public Records Act (Division 10 (commencing with Section 7920.000) of Title 1 of the Government Code).
(D) If the safety and security protocol is materially modified, conspicuously publish and transmit to the Attorney General an updated redacted copy within 30 days of the modification.
(8) Take reasonable care to implement other appropriate measures to prevent covered models and covered model derivatives from posing unreasonable risks of causing or materially enabling critical harms.
(b) Before using a covered model or covered model derivative for a purpose not exclusively related to the training or reasonable evaluation of the covered model or compliance with state or federal law or before making a covered model or covered model derivative available for commercial or public, or foreseeably public, use, the developer of a covered model shall do all of the following:
(1) Assess whether the covered model is reasonably capable of causing or materially enabling a critical harm.
(2) Record, as and when reasonably possible, and retain for as long as the covered model is made available for commercial, public, or foreseeably public use plus five years information on the specific tests and test results used in the assessment pursuant to paragraph (1) that provides sufficient detail for third parties to replicate the testing procedure.
(3) Take reasonable care to implement appropriate safeguards to prevent the covered model and covered model derivatives from causing or materially enabling a critical harm.
(4) Take reasonable care to ensure, to the extent reasonably possible, that the covered model’s actions and the actions of covered model derivatives, as well as critical harms resulting from their actions, can be accurately and reliably attributed to them.
(c) A developer shall not use a covered model or covered model derivative for a purpose not exclusively related to the training or reasonable evaluation of the covered model or compliance with state or federal law or make a covered model or a covered model derivative available for commercial or public, or foreseeably public, use, if there is an unreasonable risk that the covered model or covered model derivative will cause or materially enable a critical harm.
(d) A developer of a covered model shall annually reevaluate the procedures, policies, protections, capabilities, and safeguards implemented pursuant to this section.
(e) (1) Beginning January 1, 2026, a developer of a covered model shall annually retain a third-party auditor that conducts audits consistent with best practices for auditors to perform an independent audit of compliance with the requirements of this section.
(2) An auditor shall conduct audits consistent with regulations issued by the Government Operations Agency pursuant to subdivision (d) of Section 11547.6 of the Government Code.
(3) The auditor shall be granted access to unredacted materials as necessary to comply with the auditor’s obligations under this subdivision.
(4) The auditor shall produce an audit report including all of the following:
(A) A detailed assessment of the developer’s steps to comply with the requirements of this section.
(B) If applicable, any identified instances of noncompliance with the requirements of this section, and any recommendations for how the developer can improve its policies and processes for ensuring compliance with the requirements of this section.
(C) A detailed assessment of the developer’s internal controls, including its designation and empowerment of senior personnel responsible for ensuring compliance by the developer, its employees, and its contractors.
(D) The signature of the lead auditor certifying the results of the auditor.
(5) The developer shall retain an unredacted copy of the audit report for as long as the covered model is made available for commercial, public, or foreseeably public use plus five years.
(6) (A) (i) The developer shall conspicuously publish a redacted copy of the auditor’s report and transmit to the Attorney General a copy of the redacted auditor’s report.
(ii) A redaction in the auditor’s report may be made only if the redaction is reasonably necessary to protect any of the following:
(I) Public safety.
(II) Trade secrets, as defined in Section 3426.1 of the Civil Code.
(III) Confidential information pursuant to state and federal law.
(B) The developer shall grant to the Attorney General access to the unredacted auditor’s report upon request.
(C) An auditor’s report disclosed to the Attorney General pursuant to this paragraph is exempt from the California Public Records Act (Division 10 (commencing with Section 7920.000) of Title 1 of the Government Code).
(7) An auditor shall not knowingly make a material misrepresentation in the auditor’s report.
(f) (1) (A) A developer of a covered model shall annually submit to the Attorney General a statement of compliance with the requirements of this section signed by the chief technology officer, or a more senior corporate officer, that meets the requirements of paragraph (2).
(B) This paragraph applies if the covered model or any covered model derivatives controlled by the developer remain in commercial or public use or remain available for commercial or public use.
(2) In a statement submitted pursuant to paragraph (1), a developer shall specify or provide, at a minimum, all of the following:
(A) An assessment of the nature and magnitude of critical harms that the covered model or covered model derivatives may reasonably cause or materially enable and the outcome of the assessment required by paragraph (1) of subdivision (b).
(B) An assessment of the risk that compliance with the safety and security protocol may be insufficient to prevent the covered model or covered model derivatives from causing or materially enabling critical harms.
(C) A description of the process used by the signing officer to verify compliance with the requirements of this section, including a description of the materials reviewed by the signing officer, a description of testing or other evaluation performed to support the statement and the contact information of any third parties relied upon to validate compliance.
(g) A developer of a covered model shall report each artificial intelligence safety incident affecting the covered model, or any covered model derivatives controlled by the developer, to the Attorney General within 72 hours of the developer learning of the artificial intelligence safety incident or within 72 hours of the developer learning facts sufficient to establish a reasonable belief that an artificial intelligence safety incident has occurred.
(h) (1) A developer shall submit to the Attorney General a statement described by subdivision (f) no more than 30 days after using a covered model or covered model derivative for a purpose not exclusively related to the training or reasonable evaluation of the covered model or compliance with state or federal law or making a covered model or covered model derivative available for commercial or public, or foreseeably public, use for the first time.
(2) This subdivision does not apply with respect to a covered model derivative if the developer submitted a statement described by subdivision (f) for the applicable covered model from which the covered model derivative is derived.
(i) In fulfilling its obligations under this chapter, a developer shall consider industry best practices and applicable guidance from the U.S. Artificial Intelligence Safety Institute, National Institute of Standards and Technology, the Government Operations Agency, and other reputable standard-setting organizations.
(j) (1) This section shall not apply to products or services to the extent that the requirements would strictly conflict with the terms of a contract with a federal government entity and a developer of a covered model.
(2) This section applies to the development, use, or commercial or public release of a covered model or covered model derivative for any use that is not the subject of a contract with a federal government entity, even if that covered model or covered model derivative has already been developed, trained, or used by a federal government entity.