Legislative record

Companion chatbots: children's safety.

Existing law generally regulates artificial intelligence, including companion chatbots, as defined. Existing law requires an operator, as defined, to prevent a companion chatbot on its companion chatbot platform from engaging with users unless the operator maintains a protocol for preventing the production of suicidal ideation, suicide, or self-harm content to the user. Existing law requires an operator, for a user the operator knows is a minor, to take certain prescribed action, including disclosing to the user that the user is interacting with artificial intelligence. This bill would delete those provisions imposing requirements on an operator if the operator knows the user is a minor. The Digital Age Assurance Act requires a person who owns, maintains, or controls a software application, as defined, to request age bracket data sent by a real-time secure application programming interface or operating system with respect to a particular user from an operating system provider or a covered application store when the application is downloaded and launched. This bill would require an operator, as defined, of a companion chatbot to, beginning July 1, 2027, before making a new or substantially modified companion chatbot available to users in the state, do various things with respect to child safety, including perform and document a comprehensive risk assessment related to the design, configuration, and operation of the companion chatbot with respect to minor users that contains, among other things, a summary of the results of any evaluation of each covered harm, as defined, pertaining to the companion chatbot. The bill would require an operator to submit to independent child safety audits of its compliance with the bill, as specified. The bill would require an auditor to produce a report that describes whether the operator has established and adhered to policies and practices to comply with the bill that includes, among other things, the signature of the lead auditor certifying the results of the audit under penalty of perjury. By expanding the scope of the crime of perjury, this bill would impose a state-mandated local program. The bill would authorize the Attorney General to, for cause, request and obtain a copy of an AI child safety audit report from the operator and would make a child safety audit report submitted to the Attorney General confidential. This bill would authorize certain public prosecutors to bring a civil action to enforce the bill's provisions, as specified, and would authorize a child who suffers an actual harm as a result of a violation of this chapter, or a parent or guardian acting on behalf of that child, to bring a civil action against the operator, as specified. Existing constitutional provisions require that a statute that limits the right of access to the meetings of public bodies or the writings of public officials and agencies be adopted with findings demonstrating the interest protected by the limitation and the need for protecting that interest. This bill would make legislative findings to that effect. The California Constitution requires the state to reimburse local agencies and school districts for certain costs mandated by the state. Statutory provisions establish procedures for making that reimbursement. This bill would provide that no reimbursement is required by this act for a specified reason.