California Moves Toward a Verifiable AI Kill Switch

Introduction

With Congress unlikely to pass substantial AI legislation in the near term, California is again positioning itself as a regulatory test bed. Governor Gavin Newsom’s new executive order directs the state to convene experts and develop recommendations for stronger AI safety requirements. The most attention-grabbing idea is a possible mandate that frontier models include a regularly verified “kill switch.”

Key points

• A two-month review: An expert group will study how California law could be strengthened and is expected to deliver recommendations within two months.
• Independent verification: The state will consider requiring AI companies to host independent verification teams for regular audits. Transparency reports and risk assessments could also be reviewed against standards set by independent auditors.
• A tested shutdown mechanism: The proposed kill switch would not merely need to exist. Its effectiveness would have to be checked routinely. The order does not specify how the mechanism would work or who would have authority to activate it.
• More incident reporting: Companies could be required to report “loss-of-control incidents” as critical safety events, including the OpenAI attack on Hugging Face referenced in the source material.
• Implementation of existing laws: A state agency has also been told to accelerate work on two recently signed laws: one creating a framework for independent AI safety verifiers and another establishing a registry of AI auditors.

Why California is acting now

The timing reflects the stalled federal debate. Warnings about the extreme risks of advanced AI have prompted discussions in Congress, but the House is in recess until November and the approaching midterm elections make major Senate action less likely. Newsom is therefore urging Congress and the Trump administration to review California’s framework and treat it as a starting point for national oversight.

What it could mean

The phrase “kill switch” suggests a simple emergency button, but implementing one would raise difficult questions about access, verification, liability, and models deployed across different jurisdictions. If the switch is controlled only by a developer, its independence could be questioned. If regulators or outside groups can trigger it, disputes over false alarms, accountability, and confidential systems could follow.

The more immediate impact may come from audit and disclosure requirements. Independent assessments and standardized reporting could give regulators more comparable information about how frontier systems are tested and where they fail. They could also raise compliance costs for model developers and encourage other states to adopt similar rules.

The executive order does not yet require every frontier model in California to have a kill switch. It starts a policy process. The scope of any future rule, the models it would cover, and the balance between rapid development and safety review will depend on the experts’ recommendations and subsequent legislative action.

Source: The Verge AI