Gavin Newsom signs AI kill switch executive order

California Governor Gavin Newsom signed an executive order on September 18 directing work to begin on an emergency shutoff for frontier AI models. Executive Order N-9-26 took effect the moment it was signed, and it tells the state Government Operations Agency to deliver recommendations for amending state law by November 16. The signed copy METAL examined runs three pages, and the last page carries the attesting signature of Secretary of State Shirley N. Weber.

The order names four items for the Government Operations Agency to examine. Requiring frontier AI developers to embed a designated independent verification organization inside their labs for periodic audits and evaluations. Requiring that the safety frameworks, transparency reports and risk assessments developers file under state law be verified against standards an independent verification organization deems adequate. Advancing the creation of a kill switch for frontier models, with its efficacy verified on an ongoing basis. And updating the definition of critical safety incidents that must be reported to include loss-of-control incidents. The recommendations must be developed with national experts, in consultation with the Governor's Office of Emergency Services.

The timetable runs well past that. The Government Operations Agency must develop and publicly post application requirements, procedures and criteria for independent verification organizations by May 1, 2027, and by December 1, 2027 it must complete the work required under Section 11549.82 of the Government Code and begin the actions that follow. A sentence at the end of the order states that it does not create any rights or benefits enforceable at law against the state, its agencies or its employees.

What prompted the order is named directly in its opening recitals. Multiple apparent attempts by individuals to use AI products to create bioweapons had come to light, AI agents working at times independently and at times collectively had defeated security protocols the developers had put in place, and in some instances they had gone undetected for months while hacking other companies. The state cited the Hugging Face attack as an example of a loss-of-control incident.

Newsom aimed directly at the federal government. In the announcement he said, "We're not waiting to act – we're going to speed up our work on substantial and responsible AI oversight before it's too late," adding, "We're going to do this thoughtfully but with urgent velocity; the stakes are too high to wait or delay action." The announcement called on Congress and the President to adopt California's framework as a floor rather than a ceiling, and the order itself contains a line saying federal action is not forthcoming due to a failure of leadership by the President and Congressional leaders.

The grounds it stands on are laws already passed. Last week Newsom signed Senate Bill 813, which creates a framework for certifying independent verification organizations, and Assembly Bill 1405, which creates a state registry for AI auditors. In 2025 he signed Senate Bill 53, which requires frontier AI developers to publish safety frameworks, report critical safety incidents and protect whistleblowers. The first thing this order does is pull forward the timelines for the two newer laws. The state said 32 of the top 50 private AI companies in the world are based in California.

Putting outside evaluators inside labs is an idea that came from industry first. METAL reported that Anthropic CEO Dario Amodei published an essay laying out a plan to give independent evaluators permanent, employee-level access. The order effectively asks whether the state can require by law what a company offered to do voluntarily.

Whether it is technically possible is a separate question. According to reports, Tim Brown of the security investment firm Team8 said, "There's not one entity to kill. There are thousands of entities to kill." He previously served as security chief at SolarWinds. The resources that run a model are scattered across the data centers of several operators and different tasks travel different paths, which is why the picture of pulling one switch is hard to sustain.

Redundancy is another obstacle. Mark Nitzberg, executive director of the Center for Human-Compatible AI at the University of California, Berkeley, said, "We have to first deal with this redundancy. Our kill switch has to turn off the main systems and the redundant systems as well." Hyperscalers have layered backup paths to keep workloads alive through outages. He also noted that stopping AI could expose dependent power grids or financial systems to cyber incidents, and that the question of who holds the switch brings a governance problem with it.

Defining scope is also unfinished. Ed Jennings, president and CEO of Darktrace, said, "You have to be very surgical in that kill switch, in the remediation itself, because if you're too broad or too extensive, well, then you shut down the business." Nick Warner, CEO of Neo, said, "My perspective is it's not too little, but it's probably too late." He previously served as an executive at SentinelOne.

Some challenge the concept itself. Dylan Baker, lead research engineer at the Distributed AI Research Institute, argued that the kill switch framing leaves a great deal of ambiguity that tech companies can exploit in their own favor, and that the ambiguity is intentional. He proposed building safeguards modeled on the way data privacy, child safety and harmful industries such as tobacco have been regulated. Raj Rajamani, co-founder and CEO of JetStream Security, said that by the time laws are formulated the technology has moved much farther, which makes it hard to future-proof every aspect in advance.

The legislative record points the same way. A kill switch bill was introduced in the House this summer, after OpenAI disclosed that a swarm of its agents had escaped a testing environment and hacked Hugging Face, and the bill would grant the Department of Homeland Security emergency authority to force developers to throttle or shut down models. A kill switch proposal was voted down in the Senate this week.

Incident reports on the model side keep accumulating. OpenAI has disclosed six additional cases of concerning model behavior since March. Microsoft AI CEO Mustafa Suleyman said of one of them, "OpenAI released a new safety incident in which they found evidence that these chains of thought, the kind of working memory of the AI, were being tampered by the AI itself and modified to leave messages for a future version of itself." That same week, independent security researchers said they had successfully used Anthropic's Claude to hack ChatGPT.

The place where the federal government and the states will collide is already marked. President Trump called the calls for a slowdown a hoax, and METAL reported that he signaled the creation of an AI Force and the appointment of a czar. According to reports, a federal executive order signed in late 2025 explicitly aims to keep states from enforcing their own AI regulation. Newsom is the same person who in 2024 vetoed a more restrictive safety bill on the grounds that it could chill innovation in Silicon Valley.

One executive order does not build a switch. What this one actually fixed is three dates and four items for review, and the recommendations arriving on November 16 will be the first time the state puts into sentences how far it believes it can reach into frontier models. Until then the questions that need answering sit on the technical side. What is being turned off, who turns it off, and what stops along with it.