SACRAMENTO — Gov. Gavin Newsom on Friday ordered the formation of a panel to come up with safety regulations for the state’s artificial intelligence companies, including the development of a possible “kill switch” for AI programs that go rogue.
Newsom issued an executive order to convene “a group of world-leading experts” to provide a “guide” for the state to bolster its laws around AI security, according to statement from the governor’s office.
The panel could put consider proposals for independent third parties to write safety plans or for companies to develop an emergency shutoff.
Newsom, who is considering a 2028 run for president, two years ago vetoed a state bill that would have mandated a kill switch.
Some Democratic and Republican leaders around the country are scrambling to come up with safety proposals after employees at AI companies recently warned that the technology could create catastrophe, including ending humanity.
A researcher for AI company Anthropic said he left the company over concerns that AI companies, including OpenAI, are “gambling with our lives” as they race ahead to improve AI that could surpass human intelligence.
The researcher, Jacob Coxon, said in a social media post: “People building AI earnestly believe that it could kill us all by the end of the decade.”
State Sen. Chris Cabaldon (D-West Sacramento), who chairs the state committee overseeing technology issues, told the Times he’ll hold hearings on artificial intelligence this fall. Those meetings were planned before the recent warnings about AI’s dangers, but will be informed by the events of the last few weeks, he said.
He said the state’s rules around AI don’t go far enough.
“We regulate your barber or your sandwich shop more than we regulate [a company] telling us that they may be ending humanity,” he said.
Any new laws could have a disproportionate effect on the global tech industry because many top AI companies are based in California.
A proposed law in 2024 by Sen. Scott Wiener (D-San Francisco) would have required tech firms to be able to turn off the AI models they directly control if things go awry.
Newsom, in his veto message at the time, cited concerns about stifling innovation. He also expressed concern that smaller AI models, which would have been exempt from the law, could also be dangerous.
