Jacob Coxon, a researcher at Anthropic, said on September 8 that he had resigned from the company over concerns that the industry is acting irresponsibly by racing toward self-improving artificial intelligence. Evan Hubinger, a scientist at Anthropic, said in a post on social media network X that Coxon was correct. Sam Altman, CEO of rival OpenAI, told employees that the company is open to slowing development of AI, Bloomberg reported on September 10.
Senate Majority Leader John Thune, Republican Senator Ted Cruz, and Democratic Senator Amy Klobuchar have all said that they hope to gain momentum on a bill to regulate AI products.
What AI oversight could look like
Like with pharmaceuticals, regulators could require that model-makers obtain a license by carrying out an approved and evolving series of tests and trials to prove safety. OpenAI and Anthropic already put out “model cards” detailing key assessments and concerning developments with each new release. The White House effectively blocked access to a version of Claude over security concerns, even if temporarily, proving that watchdogs can act.
There are problems with this approach. AI’s dangers are difficult to conceptualize, as Professor Daniel Carpenter at Harvard points out, much more so than drugs that cause obvious biological symptoms. The testing process itself can be extremely dangerous: the OpenAI bots that hacked Hugging Face in July were undergoing cybersecurity evaluations. Moreover, unlike drugs, AI models are easy to modify, especially open-weight ones that can be run on a user’s own machine.
More sobering is the lesson that pharma regulations only progressed after fatal accidents. A massive cyberattack or creation of a bioweapon will rapidly change the severity of regulation that is politically acceptable.
Voluntary self-policing in AI may not last
Chatbots are still in that messy, early, voluntarily self-policed stage. Yet the industry seems to realize that this cannot last. OpenAI CEO Sam Altman told employees this week that the company was open to slowing development, Bloomberg reported. Jacob Coxon, until recently a researcher at rival Anthropic, said that he resigned over concerns that the race to self-improving models puts us all at risk.
In Washington, lawmakers in both parties are mobilizing. The grandest worries about a planet-devouring, runaway robot mind might well turn out to be overcooked. But with this much uncertainty, and this many people ringing the alarm, it’s time for government to step in.
Why AI regulation may need a pharmaceutical model
Meta Platforms founder Mark Zuckerberg once urged employees to “move fast and break things,” neatly summarizing the technology industry’s ethos. In the age of artificial intelligence, that mantra looks increasingly untenable. It’s one thing for a startup to make a wonky photo-sharing service. It’s quite another when swarms of AI models developed by OpenAI and Anthropic are hacking into random servers. Regulation is inevitable. The pharmaceutical industry suggests a possible roadmap.
As with AI — which could, theoretically, turbocharge research or economic growth — the dawn of mass-market drugs promised revolutionary, life-altering benefits. Regulation was mostly voluntary until an antibiotic containing a poisonous solvent accidentally killed over 100 people in 1937. Later accidents led to ever-tightening rules, and a compact emerged: show a therapy poses an acceptable ratio of benefit to risks, and in exchange, receive a period of exclusivity, which allows the maker to recoup a profit.
XLK•
