Anthropic has pitched Congress a draft bill that would let the United States stop an AI model only when it poses a foreseeable, imminent risk of killing or seriously injuring more than 50 people, or causing more than $1 billion in damage. The petition would come from the Commerce Department alone, and the order from one federal judge in Washington, DC. It is the narrowest formal kill-switch design ever put in front of Congress. It is also not law, has no sponsor, and arrived while Congress is out of session. So the real question is not what the draft says. It is which of the six kill-switch designs on the table could actually stop a dangerous model, and why the rest could not.
First, define the switch
"Kill switch" gets used for three different things, and most proposals only do one of them. A release block stops a model from shipping. An API shutdown pulls a model that is already out. Weight deletion or compute seizure removes the thing itself. Almost every proposal on the table aims at the first and weakest of the three: preventing a future release. None of them, enacted or proposed, gives anyone standing authority to walk into a lab and take a dangerous model offline. That distinction decides everything that follows.
The matrix: who can stop a model, and how
| Proposal | Who can pull it | Trigger | Speed | Scope | Status | Verdict |
|---|---|---|---|---|---|---|
| Anthropic draft bill (US) | One DC federal judge, on petition from the Commerce Department only; every other agency is barred | Foreseeable, imminent risk of 50+ deaths or serious injuries, or $1B+ in damages, from bioweapons, model escape, or cyberattack | Months: evaluator assessment, then 72-hour notice to Commerce, then a court case | Blocks release or use of a frontier model | Proposed Oct 16, 2026; no sponsor; not law | Paper tiger on speed, but the only US design with an explicit legal suspension mechanism |
| House FRONTIER Act (US) | Developers must keep shutdown capability; DHS gets emergency authority to order slowdowns | "Catastrophic harm," undefined in public reporting | Days, if DHS authority is used: an order, not a court case | Slow, suspend, or shut down powerful AI systems | Introduced in the House, summer 2026 (reported) | Fastest legal design on paper; unpassed, details sparse |
| Kennedy bill (US) | The developer itself | Undefined | Hours to days, technically | Requires companies to maintain the ability to shut down their own systems | Planned Sept 2026 (reported) | A capability mandate with no trigger; useful only if someone else orders it used |
| EU AI Act systemic-risk rules | No one: the Act gives nobody suspension authority over a model | None; the gate is a compute threshold of 10^25 FLOPs (Regulation (EU) 2024/1689, Art. 51), which triggers duties, not shutdowns | N/A | Evaluations, systemic-risk mitigation, incident reporting, cybersecurity; fines up to 3% of global turnover | Enacted; in force since Aug 2024; GPAI duties from Aug 2026 | Paper tiger as a kill switch; it is a compliance regime, not an intervention regime |
| California SB 53 (TFAIA) | No one: the law creates no halt authority for anyone | "Catastrophic risk" defined as 50+ deaths or $1B+ in damages, CBRN assistance, autonomous crime, or loss of control | N/A | Safety frameworks, pre-deployment transparency reports, incident reporting to Cal OES, whistleblower protections; fines up to $1M per violation | Enacted Sept 29, 2025; in force Jan 1, 2026 | Paper tiger as a kill switch; it documents risk, it does not stop models |
| Seoul Commitments | The lab itself, voluntarily | Lab-defined "intolerable risk" | Hours to days: the lab holds the weights | 16 companies pledged not to develop or deploy models whose risks cannot be mitigated | Voluntary since May 2024; no enforcement | Fastest actual path, weakest guarantee; it works only if the lab wants it to |
| Anthropic Responsible Scaling Policy | The lab itself, voluntarily | Capability gates (ASL levels); pause if safeguards unproven | Hours to days | One company's training and deployment decisions | Voluntary since Sept 2023 | The strongest self-binding design; still a promise, not a power |
Why 50, and why a billion
The two numbers that anchor this whole debate did not come from actuarial science. No public document derives them. They entered statute first in California: SB 53, signed September 29, 2025, defined "catastrophic risk" as a foreseeable risk of 50 or more deaths or serious injuries, or more than $1 billion in damages (bill text via LegiScan; Wharton analysis). A year later, Anthropic's draft used the identical pair. Politico noted the draft's language tracks SB 53. So the bar is political, not technical: 50 deaths is the smallest number that reads as "catastrophic" to a legislator, and $1 billion is the roundest number that reads as "catastrophic" to a budget committee. Both are thresholds of perception. Nothing in the draft explains why 49 deaths would not qualify, or why $900 million would not.
The numbers also reveal what the draft is designed to exclude. Everything below 50 deaths or $1 billion, which covers fraud at scale, biosecurity near-misses, autonomous cyber operations, and election-scale manipulation, sits outside federal blocking authority under this framework. Mother Jones called the threshold "alarming." Charlie Bullock of the Institute for Law and AI called the draft "pretty reasonable," while noting it is weaker than the House FRONTIER Act. Both reactions are really about the same fact: the trigger is a choice about which harms count, and the draft counts almost none of the harms that have actually gone wrong with deployed AI so far.
Three years of proposals, dated
The pattern across three years is consistent: voluntary pledges arrive first and fastest, enacted laws arrive with paperwork instead of power, and proposals with real teeth stall. The Frontier Model Forum (July 2023) and the Seoul Commitments (May 2024, 16 companies) promised self-restraint with no enforcement (Seoul summit coverage). The EU AI Act (in force August 2024) built the most sophisticated oversight machinery in the world, then gave nobody the power to switch a model off: Article 51 sets the 10^25 FLOPs gate, Article 55 piles on evaluations and incident reporting, and fines reach 3% of global turnover, but suspension authority for models simply is not in the text (Regulation (EU) 2024/1689 via EUR-Lex). We compared the seven enacted AI laws themselves on October 12; the enforcement gap we found then is exactly this gap.
The honest verdict
Here is the uncomfortable part. The only kill switch that has demonstrably worked in 2026 did not come from any AI law. In June 2026, the US government directed Anthropic to suspend two models, Claude Fable 5 and Mythos 5, for foreign nationals after a jailbreak bypassed their cyber safeguards, and the models went dark in about 90 minutes. The authority was export control, not AI safety law. The account is single-sourced (reported here, citing The Decoder and CAISI), so treat it as a reported case rather than a confirmed fact. But if even roughly true, it proves the point: the switch that works is the one that already exists, held by a government that does not need a new AI statute to use it.
Measured against that, the designs sort themselves. The voluntary frameworks are the fastest: a lab that holds its own weights can stop a model in hours, and the Seoul signatories plus Anthropic's own scaling policy all commit to exactly that. Satya Nadella's call for an emergency brake on every AI model, which we mapped in October 2026, belongs in this family: industry asking for a brake it would hold itself. But speed without obligation is a promise, and promises have an exit. The enacted laws are the slowest by design: the EU AI Act and SB 53 can fine, investigate, and demand paperwork, and neither can halt a model at all. That is not a bug in their drafting; they were never written as intervention regimes.
Which leaves the two American proposals. The House FRONTIER Act's reported DHS emergency authority is the only design that pairs speed with law: an order, not a court case, against catastrophic harm. It is also unpassed and thinly detailed. Anthropic's draft is the mirror image: the most carefully engineered legal mechanism ever proposed for stopping a model, and the slowest. Evaluator assessment, a 72-hour notice to Commerce, a petition, a judge, while the model's weights sit on a server the government cannot touch. Add the preemption clause, which would override state disclosure and evaluation rules, and the bar on every other agency, and the draft reads as two things at once: a genuine off-ramp for the worst case, and a ceiling on everyone else's authority. That is why the fight over it will not be about 50 versus 49. It will be about whether a kill switch should be a fire alarm that anyone can pull, or a single locked box in one department, opened only by a judge.
Lawsuits, subpoenas, and firings are accountability responses, which we cataloged on October 17; they punish what already happened. A kill switch is supposed to prevent what has not happened yet. By that standard, the map has one design that could act within days and is law (none), one that could act within days and is voluntary (the labs' own policies), and five that would need months, a court, or a new statute to do anything at all. The next dangerous model, if one comes, will be stopped by a lab that chooses to stop it, or a government using powers written for something else. Nobody has built the thing everyone keeps describing.
By ByteHerald Staff. Threshold chart and timeline are ByteHerald originals, built from the SB 53 bill text (LegiScan), Regulation (EU) 2024/1689 (EUR-Lex), the Seoul Frontier AI Safety Commitments, and Politico/Reuters reporting on the Anthropic draft bill. The Tom's Hardware thread that surfaced the draft's details is linked in our research notes. Claims about the June 2026 export-control case are single-sourced and marked as reported.
Comments
Post a Comment