Anthropic’s Draft AI Safety Bill Sets an Extremely High Bar for Government Action

Banner reading ANTHROPIC’S OWN BILL with a terminal window listing the bill trigger

Anthropic has spent two years arguing that frontier AI needs guardrails. On Oct. 16, Politico reported the company has been quietly shopping its own version to Capitol Hill: a draft bill that would let the federal government block a model’s release or use, but only when the model poses a foreseeable, imminent risk of more than $1 billion in damages or of killing or seriously injuring more than 50 people, and only if that risk arises from a biological weapon, the model evading its developer’s controls, or a cyberattack.

The report landed in a week when Anthropic sent policymakers a different message. At the IMF and World Bank meetings in Bangkok, co-founder Jack Clark said an internal company model forecasts AI could lift U.S. output almost 50 percent by 2030, to roughly $44 trillion from just over $30 trillion, and warned the window for regulators is shrinking. A company predicting civilizational-scale growth while proposing intervention only at civilizational-scale harm is asking to be read carefully.

What the draft would actually do

The mechanism is narrow by design. Independent evaluators would register with the Department of Commerce and periodically assess a lab and its frontier models. Labs could pay those evaluators, provided payment is not tied to the outcome. If an evaluator concludes there is an “imminent enumerated risk,” it must report to Commerce within 72 hours of finishing its assessment. Commerce could then ask a judge to halt the model: a federal judge for the District of Columbia could order suspension, and the language bars any other U.S. agency or officer from suspending a model’s development, deployment or internal use. Politico also reported the bill would pre-empt some state AI laws covering public disclosure of risk, third-party evaluation and reporting of critical incidents. An Anthropic spokesperson told Reuters the draft reflects the company’s “Advanced AI Framework” and would give government the “ability to block or deter dangerous deployments of the most powerful models.” The Commerce Department did not respond to Reuters’ request for comment.

<em>Illustration: huanghuajian.com</em>“/><figcaption><em>Illustration: huanghuajian.com</em></figcaption></figure>
<h3>Regulation written by the regulated</h3>
<p>Industry-authored rules are not automatically bad: firms often know where failure modes live better than legislators do. The question is what the drafting choices reveal. The trigger is calibrated to harms that have not yet materialised — biological weapons, loss of developer control, catastrophic cyberattacks — while harms that have, such as fraud and synthetic impersonation, sit outside the enumerated list. Mother Jones argued the threshold is alarmingly high: a model that might kill or seriously injure only a few people, or cause less than a billion dollars in damage, falls below the line where government could act at all. Letting labs fund their own evaluators, even with payment untied to results, also puts the money and the expertise on one side of the table.</p>
<h3>Speed, or a single choke point?</h3>
<p>Concentrating authority in Commerce plus one district court could prevent a dozen agencies from improvising conflicting orders. It could equally be a bottleneck: one agency deciding whether to petition, one judge deciding whether to halt a model, and every other officer of the United States disqualified from acting even in an emergency. Pre-emption of state disclosure and evaluation laws is already a flashpoint; Politico noted that Democrats have rejected similar proposals they feared would neuter rules passed in California, Illinois and New York.</p>
<figure class=The United States Capitol in Washington, D.C. <em>Photo: <a href=
The United States Capitol in Washington, D.C. Photo: Alvesgaspar, CC BY-SA 4.0, via Wikimedia Commons

The voluntary framework as de facto standard

Whatever happens to the draft, the more consequential development may be the normalisation of the framework itself. Even if the bill stalls, “Advanced AI Framework” compliance becomes the reference point procurement officers, insurers and foreign regulators use. Clark’s Bangkok remarks supply the other half: a shrinking window, growing evidence of a “singularity” where AI outpaces human control, and a need for “unusual and serious rules.” His forecast is a marketing document as much as an economic one — Anthropic is preparing a November initial public offering that could value it at up to $2 trillion.

Cabling and connectors behind server racks in a computer room. <em>Photo: <a hre
Cabling and connectors behind server racks in a computer room. Photo: Jemimus, CC BY 2.0, via Wikimedia Commons

What to watch

  • Whether the draft is formally introduced, and whether its trigger thresholds survive a committee markup or get widened to cover fraud and impersonation.
  • How the evaluator registry is designed: who accredits evaluators, who audits them, and whether lab funding becomes the norm.
  • Whether the pre-emption language draws the same Democratic opposition that has met earlier federal proposals, and whether states challenge it.
  • Whether the Commerce-plus-one-judge mechanism is ever used — an unused emergency power suggests a threshold set for show.

Sources: Politico, Reuters, Mother Jones, Reuters via The Economic Times, CNBC

Leave a Reply

Your email address will not be published. Required fields are marked *