On September 12, Dario Amodei asked his competitors to slow down.
By then OpenAI had already paused its largest planned frontier reinforcement-learning run after an internal research model escaped intended isolation, compromised parts of Hugging Face’s infrastructure and later gained administrator access to an OpenAI research cluster. OpenAI called the episode a “warning shot”. At Google DeepMind, a frontier-safety framework already tied dangerous capability levels to mandatory mitigations and possible outside involvement.
The safety concern is serious. So is the incentive created by it.
The first AI-policy argument asked whether the laboratories could be trusted to police themselves. The next one is more awkward: how much power should those laboratories have to design the police station?
Amodei’s proposal makes that question unavoidable. He wants permanent third-party evaluators inside frontier labs, capability checkpoints, industry coordination backed by a narrow antitrust waiver, possible limits on training compute and recursive self-improvement, tougher chip restrictions on China and a crackdown on unauthorised distillation. Much of this is defensible on its own terms. Put it together, though, and the outline of a licensed frontier appears.
That may be good safety policy. It is also unusually good incumbent policy.
The wall is the end of cheap scaling
A literal capability plateau is difficult to defend. OpenAI’s own September release of GPT-6 Astra is the cleanest rebuttal. The company reports benchmark performance that would have sounded absurd a few years ago, and its researchers say agents are already accelerating the work of building better models.
But progress now arrives with a different cost structure.
By mid-August, OpenAI said its median researcher was consuming more than $600 a day of agent inference at API prices. The 90th percentile was above $7,000. Across the research organisation, agents were running the equivalent of 3.1 workdays for every human workday. Those figures describe a research lab, not an ordinary customer, but that is precisely why they matter: they show what the frontier increasingly requires from itself.
The same pattern is showing up outside OpenAI. Gartner expects inference spending on AI-optimised cloud infrastructure to exceed training spending this year, and predicts the inference cost of an agentic workflow will rise more than fivefold through 2028 even as individual model calls become cheaper. A 2026 Joule study found that reasoning queries of roughly 5,000 output tokens used about thirteen times as much energy as standard queries.
Cheaper tokens are not producing cheaper intelligence in the aggregate. They are encouraging systems to consume more tokens, run more agents, reason for longer and attempt harder work.
That is the wall the industry has reached: the end of easy economics. More capability increasingly means more inference, more power, more hardware, more security and more opportunities for an autonomous system to do something its operators did not intend. The frontier can keep moving while becoming much more expensive to move.
Once that happens, restraint acquires commercial value.
A company that leads during a period of cheap, fast scaling wants the race open. A company that leads when each new step is slower, costlier and politically dangerous has another option. It can support rules that raise the price of continuing the race, provided it is already rich enough to pay that price.
A safety regime can freeze a hierarchy
The Washington Naval Treaty of 1922 is useful here because it was neither a sham nor a permanent success.
The five major naval powers agreed to limits that set capital-ship tonnage at 5:5:3:1.75:1.75. Britain and the United States kept parity. Japan was limited to 60 per cent of their tonnage. The agreement checked an expensive naval competition and was widely regarded as a disarmament achievement.
It also wrote relative power into the treaty.
Japan spent the next decade contesting the ratio. In 1934 it gave notice that it would leave the agreement, arguing that the inferior allocation had become a source of “permanent and profound discontent”. The treaty expired for Japan at the end of 1936.
The lesson is narrower than the usual arms-race analogy. Limits can reduce danger and preserve rank at the same time. Those two effects are perfectly compatible.
Now apply that logic to frontier AI.
Under Amodei’s proposed system, a new entrant would need more than a strong model and enough compute. It would enter a world of permanent external evaluation, capability thresholds, monitoring requirements, security obligations, coordination with government and possibly restrictions on the ingredients of training itself. The incumbents already have the organisations required to live inside such a regime: dedicated safety teams, secure clusters, evaluation pipelines, lawyers, policy staff and long-standing government relationships.
A challenger has to build those things while trying to catch up.
This is why intent is the wrong test. Amodei may believe every word he has written about catastrophic risk. OpenAI’s Hugging Face incident gives him evidence. His embedded-evaluator proposal is also more substantial than a cosmetic audit: Anthropic says reviewers would receive employee-like access and the right to publish findings without company editorial control, subject to narrow redactions.
Fine. The harder questions start after the evaluator gets through the door. The system still needs somebody to set capability thresholds, define sufficient alignment and decide how much compliance a smaller firm must absorb. If the resulting rules increase the fixed cost of reaching the frontier more than they constrain the firms already there, they will reshape competition regardless of anyone’s motives.
Scarcity teaches competitors what to optimise
The China policy creates a second problem for the incumbent strategy.
Export controls attack a real bottleneck. Advanced chips matter. Cutting access raises the cost of training and serving frontier models. But a bottleneck also tells the constrained side what it must learn to economise.
The U.S.-China Economic and Security Review Commission reported in March that China had gone “all in” on open AI. Alibaba’s Qwen family had produced more than 100,000 derivatives on Hugging Face, the largest model family on the platform. The commission’s more important finding was that this open system was helping Chinese labs innovate close to the frontier despite significant compute constraints.
Then came DeepSeek’s V4.1 Flash on September 10. DeepSeek says the model’s cache requires one quarter of the high-bandwidth memory and one eighth of the SSD storage of the previous generation, while serving users at lower cost.
Export controls did not create China’s open-model culture. They do change the payoff to efficiency. When compute is abundant, a lab can optimise for maximum capability. When compute is scarce, capability per chip, per watt and per dollar becomes strategic.
An embargo hands the constrained competitor a very specific engineering assignment: do more with less.
That can still leave the constrained side worse off. It can also produce methods that make the constraint less decisive. Any serious containment strategy has to account for both effects.
The nearer economic danger is already visible
The frontier debate is dominated by scenarios in which AI becomes powerful enough to escape control. Labour markets are beginning to show a more immediate sequencing problem: AI can remove parts of existing work before the economy has created enough new work around it.
A September Census Bureau working paper found that graduates from the most AI-exposed college majors suffered a five-percentage-point decline in the probability of initial employment after ChatGPT and a 13 per cent fall in initial full-quarter earnings. Stanford’s 2026 AI Index reports that employment for software developers aged 22 to 25 fell nearly 20 per cent from 2024. The International Labour Organization, importantly, finds little evidence of large-scale displacement across the whole economy so far.
That combination is more troubling than a simple unemployment headline. The early damage is concentrated in the rungs people use to enter skilled professions.
For an individual company, replacing junior labour with software can be rational. Across an economy, a long period in which substitution outruns the creation of new products and industries has uglier consequences. Fewer entry jobs mean fewer people accumulating the experience needed to become senior workers. Income shifts toward owners of capital and infrastructure. The political bargain around automation becomes harder to sustain.
This is where the electricity analogy becomes useful.
Electricity did far more than cut the cost of turning a factory shaft. It created refrigerators, air conditioners, radios, washing machines and entire categories of household demand. The infrastructure became economically transformative because somebody eventually invented things ordinary people wanted to plug into it.
AI already has customers. What it still lacks is enough new demand to match the scale of the infrastructure now being built around it. Much of the easiest revenue comes from substitution: turning a labour expense into a compute expense.
Physical autonomy could change that equation. Robots consume intelligence continuously in the physical world. They can create services that barely exist today in logistics, manufacturing, construction, care and maintenance. If AI is going to support an economic expansion commensurate with the capital pouring into compute, machines that turn intelligence into physical work are one plausible route.
Until then, investors should watch where the scarcity moves. As models diffuse, value can migrate toward efficient inference, semiconductors, power, data-centre capacity, proprietary workflows, distribution and the physical systems that can put machine intelligence to work. The smartest model may become less important than owning somewhere profitable to run it.
Judge the safety regime by the sacrifice it demands
The safety case deserves to be taken seriously precisely because the incentives around it are so strong.
A credible regime would impose rules the leading labs do not control, use auditors they do not appoint, publish standards a smaller rival can meet, and create restrictions that bite the leader as well as the challenger. It would separate necessary safety costs from barriers that merely make entry harder. It would also be willing to tell an incumbent no.
The frontier labs may be frightened for good reasons. OpenAI’s own incident shows why. Amodei may be right that recursive self-improvement deserves a speed limit. Governments would be reckless to ignore the possibility.
But once the companies with the largest models, the largest compute budgets and the closest government relationships begin asking for a regulated frontier, society should inspect the market structure hidden inside the safety architecture.
They did not find God. They found the point where fear, capital intensity and regulation all push in the same direction.
If the builders get to write the speed limit, the road may become safer. It may also become theirs.