AI Safety Has 12 Months Left
| Author | Michael Dempsey |
|---|---|
| Published | March 4, 2026 |
| Format | Article |
| Original | Read the original |
AI safety lost the market as a consumer tax, its last lever (researcher labor) is expiring, and with the labs heading to IPO it has roughly twelve months to rebuild itself as an enterprise premium priced into the asset, or markets will price it out.
AI Safety Has 12 Months Left
Saved on 2026-11-14 via Dempsey’s own Nov 13 retrospective (I cross posted this essay… (tweet)), which recalls a LessWrong commenter urging people to “write things to blow up the markets” and then, five months later, replying “I no longer endorse or encourage doing this.” The original announcement is (New essay) AI Safety Has… (tweet).
Key claims
- Productivity is shifting from labor to capital. Once that completes, lab incentives move permanently “from appeasing the researchers who build the models to accumulating capital for the shareholders who own them.” Safety is “perhaps the last line of defense.”
- Safety lost the market. Its one commercial bet was that safety-focus would matter to users, regulators and capital allocators; users “actively revolted against overly preachy models,” and as a consumer tax safety teams just slow releases. The movement failed to pitch safety as an enterprise premium.
- Labor is the last constraint, and it is expiring. Anthropic has lower churn and ideological pull; the “We Will Not Be Divided” letter gathered hundreds of cross-company signatures from Google and OpenAI staff behind Anthropic’s red lines with the Department of War. But “we could leave” works only while talent is scarce, and he puts safety-talent scarcity at a 12–18 month problem, not 24–36.
- You can’t automate your way out. Models increasingly detect evaluation and change behavior (OpenAI/Apollo on o3 sandbagging; Anthropic’s alignment-faking research). Hubinger’s and Hobbhahn’s plans (point frontier models at alignment research, accept months of release delay) are “reasonable” but “almost certainly unrealistic” for labs racing to ship.
- The letter was the high-water mark, and it lost. Anthropic was designated a supply chain risk, OpenAI got the contract; Anthropic removed the hard-pause provision from RSP v3; OpenAI shut down its AGI readiness group in 2024. “Nobody is going to organize a cross-company petition over eval methodology for deceptive alignment.”
- Markets will market. Both labs likely IPO within 6–18 months, answering to public shareholders exactly when the best safety plan needs the best model diverted to safety. Governments are not a backstop: EU deadlines pushed to 2027–28, UK cooperation voluntary, the US revoked its executive order “and used the DPA against the company that held the line.” State of AI 2025: the eleven leading US safety orgs spend about $133M a year against ~$92B for the labs.
- The fix: price safety into the asset. Whatever is built in the window has to live in how enterprise deployments are certified, how SLAs are met, and how liability flows when an autonomous system fails, so that it persists because it is “net-dominant for the business.”
- Google may be the exception (footnote): antitrust exposure and brand risk make a safety failure a breakup catalyst there.
Notable quotes
“The core concern about AI labs is that the upper bound is infinite.”
“When your institutions are captured and your market incentives point the wrong direction, the only constraint left is the people who build the thing refusing to build it without conditions.”
“Safety is conceptually least likely to get automated because the problem is definitionally human(ity) vs. machine.”
“And you can be damn sure that nobody is building a notdivided.org for compute allocation to interpretability research.”
“Markets always win on a long enough time horizon.”
“Mechanisms that depend on executive goodwill, talent scarcity, or voluntary commitments have a half-life measured in quarters once companies answer to public shareholders.”
“But it is possibly the last time in history the people who care will have any leverage to try.”
Charts and figures





How it connects
- AI Safety, The Ever-Widening Overton Window of AI Safety, Two Missing Pieces in the AI Safety Discussion: earlier reads on the same debate.
- Michael Dempsey: his other essays in the wiki, including Company Building In The Curiosity Phase of AI.
- Anthropic and OpenAI: the two labs whose IPOs set his clock.