Why Not Slow AI Progress?

Key claims

• The analogy. Imagine oil companies and environmentalists both counted as the “fossil fuel community,” Exxon as “fossil fuel capabilities” and Greenpeace as “fossil fuel safety,” going to the same parties. That, he says, is how AI safety and AI capabilities relate: two of the biggest safety teams sit inside DeepMind and OpenAI.
• Environmentalism as the template people keep proposing. The most common question he gets is why the rationalist/EA/safety movement doesn’t copy environmental activists: social disapproval of friends who go into capabilities work, and agitation for regulation. He grants that some regulation has “effectively stopped” an industry, citing the EU’s GMO ban and US restrictions on nuclear power (Nuclear Energy).
• The safety movement built the labs. DeepMind was co-founded by Shane Legg (a 2007 PhD thesis on superintelligence) and Demis Hassabis; OpenAI was launched as a company “to make sure AI doesn’t destroy the world”; Anthropic split off from OpenAI as a more safety-conscious alternative and raised $580M, mostly from rationalists and effective altruists.
• Shaming hits the wrong targets. A climate-style campaign would hurt the companies most capable of shame (DeepMind, OpenAI, Anthropic) and pass the lead to firms that care less, like Facebook. His framing: “OpenAI gets superintelligence in 2040? Or Facebook gets superintelligence in 2044?”
• Regulation options, each with a cost. A broad coalition (fairness advocates, anti-tech conservatives, labor, civil libertarians); company-approved Regulatory Capture (“only these three big tech companies can research AGI, and they have to do it really slowly”); data-use limits; or NIST/IEEE-style safety standards that might even speed safe deployment.
• China is the binding constraint. US rules don’t bind China, which treats AI leadership as national security; an arms-control-style deal is “not technically prevented by the laws of physics, but it sounds really hard.”
• Keep the alliance. Labs fund real safety teams and give them model access, and declaring war would sour neutral AI researchers the way “Texas oilmen think of environmentalists.” The case for betrayal is “pretty slim.”
• No one has the plan. AI policy people tell him “somebody else is working on the plan” or “maybe the plan is a secret.”

Notable quotes

“Imagine if oil companies and environmental activists were both considered part of the broader ‘fossil fuel community’. Exxon and Shell would be ‘fossil fuel capabilities’; Greenpeace and the Sierra Club would be ‘fossil fuel safety’ - two equally beloved parts of the rich diverse tapestry of fossil fuel-related work.”

“While there are salient examples of government regulatory failure, some regulations - like the EU’s ban on GMO or the US restrictions on nuclear power - have effectively stopped their respective industries.”

“First, a history lesson: the best AI capabilities companies got started by AI safety proponents, for AI safety related reasons.”

“It’s not just that they’re not fighting AI companies, it’s that they keep creating them and leading investment in them.”

“Suppose that the alignment community, without thinking it over too closely, started a climate-change-style campaign to shame AI capabilities companies. This would disproportionately harm the companies most capable of feeling shame.”

“What if they started thinking of us the way that Texas oilmen think of environmentalists - as a hostile faction trying to destroy their way of life?”

“Until then, we’re all just happy members of the Broader Fossil Fuel Community.”

How it connects

• a certain kind of environmentalist… (tweet) — the November 2026 joke (“ten years to decarbonize … a review process that uses seven of them”) is the other half of Scott’s analogy: environmentalism’s tools (review, regulation) ended up slowing the clean buildout it wanted. Scott names that same power, regulation that “effectively stopped” nuclear, as the thing AI safety people are tempted to borrow.
• AI Safety — the movement whose strategy the post is weighing.
• Regulatory Capture — his “company-approved regulatory capture” route, which he says he hates as a libertarian and would accept as a utilitarian.
• Nuclear Energy and Climate Change — the precedent cases.
• OpenAI, Anthropic, DeepMind, Sam Altman — the labs and founders in the history lesson.
• Meditations on Moloch — Scott’s earlier essay on race dynamics, the same coordination trap this post worries about.
• Scott Alexander · Slate Star Codex