The Essay That Started the AI Race
| Author | Kevin Roose |
|---|---|
| Published | October 5, 2026 |
| Format | Article |
| Original | Read the original |
The Essay That Started the AI Race

Kevin Roose published this the day before his book The AGI Chronicles came out. It carries the first ten pages of “The Big Blob of Compute Hypothesis,” an internal research memo Dario Amodei wrote in 2017 while he was a researcher at OpenAI, published for the first time.
Summary
Roose’s framing. In 2017 most AI researchers built narrow, specialised models for one task at a time. Amodei’s hunch, drawn from his own experiments (including speech recognition at Baidu), was that the reliable way to make AI smarter was to make general-purpose models bigger: more data, more compute, longer training. Roose notes the idea predates Richard Sutton’s The Bitter Lesson by a few years. The memo became a classic among early OpenAI researchers, and Roose argues it may have started the AI race, because OpenAI scaled up its language models to test it, which led to GPT-2, GPT-3 and everything since. Amodei declined to comment. Roose publishes only the first 10 of 25+ pages; the rest is technical argument with other safety researchers.
He gives three reasons for publishing: it is a historic document (he compares such papers to Darwin’s notebooks); it shows how Amodei thinks and communicates; and it undercuts the theory that AI leaders talk up doomsday risk as cover for regulatory capture, since Amodei was writing about AI safety years before there was anything to regulate. The third reason, stated openly, is to sell the book.
The memo itself. Amodei’s claim is that building any intelligent behaviour is mostly a matter of supplying a large, minimally structured mass of compute and shaping it with a rich environment and the right training process. He names seven things that matter:
- Compute, flowing freely between observation and action.
- Parameters, to store what is learned.
- Quantity of experience (he prefers “experience” to “data”).
- Distribution of experience, varied enough that the simplest way to succeed is a general solution.
- Normalization and conditioning (Adam, BatchNorm, ResNets).
- Symmetries and shape (convolutions, recurrence).
- Training target: you learn whatever the training process rewards, and the wrong target teaches the wrong task.
Get the first six right and the system will usually learn what the seventh specifies. Clever algorithms matter mostly in the negative; it is easy to block the flow of compute and hard to beat the unstructured blob.
His analogy is the snowflake. You do not build one piece by piece. You set the conditions (physics as the training target, raw material as compute, the chamber as parameters, temperature and humidity as normalization and shape) and wait. He credits Ilya Sutskever with the shorter version: “networks want to learn.”
The evidence section runs through cases where structure imposed by researchers lost to scale: hand-built vision pipelines before CNNs, model-based and hierarchical RL collapsing into what one bigger network would do anyway, LSTMs matching tuned simple RNNs, Bayesian methods relabelling the same computation, and generalisation coming from wider training data (training a speech model on five or six accents made it handle unseen ones). He frames human evolution the same way: enough compute, enough loosely structured memory, broad experience and a competitive objective.
He is careful about the limit of the claim. It does not say scaling today’s exact algorithms reaches AGI, only that future algorithmic gains will be simple and that scale will be a large part of the story. The ten published pages end before his sections on safety, which argue that the same picture makes MIRI-style, framework-first safety work operate at the wrong level of abstraction.
Kevin Roose’s post
On the eve of The AGI Chronicles (which comes out tomorrow!) I’m publishing an excerpt from “Big Blob of Compute,” a 2017 internal research document by Dario Amodei, back when he was a researcher at OpenAI.
It’s never been published before, but BBOC is, in my mind, the essay that started the AI race. It described Amodei’s hypothesis that scaling up AI models was the best way, by far, to make them smarter. GPT-2 was an experiment to test the hypothesis, and when it worked, OpenAI went all-in on scaling, which led to the race we have today.
It’s a fascinating historical artifact that deserves to see the light of day. And it’s taken on mythical proportions inside the AI industry, which is now spending trillions of dollars building the “big blobs” that Amodei thought would be necessary to get to AGI.
Enjoy! And please pre-order the book if you do!
@kevinroose on X, October 5, 2026 (US time)
Connections
- The Bitter Lesson: Sutton’s 2019 essay makes the same case for general methods plus compute; Roose says Amodei got there first.
- The AGI Chronicles: Roose’s book, where this memo is one of several previously unpublished documents.
- Anthropic: Amodei left OpenAI to found it in 2020.
- AGI