The Bitter Lesson

Author: Rich Sutton URL: http://www.incompleteideas.net/IncIdeas/BitterLesson.html One-line: Seventy years of AI research say that general methods which leverage computation win, by a large margin, and that building human knowledge of the domain into a system feels productive in the short run and loses every time over a long enough horizon.

The argument

Sutton’s essay is barely 1,100 words and is probably the single most consequential short document in modern AI. Its claim is simple and empirical: general methods that leverage computation are ultimately the most effective, and by a large margin. The underlying reason is Moore’s law, or rather the generalised form of it — continuously and exponentially falling cost per unit of computation.

The interesting part is his account of why researchers keep getting this wrong, because it is a claim about incentives rather than about intelligence. Most AI research is conducted as if the available computation were fixed. On that assumption, leveraging human knowledge of the domain is one of the only routes to better performance. But over a span slightly longer than a typical research project, vastly more computation becomes available anyway. Researchers chasing an improvement that shows up on a shorter timescale reach for their domain knowledge — and the two approaches, while not necessarily opposed in principle, compete in practice. Time spent on one is time not spent on the other, psychological commitments accumulate, and knowledge-heavy methods tend to be structured in ways that make them harder to scale with more computation later.

He then walks the same story through four domains, and the repetition is the argument:

• Chess. The methods that beat Kasparov in 1997 were massive deep search. This dismayed the majority of computer-chess researchers, who had pursued approaches built on human understanding of the game’s structure, and who argued that brute force had won this once but was not a general strategy and was not how people play.
• Go. The same pattern, delayed by roughly twenty years. Enormous effort went into avoiding search by exploiting human knowledge and the game’s special features; all of it proved irrelevant or worse once search was applied effectively at scale. Learning from self-play mattered too — and Sutton’s framing is that search and learning are the same kind of thing, in that both are ways of bringing arbitrary amounts of computation to bear.
• Speech recognition. The 1970s DARPA competition, where a field of special hand-built methods lost to statistical approaches.
• Computer vision. The same succession from hand-designed features to learned representations.

The lesson is bitter for a specific reason Sutton is careful to name: it is not merely that a preferred approach lost, but that it tells researchers their accumulated understanding of the problem domain — the thing that makes them experts — is the wrong thing to be building into the system. The essay’s positive prescription is that what should be built in are meta-methods: approaches that can find and capture arbitrary complexity, rather than approaches that already contain someone’s model of what the complexity is.

Read in 2026, the essay is both vindicated and contested. It is the intellectual licence for scaling as a research programme, and the scaling-law era is very largely what it predicted. It is also the argument most frequently cited against itself by people who point out that the modern stack is full of engineered structure, and that Sutton’s own later work on continual learning has taken him to positions his most enthusiastic readers do not hold.

Notable quotes

General methods that leverage computation are ultimately the most effective, and by a large margin.

Search and learning are the two most important classes of techniques for utilizing massive amounts of computation.

Archived text

Source page saved locally against link rot — this one matters, since it lives on a personal server with no CDN: ../attachments/the-bitter-lesson/the-bitter-lesson.md

Connections

• The Openness of AI — Kyle’s own essay. The bitter lesson is the structural reason capability tracks compute, which is what makes the openness question a question about access to compute rather than to ideas.
• Compute — the essay’s single independent variable.
• Moore’s Law — named explicitly as the underlying cause, in its generalised form as falling cost per unit of computation.
• Machine Learning and Deep Learning — the chess, Go, speech and vision sequence is a compressed history of how the field arrived at learned representations.
• Artificial Intelligence — the field whose seventy-year record the essay is summarising.