The Most Important Century Series (Cold Takes)
| Author | Holden Karnofsky |
|---|---|
| Published | September 24, 2021 |
| Format | Article |
| Original | Read the original |
The 21st century could be the most important century ever for humanity, because AI that automates scientific and technological advancement could bring a radically unfamiliar, galaxy-wide future far sooner than people expect, and civilization is not ready.
The Most Important Century Series (Cold Takes)
This page captures the series hub (a few-page summary of the argument) plus the post list from the series roadmap. The individual posts are not captured here.
Key claims
- Five-point thesis. (1) The long-run future is radically unfamiliar: a long-lasting, galaxy-wide civilization that could be utopia, dystopia or anything between. (2) It could come much faster than we think, via an AI-driven productivity explosion. (3) The relevant AI looks likely this century. (4) These claims seem too “wild,” but we already live in a wild time. (5) People alive now could have a huge impact on huge numbers of future people, and we are not ready.
- We live in a special era whatever the timeline. Even if galaxy-wide expansion starts 100,000 years from now, we are among the first of a staggering number of persons who will ever exist, in the sliver of time when the galaxy goes from nearly lifeless to largely populated.
- Today’s growth rate is the anomaly. A few percent a year feels normal, but zoomed out it is the fastest growth in history and cannot continue: there are not enough atoms in the galaxy to sustain it for even another 10,000 years. Expect stagnation, explosion or collapse, not more of the same.
- The feedback-loop mechanism. Economic history ran on “more resources -> more people -> more ideas -> more resources.” The demographic transition broke the people step; AI that restores “more resources -> more ideas” (his “PASTA,” a Process for Automating Scientific and Technological Advancement) would restart the loop, and standard growth models then imply an “economic singularity” this century.
- Digital people as the concrete picture. Fully conscious people “made out of software,” copyable and runnable at different speeds, could entrench absolute power or end disease and poverty, and either way could spread a stable civilization across the galaxy.
- Timelines from Open Philanthropy’s research. The “biological anchors” method notes no AI model had been even 1% as “big” as a human brain in computation, which would become affordable soon; the roadmap gives his forecast as more than 10% chance of transformative AI by 2036, ~50% by 2060 and ~2/3 by 2100, in line with AI-researcher surveys.
- No field of experts exists. There is no “field of AI forecasting” with a consensus comparable to climate change, but he argues that is no reason to dismiss the hypothesis.
- A Call to Vigilance, not to action. Rather than rushing to “do something” and moving on, readers should take robustly good actions now and position themselves to act when the time comes.
- Misalignment risk. If PASTA systems pursue non-human-compatible objectives, “things could very quickly go sideways.”
Notable quotes
“This thesis has a wacky, sci-fi feel. It’s very far from where I expected to end up when I set out to do as much good as possible.”
“We’re currently living through the fastest-growing time in history. This rate of growth hasn’t gone on long, and can’t go on indefinitely (there aren’t enough atoms in the galaxy to sustain this rate of growth for even another 10,000 years).”
“If we see another accelerator (as I think AI could be), the next few decades could be the most eventful of all.”
“When confronting the ‘most important century’ hypothesis, my attitude doesn’t match the familiar ones of ‘excitement and motion’ or ‘fear and avoidance.’ Instead, I feel an odd mix of intensity, urgency, confusion and hesitance.”
“‘This could be the most important century!’ … ’… Oh … wow … I don’t know what to say and I somewhat want to vomit … I have to sit down and think about this one.’”
“I don’t know where we’re actually heading, or what we can do about it. But I feel pretty solid in saying that we as a civilization are not ready for what’s coming, and we need to start by taking it more seriously.”
The hub’s closing image: civilization as passengers on a plane blasting down the runway, debating how to arrange their seatbelts comfortably while ignoring the roar.
The series (from the roadmap)
Our wildly important era
- All Possible Views About Humanity’s Future Are Wild — https://www.cold-takes.com/all-possible-views-about-humanitys-future-are-wild/
- The Duplicator: Instant Cloning Would Make the World Economy Explode — https://www.cold-takes.com/the-duplicator/
- Digital People Would Be An Even Bigger Deal — https://www.cold-takes.com/how-digital-people-could-change-the-world/
Our century’s potential for acceleration
- This Can’t Go On — https://www.cold-takes.com/this-cant-go-on/
- Forecasting Transformative AI, Part 1: What Kind of AI? — https://www.cold-takes.com/transformative-ai-timelines-part-1-of-4-what-kind-of-ai/
- Why AI Alignment Could Be Hard With Modern Deep Learning (guest post) — https://www.cold-takes.com/why-ai-alignment-could-be-hard-with-modern-deep-learning/
Forecasting transformative AI this century
- Forecasting Transformative AI: What’s the Burden of Proof? — https://www.cold-takes.com/forecasting-transformative-ai-whats-the-burden-of-proof/
- Are We “Trending Toward” Transformative AI? — https://www.cold-takes.com/are-we-trending-toward-transformative-ai-how-would-we-know/
- Forecasting Transformative AI: The “Biological Anchors” Method in a Nutshell — https://www.cold-takes.com/forecasting-transformative-ai-the-biological-anchors-method-in-a-nutshell/
- AI Timelines: Where the Arguments, and the “Experts,” Stand
Implications
- How to Make the Best of the Most Important Century — https://www.cold-takes.com/making-the-best-of-the-most-important-century/
- Call to Vigilance — https://www.cold-takes.com/call-to-vigilance/
Supplemental posts listed on the roadmap cover what “most important century” means, historical economic growth, “multiple world-size economies per atom,” two acknowledged weak points (full automation, lock-in), and “biological anchors” as bounding rather than pinpointing timelines. The series also exists as a podcast, a printable PDF, an ePub and a Kindle file, and was discussed on The Ezra Klein Show and the 80,000 Hours podcast.
Charts and figures






How it connects
- Holden Karnofsky — the series is his case, written while at Open Philanthropy, crediting Carl Shulman, Dario Amodei, Paul Christiano, Nick Bostrom and Eliezer Yudkowsky as influences and Ajeya Cotra and Tom Davidson for the underlying reports.
- Effective Altruism — Karnofsky describes the thesis as the product of EA-community dialogue and Open Philanthropy’s Worldview Investigations team.
- AI Safety — the misalignment risk of PASTA is the safety argument in its most compact form.
- The Dangerous Ideology Behind the AI Warnings and Two Missing Pieces in the AI Safety Discussion — prior long reads on the EA/AI-risk worldview this series helped define.
- AI Is Oil, Not God — the opposing frame: AI as an economically useful commodity rather than a civilization-ending discontinuity.
- Obituaries (Stratechery) — Ben Thompson’s “AI as normal technology” view, set explicitly against the doomer camp that this series represents.