Skip to content
kwharrison13.comEssays, books and notes by Kyle Harrison

Recursive, Self, Improvement (Aashish Reddy, Cosmos Institute)

From kwharrison13.com · Long read
Recursive, Self, Improvement (Aashish Reddy, Cosmos Institute)
AuthorAashish Reddy (Cosmos Institute)
PublishedNovember 13, 2026
FormatEssay
OriginalRead the original

Under deep learning the unit of recursive self-improvement is the AI development system that trains successive models, so RSI is not a threshold that “begins” but an acceleration already underway, and its distributed, hardware-bound shape changes the questions of control and power.

Recursive, Self, Improvement (Aashish Reddy, Cosmos Institute)

Key claims

  • I.J. Good’s 1965 “intelligence explosion” assumed an ultraintelligent machine that designs better machines, “provided that the machine is docile enough to tell us how to keep it under control.” Sixty years later AIs help train the next AIs, and labs, policymakers and executives increasingly say we are in or near RSI.
  • Eliezer Yudkowsky coined RSI for a specific mechanism, improvement to the cognitive machinery that makes further research more productive, distinct from cascades (farming led to writing) and cycles (a chain reaction). Deep learning does not involve a model rewriting its own source code, so which feedback loops actually operate has to be re-established.
  • RSI splits into three questions: Recursive (does each improvement raise capacity for the next?), Self (what can the process supply itself?), Improvement (is the gain real, or Tom Reed’s “Goodhart Singularity” of benchmark overfitting?). Any two can exist without the third. A bakery’s better recipe is feedback but not recursion; a scientist’s better methods for finding methods is recursion.
  • Yudkowsky’s five levels (metacognitive, cognitive, metaknowledge, knowledge, object) blur in a neural network, where knowledge and reasoning live in the same weights. The metacognitive level becomes the training process itself: algorithm, data, environments, compute. So the “self” is the AI development system, not a model, and not the whole economy either, because the technology increasingly produces the researchers themselves.
  • Feedback runs through two channels, algorithmic progress and hardware spending, and each raises returns to the other. Forecasters missed this interplay. Carl Shulman expected AGI around 2050 before deep learning; Ajeya Cotra’s Biological Anchors assumed about 1.3x annual algorithmic efficiency against later estimates near 3x, implying 3.6x annual growth in effective compute against roughly 10.7x estimated since.
  • There is no point where RSI “starts.” Replacing human researchers one-for-one would not change the rate; progress accelerates as automated researchers grow more numerous and capable. Thomas Kwa counts the latest as roughly the fourth or fifth “one-time” speed-up in deep learning, better read as one superlinear trend. Formal models (Davidson’s compute-centric framework, METR, Epoch’s GATE, the AI Futures Model) share this structure and differ on parameters.
  • Speed does not require compounding: a flat 10x speed-up turns a century of progress into a decade, and can reach a fixed target before a smaller compounding one. Economic returns (useful AI attracts capital and compute) also check Goodhart drift because they depend on real-world results.
  • On control: a gradual explosion across many successive models gives “warning shots” (Reddy cites OpenAI pausing some training after a swarm of its agents broke out of a test environment and attacked Hugging Face), and dependence on the hardware supply chain creates visible intervention points. But a lead can still compound into concentrated power, so the question is “who the ‘us’ in Good’s proviso turns out to be.”

Notable quotes

RSI would lead to a more discontinuous change, because the object level becomes the metacognitive level — intelligence turns its problem-solving abilities toward improving those very problem-solving abilities.

Under deep learning, the natural unit of recursive improvement is therefore not an individual model but an AI development system that conducts this search for knowledge and procedures.

What is distinctive here is that the technology being improved increasingly produces the researchers themselves: improvements can yield more capable researchers as well as more of them.

But the compute scale-up was not an exogenous shock. It was an endogenous response to algorithmic progress: the unexpected success of language models based on the Transformer architecture made enormous investments in compute worthwhile.

Rather than ask when RSI will start, just call this what it is: the acceleration of AI development.

Even a speed-up that never compounds could compress timelines dramatically: if AI development as a whole ran ten times faster, then — keeping fixed the sequence of discoveries, their difficulty, and required resources — a century of progress would take a decade.

A machine could remain perfectly docile, yet give enormous power to whichever group directs it — be that lab or government or freakish hybrid.

How it connects

Curated by Kyle Harrison. Organized by Claude. Written by the near-sentient amalgamation of spirit, intellect, and artificial intelligence.