Gorio Tech Blog search

From AGI to ASI | Summary

|

Contents

This article explains the key points of From AGI to ASI.

  • 2026-06-10 (arXiv)
  • Genewein, Tim, Franklin, Matija, Lerchner, Alexander, Orseau, Laurent, Albanie, Samuel, Bales, Adam, Wyeth, Cole, Chan, Stephanie, Gabriel, Iason, Leibo, Joel Z., et al.
  • Google DeepMind, University of Waterloo (work conducted while at Google DeepMind), Australian National University, University College London
  • Paper

Read this article in Korean


Summary

  • From AGI to ASI (arXiv:2606.12683) examines how machine intelligence might develop beyond human-level AGI. It characterizes AGI as roughly median human performance on most cognitive tasks and sets a higher threshold for ASI: outperforming large, well-coordinated human-expert collectives across virtually all domains.
  • The report identifies four potentially concurrent pathways: scaling compute, models, and data; algorithmic paradigm shifts; recursive improvement; and multi-agent group formation. It synthesizes empirical trends, theoretical frameworks, and cited research rather than presenting a new experimental demonstration of ASI.
  • The central uncertainty is whether effective-compute growth, research automation, and collective organization can outpace resource constraints, declining research productivity, and architectural limitations. The report proposes continuously updated forecasts, benchmarks beyond human performance, and empirical studies of recursive and multi-agent scaling; it does not establish an ASI timeline or demonstrate that any pathway will succeed.

2. Introduction: Life as we don’t know it?

Section 1 describes options for tailored AI-generated summaries and directs readers to the static summary in Appendix A, Tables 1, 3, and 4, the glossary, and the research agenda. The introduction then separates growth in effective compute from growth in capabilities, combining historical estimates of hardware improvement at about 1.5× per year, compute investment growth at roughly 2.5× per year, and algorithmic efficiency improvement at 3× per year: 1.5 ∗ 2.5 ∗ 3 = 11.25, rounded down to about 10× per year.

  • These are uncertain historical estimates, not guaranteed future rates. The report also cites higher algorithmic-efficiency estimates and recommends consulting updated measurements.
  • Under a hypothetical 10× annual growth scenario, 1000 initial AGI instances become 10,000 after one year and 100 million after five years, or 1 million instances running a hundred times faster. This illustrates population and speed scaling rather than predicting deployment.
  • AI-assisted research could increase the growth rates themselves, but resource constraints and diminishing returns may produce an S-shaped trajectory rather than sustained hyperbolic growth. The report recommends multiple quantitative scenarios, uncertainty estimates, model ensembles, and frequent updates.

3. Characterizing Artificial Superintelligence

AGI and ASI are informal positions on an intelligence continuum rather than precisely specified benchmark thresholds. To stabilize the human reference point, the report describes ASI as reliably outperforming hypothetical groups of tens of thousands of coordinated experts working for 10 years with technology and cultural artifacts available in 2010.

  • Narrow superhuman systems such as AlphaGo and AlphaFold do not meet the generality requirement.
  • ASI may consist of many interacting instances; the characterization does not require one individually superhuman model.
  • The Legg-Hutter score supplies formal grounding through complexity-weighted performance across computable environments and tasks. The report does not literally measure its AGI or ASI thresholds with this incomputable score, and concrete capability profiles may remain jagged.

Table 1 identifies six properties through which digital intelligence can benefit from increased effective compute in ways biological intelligence cannot. Their practical benefits depend on hardware, interfaces, and learning algorithms; the surrounding discussion also notes possible disadvantages, including conversion costs at physical interfaces and the potential energy efficiency of analog computation.

  • Input / output speed: increasingly high-bandwidth information intake and output, with suitable sensors and actuators required for physical interaction.
  • Internal processing speed: faster sequential computation or greater parallel computation can expand reasoning and search budgets.
  • Working memory capacity and memorization: larger memory and greater read/write bandwidth can exceed human working-memory limits.
  • Substrate independence: systems or components can migrate to more powerful, efficient, or distributed hardware.
  • Lossless replication: source code and memory states can be copied, backed up, restored, paused, and resumed.
  • High-bandwidth sharing of (learning) experiences: digital interaction streams can be replayed, and homogeneous instances can share learning signals such as gradient updates. Third-person observations may nevertheless be causally insufficient for learning decisions.

The six rows identify distinct compute-amplified properties: faster interfaces and processing, larger memory, hardware migration, exact copying, and experience sharing. They explain why scaling digital intelligence differs from adding human workers, but do not quantify the resulting gains in intelligence.

Advantages of digital over biological intelligence that grow with faster or more compute.
Advantages of digital over biological intelligence that grow with faster or more compute.

ASI is not equated with omniscience or omnipotence. Table 2 lists fundamental limits but emphasizes that they do not readily determine whether particular achievements, such as curing ageing or restoring pre-industrial climate and biodiversity, are feasible.

  • Fundamental physics constrains information propagation, computation speed, energy costs, and information storage.
  • Real time and physical manipulation constrain experiments, construction, and processes that cannot be simulated with sufficient precision.
  • Ignorance, limited observability and controllability, computational complexity, Gödel’s Incompleteness, and the Halting Problem remain relevant regardless of intelligence.

The table separates physical, temporal, epistemic, computational, and logical limits. It rules out unlimited power or knowledge while emphasizing that these general constraints do not readily determine the feasibility of particular scientific breakthroughs.

Fundamental limitations that remain applicable to artificial superintelligence.
Fundamental limitations that remain applicable to artificial superintelligence.

4. Universal AI — An Informal Overview

Universal AI supplies an idealized endpoint through AIXI, an agent that plans using a Bayesian mixture over computable environments and reward functions. Solomonoff’s Universal Prior favors lower-complexity hypotheses, while a specified horizon or discounting scheme determines the weight of future rewards.

  • Expected cumulative reward maximization jointly addresses environmental uncertainty, long-term credit assignment, and exploration versus exploitation.
  • AIXI is optimal in expected reward under the specified universal prior, not in every individual environment. This does not establish a prior-independent ranking of useful intelligence.
  • AIXI is a learning algorithm rather than a trained model; the appropriate comparison concerns architectures and training procedures evaluated over cumulative learning experience.

Neither AIXI nor its associated universal intelligence measure is computable, and existing approximations do not provide a practical recipe for frontier-scale ASI. The report connects log-loss pretraining to resource-bounded universal compression and suggests adding planning or decision-making scaffolding to a predictor, while explicitly treating the argument for the current paradigm as incomplete and inconclusive.

  • Continual learning, very long-context processing, and robust planning remain practical limitations; theoretical capacity does not establish usable computational efficiency.
  • The original framework excludes incomputable AIXI agents from its computable environment class. The report cites an embedded multi-agent extension as progress on this representational limitation.
  • Prior choice, universal Turing machine choice, and averaging over all computable worlds complicate translation to useful systems in the actual world. Reflective oracles, logical induction, Gödel machines, learning theory, game theory, and thermodynamic approaches provide complementary perspectives.

5. Technological Pathways and Potential Bottlenecks to ASI

Table 3 organizes four pathways by their principal uncertainties rather than ranking their demonstrated feasibility. Scaling provides historical observations for fitting forecasts; paradigm shifts, recursive improvement, and emergent group agency have weaker empirical bases for forecasting, and the pathways may interact or proceed in parallel.

Each pathway presents a different forecasting problem: translating scale into capability, anticipating new paradigms, modeling recursive feedback, or understanding emergent group organization. This is a conceptual taxonomy rather than an empirical comparison or probability ranking.

Four technological pathways from AGI to ASI and their principal uncertainties.
Four technological pathways from AGI to ASI and their principal uncertainties.

5.1. Scaling compute, models, and data

The scaling pathway extends model, data, training-compute, and test-time-compute growth. Chinchilla illustrates the importance of compute-optimal co-scaling rather than parameter count alone, while the discussion of search emphasizes that practical gains require efficient priors, heuristics, or approximations rather than brute force.

  • The report cites possible exhaustion of suitable human-generated text later this decade. Synthetic data, simulations, and interactive learning are potential alternatives, not established solutions at ASI scale.
  • Mixture-of-Experts illustrates an efficiency improvement that can extend scaling, but does not demonstrate that scaling alone produces ASI.
  • Some apparent emergent capabilities may be evaluation-metric artifacts. Separately, population scaling could improve collective capabilities even if individual models plateau; forecasting requires tracking resource growth, efficiency, and uncertainty.

5.2. Algorithmic paradigm shifts and evolutions

The report distinguishes evolutions of the current paradigm from more radical paradigm shifts. Evolutions include continual learning, extended memory, adaptive test-time computation, tool-augmented planning, and robust world-model-based interaction; shifts could replace architectures, optimization procedures, learning paradigms, or hardware substrates.

  • Retrieval, recurrent methods, and state-space architectures address fixed-context or attention-cost limitations. Their discussion does not establish unlimited reliable memory or reasoning.
  • Learned world models offer compressed representations for simulation, planning, and counterfactual reasoning, with cited examples from latent imagination and model-based control.
  • Genuinely new paradigms are difficult to anticipate. The report therefore recommends paradigm-independent theory to clarify possible limits rather than precise forecasts of future resource demands or capabilities.

5.3. Recursive self-improvement

Recursive improvement occurs when AI helps produce better AI, which can then contribute more effectively to subsequent development. The report distinguishes improvements through software, hardware, training data, and division of labor, as well as weak AI-assisted feedback loops from sustained autonomous self-improvement.

  • AlphaZero-style search and distillation illustrate data-mediated recursion: stronger search outputs improve the policy and value priors guiding subsequent search. Self-play supplies an adaptive source of experience.
  • FunSearch and AlphaEvolve provide cited examples of AI-guided program discovery; architecture search, hyperparameter optimization, and assisted chip design illustrate narrower improvement mechanisms. The report also discusses verification and theoretical approaches to self-modification.
  • These examples do not establish an intelligence explosion. Larger experiments, fabrication delays, resource consumption, diminishing returns, and verification requirements may dampen recursion; measuring these mechanisms and developing recursive-improvement scaling laws remain research goals.

5.4. Multi-agent coordination & group agency

The group-agency pathway proposes that coordinated AGI systems could solve problems beyond any constituent agent through parallelization, specialization, and task decomposition. Organizational possibilities include centrally orchestrated collectives, automated corporations, and decentralized markets using prices, auctions, and local incentives.

  • High communication bandwidth and coordinated objectives could reduce some human organizational frictions, but the report provides no measured multi-agent scaling law.
  • It remains unclear when homogeneous copies generate synergy, when diverse specialists are needed, and whether adding agents uses compute more efficiently than enlarging individual models.
  • Steering groups, resisting persuasion cascades and Byzantine influence, and enabling cooperation in mixed human-AI collectives are open capability and governance problems.

5.5. Potential Bottlenecks to ASI

The data wall arises when demand for useful training data grows faster than its production. Naive recursive training on generated data can plateau or degenerate, while search-improved outputs, curated synthetic data, simulations, self-play, and environment interaction may provide useful experience beyond a base model’s current competence.

  • This bottleneck threatens data-intensive scaling and data-mediated recursive improvement.
  • Paradigm shifts that improve data efficiency could reduce exposure; multi-agent interaction and simulation could expand data production if the resulting experience is informative.
  • The unresolved question is whether useful data generation can keep pace with demand, not merely whether large volumes of synthetic content can be produced.

Economic and natural-resource demands can make continued scaling unsustainable even when larger systems remain technically possible. The report discusses investment, chips, supply chains, energy, land, water, materials, memory bandwidth, and interconnect constraints, with economic viability depending partly on returns from AI deployment.

  • Resource-intensive model scaling and population scaling are especially exposed because both require substantial infrastructure.
  • Algorithmic innovation and recursive efficiency improvements could let capabilities grow faster than physical inputs, although hardware-oriented recursion remains tied to manufacturing.
  • Infrastructure expansion and AI-generated revenue are possible counterforces, not evidence that exponential resource growth can continue indefinitely. Proposed orbital datacenters introduce additional environmental and collision risks.

The current neural paradigm may encounter limitations that extensions to pretraining cannot resolve. Candidate issues include hallucinations, prompt-injection vulnerability, uncertainty-sensitive decision-making, causally insufficient observational data, continual learning, and the abstraction barrier.

  • A fundamental paradigm ceiling would constrain current-model scaling and could prevent systems from becoming sufficiently capable for autonomous research.
  • Paradigm evolution or replacement addresses this possibility directly, including research accelerated by systems below AGI level.
  • Collective organization may bypass some individual limitations, but the report does not establish that it can compensate for every missing capability.

Research gets harder when maintaining progress requires increasing researcher effort, larger experiments, or more costly hypothesis search. The report cites Bloom et al.’s estimate that maintaining Moore’s law required about 18 times more researchers than in the 1970’s, then contrasts slow human researcher expansion with potentially rapid replication of digital researchers.

  • This friction affects algorithmic innovation, efficiency gains supporting scaling, and recursive research loops.
  • More capable or numerous AI researchers could offset declining productivity, but cognitive acceleration does not remove experimental time and resource costs.
  • The authors’ suggestion that this may be a minor friction depends on useful, affordable AI researchers. The report supplies no new productivity experiment establishing that judgment.

The Abstraction Barrier is the hypothesis that training on human conceptual products may not enable discovery of new conceptual primitives from raw data. The related Embodied Bottleneck concerns validating new abstractions against physical reality, where reaction times, manipulation speeds, and imperfect simulation impose delays.

  • If the hypothesis holds, it could constrain individual-model scaling and recursive scientific improvement despite faster computation.
  • Large collectives might still exceed human organizational capabilities without overcoming the barrier in every individual system.
  • Grounded interaction and changes to learning paradigms are proposed responses. The report does not experimentally establish the barrier or show that a particular embodied architecture is necessary.

Deliberate slowdown, regulation, and societal backlash could restrict development or deployment after accidents, misuse, loss of control, or broader social harm. Table 4 includes this sociopolitical friction alongside technical bottlenecks, while the text recognizes governance as a means of steering development as well as slowing it.

  • Governance can affect all four pathways through deployment rules, evaluations, licensing, liability, or capability restrictions.
  • Economic and military competition, regulatory arbitrage, and weak international coordination could counteract slowdown pressures.
  • The significance of all six bottlenecks remains open, and the inventory is non-exhaustive. The report provides no quantitative pathway-specific impact estimates.

Six bottlenecks are paired with possible counterforces, so their significance depends on the balance between constraints and responses as scale and capabilities change. The table identifies unresolved research questions; it supplies neither a pathway-by-bottleneck impact matrix nor quantitative severity estimates.

Potential transition bottlenecks and factors that could counteract them.
Potential transition bottlenecks and factors that could counteract them.

6. Remarks

The remarks distinguish theoretical sufficiency of compute scaling from practical efficiency. Suitable open-ended search can improve with additional compute in principle, but practical systems rely on inductive biases that reduce search costs and may impose capability ceilings; qualitative innovation may therefore remain necessary.

  • Current test-time scaling has limited headroom. The report does not treat arbitrary additional inference compute for today’s systems as sufficient for ASI.
  • Population scaling could instead produce superhuman collectives even if individual models plateau.
  • Which tasks benefit from group organization, how groups should be coordinated, and how their gains scale with resources remain empirical questions.

Predicting particular ASI capabilities requires more than worst-case complexity limits, because heuristics and approximations can achieve strong performance at much lower cost. The report favors empirical extrapolation complemented by theory, discussing benchmark stitching, saturation-resistant evaluation, and the difficulty of predicting approximation quality in advance.

  • GPQA, SWE-bench, and FrontierMath motivate evaluation beyond human-expert ceilings; the report supplies no new benchmark scores. It also discusses ARC-AGI and SuperARC as approaches to generalization and increasingly complex tasks.
  • Boden’s distinction between combinational, exploratory, and transformative creativity separates new combinations and solutions within existing spaces from the creation of new conceptual spaces.
  • AlphaGo’s Move 37 illustrates exploratory creativity. Independent discovery of transformative scientific concepts remains unresolved, while transformative artistic value also depends on human cultural context.

The report examines instrumental convergence, autonomy, reward objectives, and less-agentic alternatives. Resource acquisition and self-preservation may become useful subgoals, while theoretical corrigibility and interruptibility results do not yet provide practical guarantees for frontier systems.

  • Knowledge Seeking maximizes expected information gain. The report discusses possible advantages concerning delusion, stagnation, and cooperation, without establishing it as a complete alignment solution.
  • Oracles, Scientist AI, and myopic systems may reduce some risks of long-horizon agency, but interaction with a persistent world can still create incentives to influence users or outcomes.
  • Economic pressure to reduce human oversight may favor autonomy even though superhuman cognitive capability does not logically require autonomous goal pursuit.

The report primarily uses progress to mean increased capability or efficiency, neither of which is equivalent to societal improvement. Its broader normative discussion concerns autonomy, dignity, flourishing, and benefits recognized within particular sociocultural contexts rather than assuming faster technological change is inherently desirable.

7. Outlook: Plenty That Needs To Be Done

The outlook treats the pathway map as an incomplete starting point for research on post-AGI development. It calls for interdisciplinary study of technological trajectories and societal consequences, acknowledging that impacts on economics, politics, education, psychology, and other areas remain largely outside the report’s analysis.

7.1. From AGI To ASI: A Research Agenda

The first three research themes concern scaling bottlenecks, quantitative forecasting, and benchmarking ASI. They aim to connect measured resource and capability trends to forecasts that remain informative after human-level benchmarks saturate.

  • Bottlenecks and Frictions for Scaling: investigate useful data generation, causal sufficiency of third-party experience, compute-to-intelligence relationships, paradigm changes, economic sustainability, research difficulty, physical experimentation delays, and limits imposed by human abstractions.
  • Quantitative Forecasting: identify measurable macro-quantities, model their coupling, ensemble plausible models, locate scenario-discriminating thresholds, and continuously update estimates and uncertainty bands.
  • Benchmarking ASI: develop low-human-input evaluations through competition and cooperation, setter-solver methods, compression tasks, or indirect productivity measures. Distinguish capability changes from metric artifacts and examine how evaluation can promote human compatibility and flourishing.

The fourth and fifth themes propose direct measurement of recursive improvement and collective intelligence. They focus on rates, resource efficiency, verification, organization, and failure modes rather than assuming recursion or larger populations necessarily improve capability.

  • Recursive Improvement Dynamics: measure distinct mechanisms, study test-time search and data curation, develop a theory of recursive distillation, track AI research and hardware-design contributions, and identify constraints remaining after intellectual labor is automated.
  • Multi-Agent Scaling: compare organizational forms and task classes, measure gains with population size, and determine when increasing agent count is more compute-efficient than enlarging individual models.
  • Group steering and epistemic resilience: study resistance to falsehoods, hallucinations, self-delusion, and epistemic hijacking, including recoverability in mixed human-ASI collectives.

The sixth and seventh themes concern theoretical foundations and AI Safety, Alignment, Sociocultural questions. To isolate technological trajectories, the report assumes safety and alignment can be solved sufficiently, while explicitly acknowledging that unsafe or uncontrollable systems could obstruct automated research and deployment.

  • Theoretical foundations: connect AIXI to practical algorithms, characterize useful approximation and lossy compression, study bounded decision-making and capability jaggedness, and develop frameworks for myopic or non-agentic systems.
  • Safety and alignment: investigate implementable slowdowns, individual and group alignment, and risks from convergent instrumental subgoals.
  • Scientific and socioeconomic institutions: examine epistemic norms under overwhelming automated research output and the consequences of a shift from labor toward capital.

7.2. Conclusions

With explicitly low confidence, the authors judge it more likely that progress either plateaus before AGI or continues relatively smoothly from AGI toward weak ASI than that it stops exactly at human level. This judgment relies substantially on possible collective amplification despite individual-model plateaus; recursive acceleration could shorten the transition but is not established.

  • The authors state that reaching ASI territory within the next decade or two cannot easily be dismissed. They do not attach a calibrated probability to that possibility.
  • They recommend diverse forecasts and scenarios supported by continual benchmarking, monitoring, and timely policy responses.

AI Use

The AI Use disclosure states that upward of 90% of the document was human authored without direct language-model involvement. For parts of the manuscript (< 10%), a language model helped polish wording and draft sentences or paragraph fragments; models also assisted with structure discussions, completeness checks, simulated reviews, literature reviews, and bibliography cleanup.

Appendix

  • A. Summary restates the four pathways, digital advantages, fundamental limits, and the uncertainties motivating further research. Its pathway-specific discussion adds that paradigm shifts may be difficult to recognize before substantial scaling and integration work, recursive improvement may plateau or degenerate, and larger groups may incur orchestration and bureaucratic costs that erode capability gains.
  • B. Glossary defines the report’s terminology and gives source-page references, including AGI, ASI, AIXI, Effective Compute, the Abstraction Barrier, Group Agency, Recursive Improvement, and the Universal Prior. It is a reference aid rather than additional experimental or theoretical evidence.

Brief Thoughts

The report separates individual-model improvement, recursive research acceleration, and collective amplification, which require different evidence and forecasting methods. Tables 3 and 4 identify the associated uncertainties, while the research agenda specifies measurements of scaling, verification, resource use, and organizational efficiency.

The conclusion about progress beyond AGI depends on unmeasured collective gains and uncertain feedback dynamics. AIXI supplies a prior-dependent theoretical bound rather than a practical transition rate, the Abstraction Barrier remains a hypothesis, and the safety-and-alignment working assumption sets aside a potentially decisive constraint; the report is a research map, not a validated forecast.