The Limits Are Not All in the Chip

“We cannot keep scaling forever” is true of any physical activity. It is not a useful timetable. To understand which limit matters next, separate the resources rather than imagine one universal wall.

A model can be limited by insufficient data while its datacentre has spare power. A new building can have accelerators on order but no grid connection. An experiment can be feasible and still fail an investment test.

Physics and movement

Transistor density is only one part of useful compute. Heat must be removed. Weights and activations must move through memory. Accelerators must exchange information quickly enough that expensive hardware does not spend much of its time waiting.

This is especially relevant to distributed expert models: reducing arithmetic does not eliminate communication. The Switch Transformer paper describes the difficulty of balancing memory, operations and communication when combining forms of parallelism. Fedus et al..

The practical implication is to measure the complete workload. Peak arithmetic throughput alone cannot tell a buyer how long an answer will take or how many requests a service can support.

Energy is a local constraint as well as a global total

The IEA’s April 2026 update estimates that datacentre electricity demand grew 17% in 2025, with demand from AI-focused facilities rising 50%. It also reports falling energy use per task alongside greater usage and more demanding applications. These are estimates for defined sectors, not measurements of every model query. IEA, Key Questions on Energy and AI.

The same report identifies bottlenecks in grid connections, energy equipment and advanced computing components. A global supply figure can hide a severe local problem: sufficient power must be available at the right site, with cooling and reliable connections. Efficiency improvements help, but increased demand can absorb the saving.

Data is not an infinite pile of clean examples

Human-generated information is finite, and the useful subset is smaller than the entire web. Quality, licensing, duplication and relevance complicate any estimate of remaining training material. Research on data limits projects constraints under stated growth and data assumptions; its dates should not be treated as an appointment with a hard ceiling. Villalobos et al., Will we run out of data?.

Synthetic examples, generated environments and self-play offer ways to create further learning signals. AlphaZero is a clear historical example of learning through self-play in games with known rules and evaluable outcomes. That success does not make an unconstrained generated explanation a trustworthy source of new facts. DeepMind, AlphaZero and MuZero.

The important distinction is between producing more text and producing useful evidence. A simulated environment can supply feedback. A second copy of an error supplies repetition.

Capital is not the cost of one training run

Alphabet’s second-quarter 2026 results report $44.924 billion in purchases of property and equipment for the quarter, and $80.598 billion for the first half. These are company-wide capital expenditures. They include infrastructure serving multiple activities and must not be relabelled as the cost of training a single frontier model. Alphabet’s filed earnings release.

Comparable estimates of frontier training costs remain difficult because hardware pricing, utilisation, experimental runs and staff costs differ. Stanford’s 2026 AI Index notes that important training details are no longer disclosed for several major systems. A precise-looking comparison can therefore rest on estimates with different boundaries. AI Index, Research and Development.

For a bounded historical estimate, Cottier and colleagues modelled amortised frontier training costs growing about 2.4 times per year since 2016 in their 2024 study, with a 90% confidence interval of 2.0–2.9. This is an estimated trend from that study, not a disclosed bill for a September 2026 model. Its projection beyond the sample is conditional. The rising costs of training frontier AI models.

The limit that the customer notices

The investment question is whether another tenfold expenditure produces enough additional useful capability. That depends on the task. A small improvement in a valuable, difficult workflow might justify a large bill; a prettier answer to an already solved problem might not.

None of these constraints establishes that scaling has ended. Together they explain why allocation, efficiency and evidence of value become more important as the machinery grows.