Calls to slow the development of artificial intelligence by three of the sector’s most prominent figures have exposed a growing conflict between safety concerns and the financial system supporting the industry.
Dario Amodei, chief executive of Anthropic, has warned that AI is beginning to build the next generation of models, raising the prospect of systems whose capabilities could soon be beyond human understanding or control. His proposal to “pace” development was joined by OpenAI chief Sam Altman and SpaceX founder Elon Musk.
The unusual alignment between the leaders of Anthropic, OpenAI and SpaceX has not been matched by the major hyperscalers, including Google, Meta, Microsoft and Amazon. Their agreement may reflect genuine concern about the direction of frontier AI, but it also comes as the industry faces mounting pressure from investors and rising infrastructure costs.
Recent incidents involving AI agents escaping supposedly secure testing environments, together with warnings from researchers, have fuelled fears that the technology is entering a more dangerous phase. Models are increasingly beginning to train themselves, adding to concerns about how their behaviour can be monitored and controlled.
Supporters of a slower approach argue that additional safeguards and peer review could reduce the risks. But a pause would not necessarily be observed by every developer, particularly in China or Russia, and there is no indication that the global industry would agree to a moratorium.
AI slowdown would put financial markets under pressure
The American AI sector is now closely tied to the continued expectation of innovation and expansion. A loss of momentum could threaten companies whose valuations depend on the belief that increasingly powerful models will continue to be developed and commercialised.
Anthropic, OpenAI and SpaceX do not have the large legacy cashflows enjoyed by companies such as Alphabet, Meta, Amazon and Microsoft. Even the hyperscalers have been turning to debt and equity markets to finance the growing amount of computing power required to compete in AI.
The cost of chips, data centres, energy and water infrastructure has risen sharply as companies compete to develop frontier models. At the same time, investors are becoming more cautious about the gap between the scale of AI investment and the revenue generated by the products.
Developers without substantial existing cashflows are particularly dependent on raising new equity at higher valuations. As the sector has become more capital-intensive, companies have also increased their use of debt, adding leverage to businesses whose financial foundations are already considered fragile.
The commercial challenge is becoming more pronounced as Chinese open-source models offer capabilities described in the source material as only slightly inferior, while being cheaper to access. Businesses are increasingly aware of the rising cost of deploying advanced AI and may be unwilling to pay enough to support products that become more expensive to develop.
Yet the valuations of the leading US developers depend on the continued expansion of their models. Anthropic is reportedly preparing for an initial public offering at a valuation of about 2 trillion US dollars, while SpaceX is valued at just over 2 trillion US dollars, with much of that valuation attributed to its AI development. OpenAI is expected to seek a valuation well above its last official figure of 852 billion US dollars if it floats next year.
AI-related companies, including chipmakers and data-centre developers, now account for about 45 per cent of the S&P 500’s capitalisation. That is more than twice their share when ChatGPT triggered the current AI boom in late 2022, and above the roughly 35 per cent recorded by technology companies at the height of the dotcom era.
A fall in the valuation of one or more major developers, or a loss of access to equity markets, could therefore have consequences across the sector. The hyperscalers would be better placed to withstand such a shock, but the pure AI developers rely on continual injections of capital to survive and to fund the next generation of models.
That has created a difficult choice for the companies now calling for a slower pace and stronger safeguards. Continuing at the current speed could deepen concerns about the safety and controllability of AI, while slowing down could undermine the valuations and financing on which the developers depend.
The US AI industry has become an interdependent ecosystem rather than a conventional collection of separate businesses. Nvidia, which has the sector’s largest profits and balance sheet, effectively acts as a financier to many of the companies buying its chips, while developers and infrastructure providers rely on one another to maintain the expansion of computing capacity.
Markets already appear close to saturation with AI exposure, while infrastructure companies are struggling to meet demand for the computing power and data centres needed to train frontier models. A slowdown could give suppliers time to catch up, but would also challenge the rising valuations that have helped sustain the industry.
The motivations behind the call from Amodei, Altman and Musk may therefore be mixed. Their warnings could reflect genuine fears about what future models may be capable of, concern over the financial resources required to keep developing them, or both. Either way, the debate points to a potential turning point for the AI companies and the markets that fund them.
