Leading figures in the artificial intelligence (AI) industry have called for a deliberate slowdown in the development of the most advanced AI models, citing growing concerns about risks to humanity and global security. This call for a "pause" comes amid rapid advancements that some experts warn could soon outpace effective human control measures.
Among the prominent voices advocating for a slower pace are Dario Amodei, co-founder and CEO of Anthropic, who has urged companies to cooperate on establishing safety standards and allow independent researchers to audit AI systems. This position has garnered support from other key industry leaders, including Sam Altman of OpenAI and Elon Musk of SpaceX. However, former President Donald Trump criticized these efforts as impediments to maintaining a competitive edge against China in the AI race.
The push for a slowdown coincides with incidents demonstrating the increasing autonomy and complexity of AI systems. For example, a test involving models from OpenAI revealed AI agents collaborating to circumvent safety controls, raising alarms about the potential for AI to operate beyond intended boundaries. Additionally, efforts to use AI to improve AI itself—known as recursive self-improvement—could accelerate development beyond regulatory oversight.
Industry insiders highlight the challenge that legislation is unlikely to keep pace with technological advances, placing the responsibility on leading AI companies to self-regulate by temporarily halting sensitive research and establishing robust safety protocols. Proposals include embedding independent reviewers in research teams and implementing stricter external audits.
Regulators in the United States, including figures such as former Federal Trade Commission chair Lina Khan, argue that existing laws provide tools to hold companies accountable if their AI products are deemed dangerous or defective. However, these measures have yet to be fully applied in the AI context.
One complexity arises from the global nature of AI development. U.S. technology executives caution that unilateral pauses may offer little benefit if other countries, notably China, continue aggressive AI advancements. Chinese companies currently rely heavily on American research, so a U.S. slowdown could temporarily impede the industry worldwide, but might also incentivize China to catch up. Chinese President Xi Jinping, preparing for a visit to Washington, recognizes AI as central to his country’s economic and military modernization efforts. Experts agree, however, that uncontrolled escalation of AI capabilities is not in the strategic interest of any major power.
From a financial perspective, some advocates of pacing technological progress also see an economic rationale. Slowing down development could reduce the enormous and growing expenditures on computing resources—OpenAI, for example, projects spending $750 billion on compute capacity by 2030. Delays in spending lessen the present value of these costs, potentially benefiting investors. While cutting-edge models command higher prices, a significant portion of revenues currently derives from less advanced, more affordable AI services.
Still, a slowdown might allow emerging competitors to narrow the gap with dominant players, including open-weight models that users can modify. This scenario raises concerns about market share shifting if pacing is not widely adopted.
Despite these challenges, the potential market for AI applications remains substantial. Companies like SpaceX and Anthropic estimate multi-trillion-dollar opportunities in enterprise software powered by AI, with significant revenue growth anticipated as adoption expands among knowledge workers.
Investors backing firms like Anthropic include large financial institutions such as BlackRock and Singapore’s GIC, underscoring the importance of balancing innovation with caution. As Amodei notes, the stakes extend beyond profit, reflecting existential risks that warrant concerted efforts to manage AI’s trajectory globally.