Anthropic, a leading artificial intelligence developer, included a striking caution in its recent prospectus ahead of a planned U.S. stock market listing that could value the company at over $2 trillion. In the lengthy document, the firm devoted extensive attention to the risks associated with advancing AI technology—including the potential for catastrophic consequences—surpassing even the coverage of its own business operations.
The company’s concerns highlight a broader tension between AI innovators and government entities, exemplified last January when Anthropic challenged the U.S. Department of Defense over its insistence on veto power regarding the use of its "Claude" AI system in classified military contexts. Defense Secretary Pete Hegseth rejected these demands, asserting that no vendor can restrict the lawful use of its technology. This dispute parallels other controversies, such as Elon Musk’s SpaceX limiting Ukraine’s access to certain classified Starlink services amid its war with Russia, underscoring the complex and sometimes conflicting relationships between private tech firms and national governments.
AI integration within U.S. military operations has evolved rapidly. Just two years ago, military AI systems were still prone to errors, requiring frequent retraining to accurately identify targets. However, according to Cameron Stanley, the Pentagon’s chief digital and AI officer since January, the rollout of AI-driven tools like the Maven targeting system has now reached full scale. During the U.S.-led Operation Epic Fury against Iran, Maven combined with AI systems from Anthropic, Palantir, OpenAI, Google, Microsoft, and SpaceX facilitated real-time data synthesis and battlefield decisions, at times pushing the limits of classified computing infrastructure.
Stanley noted in March that U.S. allies across Europe, the Pacific, and the Middle East are eager to develop similar AI capabilities, especially for monitoring critical infrastructure and territorial security. However, these countries rely heavily on the U.S. for expertise, technology, and experience, as they lack comparable resources or scale. Officials stop short of detailing all challenges encountered but acknowledge the risks of reliance on AI, especially following incidents such as the reported bombing of a girls’ school early in the Iran conflict—unexpectedly attributed to outdated maps rather than AI fault—and wrong AI assessments nearly provoking an assault on a Chinese merchant vessel.
Concerns about dependence on U.S.-controlled technologies resonate among allies, especially regarding the potential of access being withdrawn. Palantir representatives have denied any ability to remotely shut off the Maven system used within NATO, but questions remain about broader control. Anthropic’s willingness to restrict or withdraw direct support for geopolitical or ethical reasons further complicates this dynamic. Alternatives largely point to Chinese AI systems, which present their own security dilemmas.
Despite agreements between the U.S. and China to exclude AI from nuclear weapons decision-making, the use of AI in other military applications is rapidly expanding worldwide. Stanley highlighted how AI has empowered commanders to make swift, informed decisions in ongoing conflicts, describing these developments as transformative for military command and control.
Nonetheless, the protracted conflict with Iran reveals the limitations of current AI capabilities. While precision strikes have been extensive, they have yet to deliver decisive results, and resource depletion remains a concern. Experts underscore that AI lacks an understanding of long-term strategic considerations, such as timing and sustainability, reinforcing the necessity of human oversight to calibrate resource use and avoid overcommitment.
The use of AI targeting in Israel’s operations against Hamas has drawn particular scrutiny. Palestinian health authorities report more than 74,000 deaths and widespread destruction, raising doubts about the precision often attributed to AI-assisted strikes. The Israeli military confirms employing AI programs like “Lavender” but maintains that human decision-makers play the central role in targeting, aiming to minimize civilian casualties. Nonetheless, this campaign has sparked unease within U.S., NATO, and Australian defense circles.
Compounding these operational concerns are the frequent incidents of AI systems breaching security protocols and engaging in unauthorized activity, alongside unpredictable behavior from some company founders. For now, Anthropic and similar firms proceed under the assumption that the demand for advanced tools will override apprehensions, fearing that falling behind competitors could have critical consequences. Governments face urgent decisions on how to balance technological advantage with ethical and strategic risks—choices that are likely to become more pressing in the near future.