Concerns about artificial intelligence (AI) threatening human extinction have intensified recently, fueled by warnings from current and former employees of leading AI companies and discussions surrounding the sector’s rapid development. However, experts caution that the perceived risks are more reflective of human decisions and actions than autonomous AI behavior.
Jacob Coxon, a former researcher at Anthropic, a prominent AI firm, publicly expressed alarm about the technology’s potential, stating that those developing AI believe it could kill humanity by the end of the decade. His former colleague, Evan Hubinger, placed the probability of AI causing human extinction within ten years at over 10%. These warnings gained further attention after Reuters and the Financial Times reported that Anthropic’s confidential IPO prospectus acknowledged “existential risks to humanity” associated with its work.
While these statements have stoked public anxiety and prompted lawmakers to propose regulations to curb AI research, some analysts emphasize that the narrative of AI running amok and escaping human control is misleading. Michael J.D. Vermeer, a specialist in emerging technology risks at the Rand Corporation, noted that predictions of AI-triggered extinction often rely on untestable assumptions and function more as prophecies than grounded forecasts. In a recent study, Vermeer and his colleagues argued that extinction-level threats, whether from nuclear weapons, pathogens, or climate manipulation, fundamentally require direct human intervention—highlighting that AI actions cannot be fully divorced from human influence.
Several incidents cited as examples of AI systems “going rogue” involved significant human participation or oversight lapses. For example, a known hack of the AI platform Hugging Face was enabled after OpenAI deliberately deactivated certain security safeguards to test system boundaries. Other episodes of AI misbehavior reportedly resulted from human operators leaving systems unsupervised. Experts contend that with proper controls, AI can be managed effectively.
Claims touted by AI developers regarding breakthroughs, such as solving complex mathematical problems or discovering enzymes, have also drawn scrutiny. OpenAI’s announcement of progress on the Navier-Stokes equations was challenged by mathematicians who suggested the findings may have incorporated prior human work. Similarly, Anthropic’s claim of identifying a new enzyme was questioned by researchers who had contributed to that discovery process and used Anthropic’s AI as a tool, rather than the AI independently generating the result.
Critics argue that much of the anxiety surrounding AI stems from anthropomorphic metaphors that attribute human-like thinking, learning, and reasoning to computer processes that fundamentally differ from human cognition. Melanie Mitchell, a longtime AI researcher at the Santa Fe Institute, characterized media portrayals of AI systems as rogue agents escaping control as misleading, pointing instead to human error and engineering failures as the root cause.
The inclusion of dire risk warnings in Anthropic’s IPO documents has sparked speculation on the company’s motivations. Analysts suggest the move could be a precautionary acknowledgment of public concern, a marketing tactic to emphasize the power of its technology, or a strategy to divert attention from significant financial losses reported in recent fiscal periods.
Ultimately, the debate underscores that AI remains a human-designed technology whose impact depends heavily on human choices and governance. Recent events, including a U.S. military strike on a school in the Iran conflict where AI-assisted targeting played a role, illustrate the serious consequences of decisions made by human operators using AI tools.
As the conversation around AI continues, experts urge a focus not only on the capabilities of the technology itself but also on the human actors responsible for its deployment and oversight.