For the first time, leaders from major artificial intelligence companies addressed the United Nations Security Council to discuss the promises and perils of AI technology, highlighting growing concerns around control, governance, and the potential societal impact of these rapidly advancing systems.
On September 23, Sam Altman, CEO of OpenAI, and Dario Amodei, CEO of Anthropic, used the high-profile platform to advocate for international cooperation and regulatory frameworks aimed at managing the development and deployment of AI. Both emphasized the dual-use nature of AI, framing it as a force capable of driving innovation or causing significant disruption, depending on how it is governed.
Altman described AI as offering humanity a “choice” between a “new Renaissance of creativity and discovery” or a “new Industrial Revolution of upheaval and disarray.” He stressed that AI systems should be designed for people’s benefit, maintaining meaningful human oversight to prevent the technology from becoming concentrated in the hands of a powerful few. Altman cautioned against advancing AI models that cannot be reliably controlled, highlighting the risks posed by potential misuse or loss of human agency.
From OpenAI’s perspective, he proposed three key principles: keeping humans central in AI development, ensuring scientific and economic progress benefits society broadly, and empowering individuals through technology. Altman called for democratic involvement in AI governance, advocating for collaboration among governments to establish “frontier AI standards” to measure capabilities, assess risks, ensure safeguards, and facilitate incident reporting. He suggested secure communication channels between governments, infrastructure operators, and technical experts to share emerging threats.
Similarly, Anthropic’s Amodei acknowledged the vast potential of AI, citing his company’s announcement of a Claude AI-led discovery of a new molecular machine with possible applications in gene therapy. At the same time, he highlighted dangers such as bioterrorism risks and “loss of control” as AI systems advance beyond their creators’ ability to manage them. Amodei argued for external evaluation mechanisms across AI firms, establishment of industrial standards to regulate the pace of development, and global cooperation on international standards.
Amodei specifically urged the Security Council to consider initial agreements banning the use of AI in biological weapons development, advocating for verification systems to monitor frontier model capabilities and establish global notification protocols for AI incidents affecting security.
Despite these calls for oversight, critics question whether these industry leaders can adequately self-regulate given their stakes in deploying AI technologies. Former technology researchers like Jacob Coxon, who previously worked with both OpenAI and Anthropic, have expressed fears that the pursuit of self-improving superintelligence risks human survival. Coxon warned that developers are “racing straight toward” uncontrollable AI and gambling with humanity’s future.
UN human rights officials expressed ambivalence over the engagement with AI firms. Volker Türk, the UN High Commissioner for Human Rights, acknowledged the importance of safeguards and due diligence to prevent a dystopian future shaped by AI but noted the challenges in establishing binding international rules. Türk referenced a recent voluntary accord signed by the U.S. government and leading AI companies on safety and risk assessment as a symbolic but insufficient step.
As AI technology continues to evolve, the intersection of innovation, governance, and ethical responsibility remains complex. The United Nations deliberations reflect an ongoing struggle to balance the transformative opportunities AI offers against the risks it poses to security, human rights, and social stability.