In recent months, Iran, China, and Israeli-linked groups have employed artificial intelligence (AI) in unprecedented ways to conduct influence campaigns on social media, raising concerns among U.S. officials and security researchers about the evolving capabilities of the technology. These campaigns utilized AI-driven "agents"—autonomous bots capable of creating and managing fake accounts and coordinating messaging with minimal human intervention.
The operation began with the use of open-source AI models developed in China, which allowed the creation of self-operating agents. These agents autonomously established networks of fake profiles across platforms such as Instagram, Facebook, X (formerly Twitter), and TikTok, rapidly disseminating false or politically charged content designed to influence public opinion and exacerbate social divisions.
U.S. officials identified the Iranian and Chinese campaigns as state-sponsored efforts, while the Israeli campaigns were linked to private companies. The Iranian operation was particularly notable for targeting U.S. audiences in the first half of 2024. The AI-generated accounts imitated everyday Americans from major cities and produced content containing anti-Republican Party messages. Hundreds of thousands of fake accounts followed these profiles, which engaged with journalists and politicians in attempts to amplify their influence.
The Chinese campaign reportedly used similar AI tactics, though fewer details have been disclosed by intelligence sources. Meanwhile, two separate Israeli campaigns were traced back to commercial entities. One of these was conducted by IntelEye, a Tel Aviv-based firm founded in 2023 by former NSO Group employees. IntelEye employed the DeepSeek AI model, based on Chinese open-source technology, to manage thousands of social media accounts throughout the Israeli summer. The accounts engaged with content related to the country's upcoming elections, posting both supportive and critical commentary concerning Prime Minister Benjamin Netanyahu.
IntelEye’s co-founder, Maor Sellek, described the operation as an experiment aimed at defensive research and detection improvement rather than an active influence campaign. He acknowledged that the AI agents autonomously selected topics and crafted responses, and roughly a thousand accounts were active. However, he conceded that the firm should have informed social media platforms of the test to avoid confusion. A Meta spokesman characterized the activity as a coordinated effort to create inauthentic accounts, though the bulk of these were removed by automated systems.
These AI-driven campaigns are significant due to their scale, autonomy, and sophistication. Experts warn that as AI technology advances, such agent-led influence efforts could become more common and harder to detect, posing risks to democratic processes globally. Concerns have heightened ahead of the U.S. midterm elections, with some officials fearing that similar tactics could be used to target American voters.
The debate around AI safety has intensified following reports of AI systems from major developers behaving unpredictably, including incidents of hacking between AI platforms. While some industry leaders have called for a slowdown in AI development to establish proper safeguards, others argue that such measures are unnecessary and could hinder innovation.
Social media companies have acknowledged the growing use of AI in coordinated inauthentic behavior. Meta has published reports documenting the integration of AI in account creation and content generation by bad actors, including Iranian and Israeli operations, describing AI as an embedded element in automated influence campaigns. X has not publicly commented on these developments.
Overall, the emerging use of autonomous AI agents for influence operations highlights the challenges faced by governments, technology firms, and security experts in addressing the rapid evolution of digital information warfare.