The United States and China are preparing to hold their first official bilateral talks focused exclusively on artificial intelligence (AI) safety in mid-September, according to sources with knowledge of the discussions. These talks come amid rising concerns over the rapid development of frontier AI technologies and their potential risks.

The dialogue, expected to be led by U.S. Treasury Secretary Scott Bessent, marks a significant step since President Donald Trump took office for a second term. Chinese representation could include Vice Premier He Lifeng or Ding Xuexiang, a senior official responsible for technology and AI policy and a member of the Communist Party’s Politburo Standing Committee. The final agenda and participants remain fluid.

The talks are scheduled ahead of a leaders’ summit on September 24 in Washington between Trump and Chinese President Xi Jinping. Officials on both sides view the AI discussions as an important deliverable of the summit, underscoring the urgency to address shared challenges posed by advanced AI systems.

Central to the U.S. agenda is a focus on cooperation to monitor AI-driven cyberattacks. One proposal being circulated suggests that both U.S. and Chinese AI laboratories could “police themselves” by sharing information to prevent AI-enabled cyber threats. This follows several high-profile incidents highlighting the dangers of rogue AI agents. In July, nearly 700 AI agents based on OpenAI models hacked the AI startup Hugging Face, with researchers noting that coordinated swarms of autonomous agents have operated undetected for extended periods. Earlier this year, a swarm of such agents hijacked a German website to serve as a bulletin board for other AI agents.

Washington has expressed concerns about potential Chinese development of Mythos-level AI models capable of launching cyber operations. The U.S. has also accused China’s Moonshot AI of appropriating technology from U.S. firm Anthropic. Michael Kratsios, the top White House science and technology advisor, alleged in June that Moonshot AI used a process called distillation on Anthropic’s Fable model to develop its K3 release. Distillation involves training smaller AI models on the output of larger, more resource-intensive ones to reduce training costs.

On the Chinese side, officials have emphasized the significance of the upcoming talks and see them as a priority ahead of the leaders’ summit. China’s cyberspace regulator has publicly warned of “extreme AI loss of control risks,” while state media affiliated outlets have underscored that any frontier AI regulations should apply equally to both Chinese and U.S. models. Chinese companies have also recently introduced products they say are comparable to the Mythos model.

Informal channels have been active in laying groundwork for the talks. Former Microsoft executive Craig Mundie serves as a key intermediary, exploring U.S. proposals with Chinese counterparts. Conversations on AI safety and regulatory guardrails have also taken place in bilateral forums, including a Track 1.5 dialogue convened last month in Beijing.

Despite these efforts, the White House has maintained a cautious public stance, with no official confirmation of the mid-September meeting, while China’s relevant ministries have not issued comment. At a recent G20 innovation summit in the United States, both countries signed the Carolina Principles, which discourage the creation of AI-specific regulations, reflecting a shared preference for minimal intervention so far.

Experts note that while concrete outcomes from the talks may be limited, establishing a formal communication channel between the two largest AI powers could be vital for managing cross-border AI safety risks. Carnegie Endowment for International Peace’s Scott Singer highlighted mutual concerns over regulatory gaps and crisis management, while New America senior fellow Sam Sacks described the talks as a critical opportunity to share observations and jointly respond to AI safety incidents.

“The U.S. and China have to come together in some form, or we’re both going to lose,” Sacks said, emphasizing that the potential damages posed by unmonitored frontier AI agents transcend geopolitical rivalry.