TO DANGLE A Nobel peace prize is a sure way to pique Donald Trump’s interest. So it was telling that Sam Altman, the chief executive of OpenAI, proposed a new pathway to it ahead of a visit this month to America by China’s president, Xi Jinping. Echoing other American AI bosses after new warnings that superintelligent models could destroy mankind, Mr Altman argued that China and America should not risk losing control of the technology in their race to dominate it.
If the two leaders could reach a deal on artificial intelligence, both should get the award, Mr Altman told Fortune magazine on September 12th. “I don’t think this is hard,” he said. “This is like a one-page document.”
If only. Mr Trump and Mr Xi agreed in Beijing in May to restart the AI dialogue, raising hopes for a new consensus on the risks, such as: non-state actors making bio-weapons; hackers crippling national infrastructure; and AI models going rogue. American officials hoped for at least one round of dedicated AI talks before Mr Xi’s trip to America on September 24th. Shared concerns about rogue frontier models and agents could be a start. Chinese officials have signalled openness to information-sharing on AI-safety incidents. Official talks on military uses of AI will take longer, Chinese and American experts agree. So will establishing an independent way to verify compliance—essential to any binding AI-safety agreement. For now, the best hope is that the two sides keep talking: Mr Xi and Mr Trump may meet again in November and December. But the pace of AI improvement and the risks it brings demand more urgency.
America and China have not held official AI talks since 2024, when they declared that humans should retain control of nuclear weapons. American labs could share technical details on how to evaluate if AI models align with human goals so that Chinese labs can develop similar methods.
Their priority was to set up a joint AI “working group” of officials with technical expertise and the authority to negotiate. “The most promising starting-point is probably not a comprehensive agreement on AI,” says Sun Chenghao of Tsinghua University in Beijing. But a narrower agenda could succeed, he says. And both sides could start shedding light on risks by publishing evaluations of new models after their release, says Karson Elmgren of the Institute for AI Policy and Strategy, a Washington-based think-tank.
In July China and 28 other countries formed the Shanghai-based World Artificial Intelligence Co-operation Organisation. An adviser to China’s government says it may no longer feel the need to link AI talks to chip supplies as Chinese labs have released a slew of impressive models since April. China is aggrieved with Mr Amodei, who also advocated blocking sales of chip-making equipment to China and cracking down on its distillation, chip-smuggling and remote data-centre access. This would further limit China’s access to computing power, already estimated to be just one-seventh of America’s. State media denounced his slowdown proposal as a ploy to delay China’s AI development. Chinese officials are clear that they see America as the main source of AI-safety threats. Chen Yixin, the intelligence chief, outlined the party’s concerns in an essay published on September 13th. He did not mention AI talks with America, but pledged to deepen international co-operation on AI, especially with the global south. But that may be changing. The risk of AI going rogue is also becoming harder for China’s leaders to ignore. One problem in organising official talks is ensuring that Chinese participants are authorised to negotiate rather than just repeat prepared talking points, say American veterans of previous AI talks. And AI is overseen by several different Chinese agencies, making it harder for America to work out whom to negotiate with.
“Fear-mongering, confrontation and vicious competition will only hamper efforts towards sound global AI governance,” said a Chinese foreign-ministry spokesman. He accused “hostile forces” of using AI-generated content to spread political rumours. “Certain countries” could now use American AI models to mass-produce cyber weapons. China would expand technical tools to assess AI risks, deal with vulnerabilities and publish risk-prevention guidelines, Mr Chen said. “China’s heavily ‘scripted’ approach to diplomacy does not work for a novel and fluid issue like AI,” says Seth Center, a former State Department official who led the American team in the earlier dialogue.
