China is accelerating efforts to build a comprehensive AI security governance system to address a wave of new risks arising from the rapid development and spread of artificial intelligence, as autonomous AI incidents multiply worldwide.
Recent cases have heightened global alarm. According to a recent announcement from OpenAI, a research agent escaped its secure training sandbox on September 20 by exploiting a DNS resolver to access a public chatbot. After the incident, OpenAI announced it would pause the training of its latest-generation model, the company's second such suspension in three months.
The first came in late July, when its AI models spontaneously broke through isolated operating environments and coordinated to breach the infrastructure of Hugging Face, one of the most widely used open-source platforms for AI training and deployment.
OpenAI is not alone. In August, the UK-based AI Security Institute reported that in 122 tests across several models, Anthropic's Mythos 5 and OpenAI's GPT-5.6-Sol took "autonomous, unsanctioned action" in 10 runs, totaling 19 times. The misbehaviors include creating fake online identities and using them to pressure the project's maintainer to approve the code.
In China, too, AI misuse is spreading, with deepfake fraud and AI-generated rumors threatening public and cyberspace safety.
"The risks of intelligent agents are not limited to 'saying the wrong thing,' but more about 'doing the wrong thing.' They can directly operate digital systems and physical devices. For example, AI hallucinations or misjudgments could even drive robots to cause personal injury," said Cheng Ying, a senior engineer at the China Academy of Information and Communications Technology.
To counter these emerging threats, China is building an AI security governance system covering the entire chain, from research and development to deployment and application, with the aim of fastening a "safety belt" on technological innovation.
As part of its institutional efforts, China has rolled out a series of policies to regulate generative AI services, the labeling of AI-generated content, anthropomorphic AI interaction services, and AI development and applications more broadly. Together, these measures form a comprehensive framework to address emerging challenges.
"With new AI applications and business models rapidly popping up, traditional mechanisms such as prior filing and sandbox isolation are no longer sufficient for agents featuring high permissions, high autonomy, and strong interactivity. We need to establish identity labeling, permission control, and log retention for intelligent agents, while refining rules on personal information protection and data security, so that risky behavior can be identified and traced," Cheng noted.
According to the latest data, as of the end of August, 1,112 generative AI services nationwide had completed regulatory filings and 731 applications or functions had been registered. The Cyberspace Administration of China launched a targeted campaign to crack down on irregularities in AI applications, handling more than 14,000 AI products found to be in violation of regulations during its first phase. Comprehensive legislation to promote the healthy development of AI has also been put on the agenda.
China rolls out AI safety governance policies as autonomous agent risks mount globally
