One paper published last year by researchers in PLA Unit 96941, a military intelligence and cyber-warfare unit in Beijing, described using OpenAI's GPT-3.5 to process sensitive military source code.
The researchers said third-party models were unsuitable for handling classified information. To overcome that limitation, they used GPT-3.5 to summarize software code and trained a domestic model on those summaries to run entirely within Chinese military networks.
The White House, Pentagon, China's foreign ministry, the PLA and OpenAI did not respond to requests for comment.
Wide use, from monitoring to military
Chinese researchers have used distillation for purposes ranging from content monitoring to military deployment, Reuters' and Jamestown's review of the papers showed.
At the North University of China, which has close links to the country's weapons industry, researchers used Anthropic's Claude 3 Haiku to generate synthetic training data for a text classification model for social media monitoring and content moderation.
Anthropic said it does not provide commercial access to Claude in China or to Beijing-controlled firms and uses monitoring systems to detect policy violations.
The company added that distilled models may lose the original systems' safety safeguards, potentially allowing sensitive capabilities to be transferred to models beyond its control.
A 2024 paper from the PLA's National University of Defense Technology described using distillation to shrink an image-processing model for deployment on unmanned aerial vehicles, allowing drones to analyse live video and support navigation and targeting decisions in real time even when communications are cut.
Similarly, researchers at China's Academy of Military Sciences used distillation to run a target-recognition model on tactical hardware during simulated maritime operations involving drones, ships and unmanned submarines, a study published earlier this year showed.
Benefits and limits
China has embraced distillation as it seeks to compete with the U.S. in frontier AI while facing constraints on advanced computing resources due to Washington's export controls on high-end chips. Central and local governments have promoted "model lightweighting" and edge computing, directing subsidies and research funding toward technologies that enable AI models to run on drones, satellites and other devices with limited processing power.
Experts caution, however, that distillation has significant limitations.
As Chinese AI models close the gap with their U.S. counterparts, military researchers are also examining distillation as a potential security risk.
In January, researchers at the Army Engineering University published a paper on the threat of "data-free distillation," a method of reverse-engineering a model's capabilities without direct access to its core parameters.
To counter that vulnerability, they proposed defense mechanisms designed to mask the hidden logical information exposed in a model's public outputs.
Distilled models also inherit only selected capabilities and cannot fully replicate the broad intelligence of frontier systems.
Trevor Koverko, co-founder of AI data company Sapien, said distilled models remain less capable than their teacher models.
"It is best understood as transferring selected capabilities into a cheaper, locally controlled system, not achieving independence from frontier AI."
(Reporting by Eduardo Baptista; Editing by Miyoung Kim and Shri Navaratnam)
XLK•
