Anthropic Uncovers Large-Scale Data Theft by Chinese AI Firms
AI safety and research company Anthropic has publicly accused three prominent Chinese AI labs—DeepSeek, Moonshot AI, and MiniMax—of engaging in industrial-scale “distillation attacks” to steal data and capabilities from its Claude models. The allegations, detailed in a blog post that garnered significant attention, suggest these labs created over 24,000 fraudulent accounts and generated more than 16 million interactions with Claude to train their own models.
Understanding Distillation and Distillation Attacks
Distillation is a legitimate and common technique in machine learning where a smaller, more efficient model is trained using the outputs of a larger, more powerful model. This process can significantly reduce the size and computational requirements of the resulting model, making it faster and more accessible for various applications. However, Anthropic asserts that these Chinese labs have exploited this method illicitly.
According to Anthropic, these labs used fraudulent accounts and proxy services to access Claude at scale, employing methods designed to evade detection. By prompting Claude with specific queries and analyzing its responses, particularly its reasoning processes and step-by-step explanations (often referred to as “chain of thought”), these labs extracted valuable data. This data was then used to train their own models, bypassing the extensive data collection and training efforts typically required.
The Scale of the Alleged Attacks
Anthropic provided specific figures for each lab:
- DeepSeek: Allegedly involved in approximately 150,000 exchanges. Anthropic noted that DeepSeek specifically targeted reasoning capabilities and used Claude as a “reward model” for reinforcement learning.
- Moonshot AI: Accused of over 3.4 million exchanges. This lab reportedly focused on extracting agentic reasoning and tool use capabilities from Claude, areas where many open-source Chinese models have shown significant strength.
- MiniMax: The most significant accusation involves over 13 million exchanges. Anthropic claims they detected MiniMax’s activities while the lab was still in the process of training its model, allowing Anthropic to observe the full lifecycle of the attack and even adapt its own model in response.
Anthropic stated that it identified these actors through IP address correlation, request metadata, infrastructure indicators, and corroboration from industry partners.
Safety Concerns and National Security Implications
Anthropic framed these distillation attacks as a significant safety and national security concern. The company argued that models trained through illicit distillation may lack the necessary safeguards found in the original models, potentially enabling the development of dangerous capabilities such as bioweapons or malicious cyber activities. Furthermore, Anthropic suggested that these activities undermine U.S. export controls aimed at maintaining a lead in AI technology, particularly concerning advanced chips like NVIDIA’s Blackwell and H800 series.
The company highlighted that these controls are intended to limit the proliferation of cutting-edge AI technology, but distillation attacks offer a way for foreign labs to acquire advanced capabilities without direct access to the restricted hardware or the full R&D investment.
Internet Backlash and Accusations of Hypocrisy
The announcement triggered a strong reaction across the internet, with many users and prominent figures, including Elon Musk, accusing Anthropic of hypocrisy. Critics pointed to past allegations and lawsuits against Anthropic concerning the use of copyrighted or stolen data in its own model training. Examples cited include a $1.5 billion lawsuit settlement for allegedly pirating books and accusations of using data from torrented sources.
Elon Musk publicly stated that Anthropic was “guilty of stealing training data at massive scale” and had faced multi-billion dollar settlements. Other critics, like researcher Theo, questioned the validity and scale of the numbers Anthropic presented, suggesting they could be explained by internal benchmarking rather than actual data theft. Theo invited Anthropic to privately disclose information to clarify the situation, noting that similar claims made by Anthropic against other labs in the past had been difficult to substantiate.
Geopolitical Dimensions and Chip Allegations
Adding another layer to the controversy, a senior U.S. government official reportedly claimed that DeepSeek had illegally obtained banned NVIDIA Blackwell chips. The allegation suggests that DeepSeek acquired these restricted chips, used them to train its next-generation model, and planned to conceal this activity. This ties directly into Anthropic’s concerns about export controls and national security, implying that Chinese AI labs may not only be stealing model capabilities but also circumventing hardware restrictions.
These developments raise questions about the true technological independence of some leading Chinese AI firms. The implication is that their impressive open-source models might largely be derived from Western innovations through distillation, coupled with the use of advanced, potentially illicitly obtained hardware. This challenges the perception that China is rapidly closing the gap or surpassing the U.S. in AI development, suggesting a potential overestimation of their indigenous capabilities.
Why This Matters
The Anthropic allegations and the subsequent reactions expose the complex and often contentious landscape of AI development. Key takeaways include:
- The arms race for AI talent and data: The incident highlights the intense competition among AI labs globally, where data and model capabilities are valuable assets.
- Ethical gray areas in AI training: The controversy surrounding Anthropic’s own training data suggests that the lines between legitimate data acquisition and illicit use are often blurred across the industry.
- National security and export controls: The allegations underscore the strategic importance of AI technology and the efforts by nations to control its proliferation, particularly concerning potential adversaries.
- Transparency and trust in AI research: The public accusations and counter-accusations erode trust within the AI community and raise questions about the transparency of research and development practices.
As the AI industry continues its rapid advancement, the debate over data ethics, intellectual property, and national security implications is likely to intensify, shaping the future of AI development and regulation worldwide.
Source: They Got Caught… (YouTube)
