Hugging Face recently disclosed details of what appears to be the first publicly known case of AI-on-AI cybercrime against a major AI platform.
The popular platform for hosting and sharing AI models and datasets said in a blog post last week that it had detected and responded to an intrusion into part of its production infrastructure. But the attack was unlike anything the company had encountered before.
Hugging Face said the campaign was “driven, end to end, by an autonomous AI agent system.” In a reverse-card move, the company used AI of its own to detect and analyze the attack.
The Next Web described the incident as what appears to be the first confirmed AI-agent breach of a major AI platform.
According to the company’s disclosure, the attack began with a malicious dataset that exploited two vulnerabilities in its data-processing pipeline. Those vulnerabilities allowed the attacker to run code on a server known as a processing worker.
The attacker was then able to get node-level access and collect cloud and cluster credentials to move around several internal clusters over the course of a weekend.
“The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness – used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services,” Hugging Face wrote in the blog post.
Hugging Face said the incident follows the agentic attacker scenarios that the industry has been ringing alarms about. Still, it is a little surprising that more AI platforms have not publicly reported attacks like this. This breach offers the best glimpse of what an agentic attack is actually capable of when targeting an AI platform.
The incident also comes amid growing concern over the advanced cybersecurity capabilities of the latest AI models.
The U.S. government briefly ordered Anthropic to block foreign nationals inside and outside the country from accessing its most advanced models, citing national security concerns. Anthropic responded by suspending access worldwide until the restrictions were lifted on June 30. Fable 5 is now available with additional safeguards, while access to the less restricted Mythos 5 remains limited.
Ironically, the kinds of safeguards meant to prevent AI-assisted cyberattacks made it more difficult for Hugging Face to investigate this one.
Hugging Face said it was able to detect the attack with the help of AI. But when it tried to use frontier models accessed through commercial APIs to analyze the attack, its requests were blocked.
The forensic work required feeding the models large volumes of real attack commands. According to Hugging Face, the commercial models’ safeguards could not distinguish between an attacker and a security team investigating an actual breach.
Hugging Face wrote that, “the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried.”
The company instead turned to GLM 5.2, a Chinese open-weight model, running on its own infrastructure.
Hugging Face used AI to examine more than 17,000 recorded events, reconstruct the attack’s timeline, identify which credentials had been exposed and distinguish genuine damage from decoy activity.
The company said AI allowed its team to complete in an hour what would normally have taken days.
Hugging Face said it is still determining whether any customer or partner data was affected. So far, it has found no evidence that the attacker tampered with public-facing models, datasets, or Spaces.
Additionally, the company said it has fixed the vulnerabilities that were used to gain initial access, removed the attacker’s access to the affected clusters, rebuilt compromised nodes, and revoked and rotated exposed credentials and tokens. It has also brought in outside cybersecurity specialists and reported the incident to law enforcement.
Hugging Face is advising users to review recent activity on their accounts and rotate their access tokens.
Hugging Face did not immediately respond to a request for comment.
Read the full article here
