AI vs AI: the Hugging Face attack that offers a glimpse into the future
We were warned that the next wave of agentic AI would expose a whole host of security vulnerabilities, and it’s no surprise that these stories are now hitting the news. Hugging Face has reported that an autonomous AI agent conducted an end-to-end attack on its systems, carrying out thousands of individual actions.
AI is also being used defensively: Hugging Face used its own AI tools to fight back. Automated systems apparently identified suspicious activity, while LLMs helped reconstruct the timeline, identify compromised credentials and distinguish genuine damage from decoy activity. Timelines are being shortened, with work that usually takes days being compressed into hours. The AI vs AI battleground is well and truly underway.
But autonomous hacking is only one part of the picture. Security systems can be secure but be exposed by human vulnerabilities, which increasingly look like easy bait. Generative AI can produce text, images and (nearly) video which is indistinguishable with the real thing, and it’s being used with great enthusiasm by scammers - DeepStrike reporting phishing attack increases of 1,265% in 2025.
As we’ve seen in the news, the consequences of cybersecurity breaches can be severe. Businesses may lose personal or confidential data, suffer prolonged operational disruption or make accidental payments. They may also face regulatory investigations, GDPR fines, litigation from customers or suppliers and reputational damage.
These issues again call into question the regulation of AI. The UK continues to take a laissez-faire approach, with no AI-specific legislation being likely in the near-term. This means that, subject to compliance with other laws, models with impactful agentic AI capabilities can be released without any tests, checks or guardrails being required.
In the EU, frontier models like OpenAI’s GPT 5.5 and Anthropic’s Mythos are likely to be classified as General-Purpose AI Models with Systemic Risk. Accordingly, these providers are required to assess and mitigate systemic risk, as well as track, document and report on serious incidents and the corrective measures taken to address them. AI systems (using these GPAI-SR models) may separately constitute prohibited or high-risk systems depending on how they are used. While regulated, problems nonetheless arise here for agentic AI – Article 14, for example, requires the system to be designed with human oversight protections but there is no guarantee that these tools will be used by end users.
Realistically, this regulatory position is unlikely to provide any real comfort to businesses facing agentic AI attacks. The models are on the market and are causing damage. A few options for businesses to consider include:
- Investment in cybersecurity: businesses need stronger cybersecurity controls, independent verification of sensitive requests, proper oversight of suppliers and regular incident-response exercises. The UK Government’s Cyber Essentials scheme is a good starting point: Cyber Essentials scheme: overview - GOV.UK.
- Cyber insurance: this can cover business interruption, incident response and data-breach claims, although separate crime/theft cover might be needed for money stolen as part of a cyber attack.
- Contractual allocation of risk: it should be clear in your contracts who takes on responsibility if something goes wrong. Quick containment is often the best solution, so any contractual allocation of risk should also be supplemented by notification provisions where issues arise.
