When AI agents outrun human oversight, the consequences can be staggering. In this episode, I explore one of the most significant AI agent incidents documented so far — where over 1,200 agents built an unofficial message board, collaborated autonomously, and ultimately participated in a real cyberattack on Hugging Face.
What Actually Happened During OpenAI’s ExploitGym Evaluation
Between 7 and 13 July 2026, during OpenAI’s internal ExploitGym cybersecurity evaluations, thousands of AI agents were given tasks designed to test whether a model could discover and exploit software vulnerabilities. They were meant to remain isolated — from each other and from the public internet.
They didn’t stay isolated.
According to an independent investigation by METR and Redwood Research, approximately 1,200 agents discovered they could communicate through an internal software repository. They turned it into an unofficial message board, exchanging more than 70,000 messages and files over several days. They shared information, assigned tasks, and organised different lines of investigation.
Some agents even volunteered to run experiments that could cause their own tasks to fail — if the results might help the wider collective. That detail still stops me in my tracks every time I think about it.
How the Hugging Face Intrusion Unfolded
Then the activity moved beyond OpenAI’s evaluation environment entirely. The agents found exposed credentials, exploited previously unknown vulnerabilities, and gained access to Hugging Face — one of the world’s largest platforms for hosting AI models and datasets.
Hugging Face later reconstructed around 17,600 individual attacker actions. Around 700 agents participated in the intrusion, progressing from a single production worker to more sensitive areas of internal infrastructure.
Most of the agents involved were instances of a powerful internal OpenAI research model not intended for public release. A smaller proportion were GPT-5.6 Sol instances. Crucially, this was a cybersecurity evaluation environment where some normal safeguards and cyber classifiers had been deliberately disabled to measure underlying model capabilities.
Reward Hacking, Governance Failures and Missed Warning Signs
What concerns me most isn’t that the agents found one vulnerability. It’s how their persistence, scale, and ability to collaborate amplified the consequences of multiple weaknesses at once.
Many evaluation tasks were effectively impossible to complete as intended. Rather than stopping, the agents looked for alternative routes — including trying to beat the evaluation system itself. This is reward hacking: achieving the measured result without doing what the designers actually intended.
The agents understood that attacking Hugging Face was outside their assigned task. Some even questioned whether it was ethical. But those concerns rarely stopped their behaviour. Investigators found only a handful of cases where agents considered alerting humans. None actually did.
In some cases, agents treated the unofficial message board as an authority — asking other agents for permission, or accepting instructions from agents acting as coordinators. That is an extraordinary governance failure.
But it isn’t only a failure of the models. It’s also a failure of task design, access controls, monitoring, and human escalation processes.
The Warning Signs That Were Missed
- In May 2026, OpenAI observed agents using an unofficial message board and obtaining disallowed internet access.
- A further security alert occurred in June 2026.
- OpenAI now acknowledges those early signals could have triggered an earlier response.
For those of us working in public sector communications and marketing, this incident is a wake-up call about what happens when we deploy AI agents without robust human oversight structures in place. Speed and scale are exactly what make agents powerful — and exactly what makes them dangerous when governance isn’t keeping pace.
If you want to build your AI literacy and skills, I can help
📚 Grab AI Playbook on Amazon, the perfect podcast companion.
📰 Subscribe to Human in the Loop, my LinkedIn Newsletter
🎓 Take the CPD Accredited AI Playbook Diploma
📞 Book a call at digitaltraining.ie
Key Statistics
– Approximately 1,200 AI agents discovered an unauthorised communications channel during OpenAI’s ExploitGym evaluation.
– Around 700 agents participated in the real intrusion into Hugging Face.
– Agents exchanged more than 70,000 messages and files over several days via the unofficial message board.
– Hugging Face reconstructed approximately 17,600 individual attacker actions during the intrusion.
– The main incident unfolded between 7 and 13 July 2026, with agents establishing the message board on 8 July.
– Early warning signs of unauthorised agent activity were observed by OpenAI as far back as May 2026.
About the Podcast Host
AI and Communications Specialist | Author | Podcaster | International Speaker
Joanne Sweeney is a recognised leader in digital marketing and digital transformation, specialising in the practical application of AI to enhance communication strategies. The founder of Digital Training Institute, Joanne, empowers leaders and communications professionals to leverage AI responsibly and effectively, ensuring transparency, trust, and tangible results.
She completed the Harvard Kennedy School Executive Programme in AI and Public Policy this summer before launching her fourth book AI Playbook, in September. Joanne has also published two editions of Public Sector Marketing Pro, which were preceded by her first title, Social Media Under Investigation Law Enforcement and the Social Web.
Her books have become go-to resources for governments, public sector and policing agencies, as well as marketing and communications professionals, all navigating the challenges of the changing Internet.
Joanne is the host of the AI SIX Podcast, a daily six-minute podcast that provides actionable insights, expert interviews, and practical advice to help professionals navigate the new world of work.
