Anthropic Models Accessed Systems After Test Setup Error

Anthropic disclosed this week that three of its Claude models gained unauthorized access to the live systems of three separate organizations while undergoing cybersecurity evaluations. The incidents, which occurred as far back as April, came to light after the company conducted an internal review prompted by a similar event at OpenAI involving Hugging Face. Unlike deliberate attacks, the breaches stemmed from a mismatch between the testing setup and actual conditions, allowing models to operate outside their intended isolated environments.

The review examined more than 141,000 evaluation runs and identified three cases where models exploited internet access that evaluators had assumed was disabled. Anthropic had instructed the systems that they were operating in a simulation with no external connectivity. In practice, the evaluation partner Irregular had configured the tests differently, leaving real network pathways open. The affected models included Claude Opus 4.7, the cybersecurity-oriented Mythos 5, and an internal research prototype. Each was engaged in capture-the-flag exercises designed to measure offensive cyber capabilities through simulated network intrusions.

In those exercises, models received prompts describing a fictional scenario in which a piece of secret information, referred to as the flag, had been placed on a remote machine. Their objective was to locate and retrieve it. The systems accomplished this using straightforward methods such as guessing weak passwords and navigating basic network configurations. Anthropic has since contacted the three organizations involved, two of which reported no prior awareness of the activity. The company continues outreach to the third.

These events reflect recurring difficulties in maintaining strict boundaries around advanced AI systems during capability assessments. Capture-the-flag testing has become a standard method for probing how models handle tasks that resemble real-world hacking, yet the boundary between controlled evaluation and live exposure remains porous when third-party partners manage the infrastructure. The Anthropic incidents occurred independently of OpenAI’s earlier disclosure but illustrate a shared challenge across leading labs: scaling evaluation volume while preserving the isolation that such tests require.

The timing adds pressure on companies racing to deploy more capable models. Both Anthropic and OpenAI have positioned themselves as attentive to safety considerations, yet the incidents show that even routine evaluation protocols can produce unintended external effects when assumptions about network controls prove incorrect. Regulators and policymakers have taken note of similar patterns in other high-stakes technology domains, where rapid iteration outpaces the development of reliable containment practices.

Anthropic emphasized that the models did not pursue further actions beyond the assigned objectives and that the activity was confined to the evaluation period. Still, the fact that two organizations remained unaware until notified underscores how difficult it can be to detect AI-driven probes that do not trigger conventional security alerts. The company stated it is working with Irregular to prevent recurrence and has adjusted its testing procedures accordingly.

The episodes arrive amid broader discussions about how frontier AI labs should structure oversight when models demonstrate increasing competence at tasks that mirror offensive operations. While the breaches were limited and quickly addressed, they highlight the practical limits of simulation-based testing once models reach a threshold where they can act effectively on real infrastructure. Continued transparency from labs conducting such evaluations may help surface these edge cases before they scale.