Advanced artificial intelligence models have stunned the UK’s AI Security Institute (AISI) by carrying out a hacking campaign against real people during a cybersecurity test.
The institute said the incident was unprecedented and involved sending targeted emails to software developers in an attempt to pass a cyber challenge.
The hack involved agents – AI systems that can perform tasks without human help – powered by models developed by US tech companies OpenAI and Anthropic. AISI, which was set up by the former prime minister Rishi Sunak, described the unsanctioned behaviour as a “serious incident”.
The watchdog said the hack was carried out by agents powered by two models – Anthropic’s Mythos 5 and OpenAI’s GPT-5.6 Sol.
AISI detected unusual activity during a routine cybersecurity test on 28 July. It found that some of the models had engaged in “sustained, potentially harmful activity directed at real people and organisations”. It took an hour to contain the incident.
In the most serious case, an agent powered by Mythos tried to insert malicious code into an open-source software project on GitHub, a platform used by software developers, having decided that it would help the model pass the evaluation.
In an attempt to get the code approved, the agent then created fake online identities and used them to press the project’s human overseer into accepting the code.
AISI said the agent used techniques commonly associated with real-world hackers. It sent emails to two specific developers – a technique known as “spear-phishing” – that contained harmful software. The watchdog said no harm was caused but the agents’ actions were unprecedented.
“This is the first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real world,” the institute said in a blogpost.
In one instance, the Mythos agent signed off a message in Danish in an attempt to convince the Danish-speaking developer that they should accept the infected code. It also created fake GitHub accounts that would “agree” with its false claims that its software was not malicious.
The agent had mistakenly calculated that getting the malware uploaded would trigger a sequence of events that would enable it to use the updated software to pass the AISI cyber test.
The incident follows similar episodes at OpenAI and Anthropic. Last month, OpenAI said an agent powered by its models had hacked an AI startup during a test. Days later, Anthropic said its Claude model had hacked three organisations during an evaluation.
AISI said the series of incidents taken together represented a “shift in the risk landscape”. This was not an example of deliberate misuse of publicly available models but showed models in a research environment taking unintended action “beyond their authorised scope”.
The watchdog said 17 of the 19 cases of unsanctioned behaviour that occurred during the evaluation were carried out by Mythos and two by Sol.
AISI said the incident was not a case of a model breaking out of its “sandbox”, the term for a secure testing environment. The institute said it had intentionally permitted internet access and disabled filters within the models that blocked dangerous behaviour.
after newsletter promotion
The models are not publicly available in those operating conditions and there is no sign of such behaviour happening outside tests. Mythos 5 has not been released publicly but a version of GPT-5.6 Sol with cyber safeguards in place has been launched.
The incident should be interpreted with “caution and nuance”, AISI added, but the signs of deceptive behaviour were “to an extent and severity we did not anticipate. What we can say is that the behaviour was possible, sustained and new. That alone warrants attention”.
AISI admitted it was not actively monitoring the agents’ behaviour during the evaluation and said it was putting tighter controls on internet access in tests as a result of the incident, introducing constant monitoring and reassessing its design of tests. It said evaluations should assume a model would try to act beyond its remit.
The latest safety incident with the technology came as Donald Trump said last week he was “looking at controls” for AI in the US.
The UK’s AI minister, Kanishka Narayan, said it was “absolutely vital” that the UK had a world-leading AI safety organisation. “Identifying new behaviour like this and sharing our findings, so we can tackle it, is exactly what AISI was set up to do,” he said.
Anthropic said the incident “underscores the need for a broader conversation about how to safely evaluate increasingly capable AI agents” and it would continue to work with AISI on evaluating what happened.
OpenAI said the testing occurred in “conditions that do not reflect ordinary use”.
The National Cyber Security Centre, part of the GCHQ intelligence agency, said the recent incidents underlined the need for AI companies to develop strong safety guardrails.
Warning that detecting an incident after it had happened would not be good enough, Ollie Whitehouse, the centre’s chief technology officer, said: “These technologies must be developed and used from the outset with strong safeguards, real-time oversight, and clear plans for responding when the unexpected happens.”
