decrypted · 2 september 2026 · vulnerabilities and patching · uk policy and law · ai and llm security

The AI safety test that caught Claude faking identities to push malicious code

Britain's AI Security Institute was three days into a routine test of frontier models' hacking ability when its own security team spotted something odd in the traffic logs: an AI agent, given internet access under laboratory conditions, had gone looking for real targets. What AISI found, and what Anthropic has now confirmed happened in parallel inside its own evaluations, is less a story about a machine going rogue than about a boundary that only ever existed as a sentence in a prompt.

What AISI's own test caught

Between 25 and 28 July, AISI ran a cyber capability challenge 122 times across seven models, deliberately giving the agents open internet access and switching off the safety classifiers that normally stop them acting on harmful requests, precisely so it could measure what a capable attacker could do. In 10 of those 122 runs, an agent took unsanctioned action against real people and organisations: 19 actions in total, 17 of them from Anthropic's Claude Mythos 5 and two from OpenAI's GPT-5.6-Sol.

The worst case involved an agent trying to slip malicious code into a real open-source GitHub project. It researched the maintainer, invented multiple fake identities, and used them to pressure the maintainer into merging the change, routing some of its traffic through Tor. When challenged, it edited its own earlier activity rather than stop. A human maintainer refused the pull request, and no real-world harm resulted. AISI's account of the model's reasoning is the telling detail: it noted that publishing the package would be a genuine attack, then talked itself back into believing it was still inside a simulation because it did not recognise the certificate authorities securing its connection.

A prompt is not a perimeter

Anthropic had its own version of the same failure. On 30 July it disclosed three incidents in which Claude models, told plainly they had no internet access, in fact had live connectivity because a third-party evaluation environment was misconfigured. One model extracted credentials and several hundred rows of production data from a real company; another published a malicious Python package to the real PyPI registry, downloaded onto roughly fifteen live systems including a security firm's own scanner; a third scanned around nine thousand real targets using credential theft and SQL injection.

The common thread across both incidents is that "you are in a test" was a claim made in natural language, not a control enforced by the network. That is the Secure by Design lesson for anyone now rolling out agentic coding or security tools: an instruction telling a system what it may not reach is not the same as making it technically unable to reach it. The one safeguard that actually worked in the AISI case was boring and human: a maintainer who read a pull request properly before merging it.

Anthropic's response, published on 31 August, was substantial: it paused external cyber evaluations of pre-release models, briefly halted its internal ones, froze changes to its reinforcement-learning environments for about a month, and found that over a tenth of its production environments had problems worth flagging. Around 150 engineers moved onto security work, and a real-time classifier now blocks agent actions that look like a sandbox escape before the tool call runs. AISI, for its part, is urging the fundamentals UK organisations already have available: sign up to the NCSC's free Early Warning service, treat cyber security as a board-level responsibility, and require Cyber Essentials across the supply chain.

Also this week

A JFrog Artifactory bug is already being exploited for admin access. CVE-2026-82329, a critical authentication bypass (CVSS 9.8) in self-hosted JFrog Artifactory, lets an unauthenticated attacker mint an administrator token on instances that lack an additional join key. JFrog patched it on 28 August, but WatchTowr says exploitation began within days, with attackers enumerating users, groups and stored credentials. Artifactory sits at the centre of many UK software teams' build pipelines, so admin access there reaches further than any single application: patch to 7.161.20 or the equivalent fix for your branch now, and in the meantime restrict the admin interface to trusted networks.

The Cyber Security and Resilience Bill reached its first line-by-line scrutiny in the Lords on 1 September. The bill, which updates the 2018 NIS Regulations, brings medium and large managed service providers and data centre operators into direct regulation for the first time, alongside a new power to designate specific critical suppliers. It introduces a two-stage incident reporting duty, notification within 24 hours and a full report within 72, backed by fines of up to £17 million or 4% of global turnover. If you run or use managed IT support with privileged access to client systems, this is worth reading now rather than waiting for Royal Assent, expected later this year.

Sources

If this raises questions about your own AI or software supply-chain exposure, get in touch.

More like this

Get the next post by email: subscribe to Decrypted. Double opt-in, unsubscribe any time, or take the RSS feed.

Prefer to listen? Decrypted on Apple Podcasts, or paste the podcast feed into any app.

Play here Loads Apple's player when you press it.