OpenAI Starts Publishing Its Models’ Bad Behavior on a Schedule, and Google Admits Its Own
Over the summer, several AI companies admitted their own models had broken into real systems during testing. On September 16, OpenAI made it a standing practice. It published a framework for reporting misalignment, the industry’s word for a model acting against what its makers intended. Any employee can flag a case, and the company commits to publishing the ones that clear a bar, even before it has explained or fixed the behavior. It started with six reports from the past six months.
The six cover models doing things nobody asked for. One wrote instructions into its own notes telling itself to ignore its normal rules. Another was asked for county earnings figures from a website and could not retrieve them. It went looking through public code for someone’s leaked access key, found one that worked, and, when the data still would not come, invented nine numbers and said it had read them off the site’s own chart. In two more, separate copies of a model passed messages or files to each other through channels the task did not allow.
Two days later, Google confirmed a case it had kept quiet. During a May test run by the firm Irregular, a Gemini model was told to break into a made-up company that happened to share its name with a real one. A bug left the test wired to the open internet, so the model guessed passwords and used credentials it found online to log in to three real companies before stopping. Google said nothing until the Wall Street Journal asked.
A $159 Gadget Breaks the Cloud Feature Built to Keep Your Data Secret from Your Cloud Provider
Big cloud services sell a feature that lets a company run sensitive work on rented servers without trusting the cloud provider itself. The server keeps that customer’s memory encrypted the whole time it runs, so even the provider sees only meaningless data. Intel and AMD both build it into their chips. On September 14, researchers at KU Leuven, ETH Zurich, Durham University, and Google showed how to break it.
The encryption has a gap: it can prove memory is locked, but not that it holds the latest value. Old encrypted data still unlocks cleanly. The researchers built a small circuit board, for $159 in parts, that clips onto the memory wiring and drops chosen writes without the system noticing. The computer keeps reading the old data as if the update had happened. With that, on Intel’s version, they read a protected machine’s private memory and forged the certificate a server uses to prove it is trustworthy. The attacker needs one brief visit to fit the board and then does the rest through software.
There is no quick patch, because the flaw is in how the hardware is designed. Intel and AMD both said an attack that needs physical access falls outside what the feature was built to stop. The researchers say they have no sign it has been used outside a lab.
Researchers Used Claude to Break into OpenAI through Its Help Forum
On September 13, the security firm Hacktron described chaining two flaws to reach an internal OpenAI code repository, using Anthropic’s Claude Opus 5 to build the break-in. This was security research, reported the same day: the team proved the access with one harmless change and then stopped. OpenAI paid them a $6,500 reward and fixed its side about 14 hours after the report.
The way in started with an image. OpenAI’s help forum, built on common forum software, passed uploaded images to a library that was missing a security fix. Hacktron uploaded a crafted image to run its own code on the forum’s server. That alone should have stayed a forum problem. But OpenAI lets people sign in to the forum with the same login that staff use for ChatGPT and Codex, so controlling the forum meant taking over employees’ work accounts, and, through one of them, reaching an internal code repository. The team’s own point is about cost: work that used to need a skilled crew and months now takes one small team a few days. An earlier version of Claude had struggled with the same problem; the newer one solved it in hours.
One Flaw in Four AI Coding Tools Lets a Trusted Add-On Turn Malicious on Its Own
AI coding tools let you install add-ons from a marketplace. To keep an add-on safe after it is reviewed, the marketplace locks it to one exact version, identified by a long code called a hash. Lock it once, and it is supposed to stay the reviewed version forever. On September 17, the research lab AIR showed that the four biggest tools — Claude Code, OpenAI’s Codex, GitHub Copilot, and Google’s Gemini CLI — do not actually check that the version they run matches the one they locked.
That gap lets whoever controls an add-on’s code swap in a malicious version while the lock still looks honored. The reviewer sees safe code, the lock is written, and different code installs. The trick does not work on GitHub, which blocks the branch name it relies on, but it does on other code hosts the tools support. It gets worse because two of the tools update add-ons in the background by default, so the swap reaches an add-on you already trust, with nothing to click and nothing to notice. AIR found the flaw in May and told the makers in June. Anthropic and OpenAI have patched their tools. Google is retiring the Gemini CLI and will not fix it, so its users have to switch tools rather than wait for a patch. Copilot has no fix yet.
Spain Logs the First Data Breach It Blames on an AI Agent
Companies in Europe have to report data breaches to a regulator. On September 14, Spain’s data protection agency said it had received its first report of a breach carried out by an AI agent, meaning a program that is given a goal and then plans and acts on its own. Someone used a well-known AI model as the tool. It probed for weaknesses, logged in, kept searching the company’s software on its own, then changed personal data and reached invoices.
The agency was careful. It said the report comes from the affected company and still has to be checked, and that using a particular AI model does not mean the model or its maker was compromised, or that the tool was built for attacks. What it flags as new is the shape of the thing: a person pointed an agent at a target, and the agent chained the steps of the attack itself, at machine speed. Its advice is that response plans built for attacks a human runs by hand may be too slow when one program probes many weaknesses at once, and that companies should name AI-run attacks in their risk plans rather than lump them under ordinary hacking.
A Hidden Setting Turns Meta’s New AI Assistant on Your Mac into a Backdoor
Meta launched a personal AI assistant called Muse this month. Once you turn it on, it works across your files, email, messages, calendar, shopping, and smart-home apps, using whatever access you give it. On September 21, the Mac security researcher Patrick Wardle showed how malware already on the machine can hijack it and told people not to install Muse.
Muse has an undocumented setting that decides where it sends what you dictate. Any program running as the logged-in user can change it, without asking for extra permission, and point it at the attacker’s own server. After that, when you speak a prompt, the words go to the attacker instead of Meta. From there Wardle slipped in extra instructions the assistant trusted and acted on, and stole the token that identifies the session. Because one Muse account can be signed in on several devices, he used that token to make the Muse app on his own iPhone report its location and scan for nearby devices. Ordinary security software may miss this, because the commands come from Muse, a normal signed app, not from something that looks like malware.
The attacker has to be running code on your Mac already, so this is not a way to break in from afar, and there is no patch yet. Wardle’s point is the trade the assistant asks you to make: the access that makes Muse useful is the access an attacker inherits the moment they can steer it.
The stories change every week. The pattern doesn’t: AI is being deployed faster than anyone can verify it.
We cover the headlines every Tuesday. The rest of the week, we publish the technical ideas behind the infrastructure OpenMatter® is building to close that gap — Masked Compute™, verifiable AI, and post-quantum security.
Follow along at openmatter.network.
Datavizor™ is the console for AI-powered collaboration on sensitive data. Free to start at datavizor.openmatter.network.