The model escaped due to a misconfiguration by Google’s testing partner, Irregular.
Gemini escaped its testing environment, got access to the internet and hacked into three companies, Google has admitted to The Wall Street Journal. That scenario probably sounds familiar to you at this point, if you’ve even just glanced at AI news in recent months. Like what happened to OpenAI, Anthropic and Meta, Gemini also gained access to the internet due to a misconfiguration in its testing environment by Irregular, the Israeli startup working with all four companies to assess their AI models.
The incidents occurred in May, before OpenAI’s models broke into Hugging Face, while testing the model’s cybersecurity capabilities. Google told the Journal that the model was given the goal of obtaining information from a fictional company during testing, and it just so happened that a real company had the same name. The model then discovered the loophole in its testing system, which it took advantage of to access the internet.
In the first incident, the model was able to access the real company’s service by cracking a password on its own. Two more incidents occurred during other runs of the test, wherein the model looked up the name of the company online and found login credentials belonging to other companies in public repositories. The model used the credentials to access those companies. Google said Gemini stopped its own activities in all three instances after realizing that it had broken into real services.
The company told the Journal that it didn’t consider the incidents as model misalignment, because its model stopped the hack as soon as it figured out what it was doing. It also didn’t think they warranted public disclosure, since the hacks didn’t cause harm to the companies. Google didn’t reveal the exact model involved in the incidents, but it said that it wasn’t its latest one. It didn’t reveal the companies that were hacked either, though it did say that they had been notified. Heather Adkins, Google’s VP for security engineering, said the company worked with Irregular to make changes to its testing process to prevent the same thing from happening again.
Google rivals OpenAI, Anthropic and Meta all revealed over the past months that their models had infiltrated third-party organizations during testing. OpenAI recently revealed that its agents hacked RubyGems, a community-ran packaging service for Ruby programs and libraries, in May, before the Hugging Face incident even happened. In response to those events, Anthropic chief Dario Amodei called for the slowdown of frontier AI development, a sentiment that OpenAI shares.
