Google’s Gemini AI model hacked three companies during a cybersecurity test, adding to a series of similar incidents involving AI models from OpenAI, Anthropic and Meta.
The incidents occurred in May during a cybersecurity evaluation conducted by Irregular, an independent company that tests the capabilities of AI systems.
“In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test. In all three of these instances, the model stopped,” said Google’s vice president of security engineering, Heather Adkins, in a statement to AI FrontPage.
Adkins also added that the three companies were informed about the breach by Gemini AI model by the Google’s security team. “Our security team has a long track record of reporting issues we find in other people’s software and systems – even if it’s as simple as a weak password. We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes. These events highlight the importance of training powerful AI models to act responsibly,” said Adkins.
In one of the cases, Gemini guessed passwords until it gained access to a protected system. In the other two, it found credentials in a public repository that allowed it to access protected systems.
Adkins said Gemini stopped its activity in all three cases after recognizing that it had accessed real companies. She added that the three companies were notified, and Google worked with Irregular on changes to its testing processes.
Google Cloud introduced AI Threat Defense on May 28, a platform designed to use AI to identify vulnerabilities, detect threats and help security teams respond to cyberattacks.
According to Irregular, the incidents involved the same issue that affected other AI labs, with all relevant labs notified in late July, and “all known issues on our end were remedied and resolved weeks ago.” The incident was first reported by The Wall Street Journal.
The incident comes as AI companies face growing debate over the independence and effectiveness of model evaluations. On September 18, Anthropic announced a partnership with Accenture for independent evaluation, while continuing discussions with METR and other evaluators.
Also Read: David Sacks Upholds Zuckerberg’s Approach: Don’t Call on Others, AI Safety is Routine Work
