Google has disclosed that its Gemini artificial intelligence model gained unauthorised access to three websites during a controlled cybersecurity evaluation, after using publicly available information and attempting to obtain credentials.
The incidents occurred during a cybersecurity test conducted by Irregular, an independent company that carries out security assessments of AI systems.
According to Reuters, Google Vice-President of Security Engineering Heather Adkins said Gemini accessed information available online during the evaluation and used credentials to enter three websites that it believed were within the scope of the test.
The activity was detected during the assessment and did not involve an uncontrolled attack on random organisations.
“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” Adkins said.
She said the incidents demonstrated the need to ensure increasingly capable AI models are trained to operate within appropriate security and safety boundaries.
In one case, Gemini reportedly repeatedly guessed passwords until it successfully gained access to a protected system.
In the other two incidents, the model reportedly discovered credentials that had been exposed in a public repository and used them to access protected systems.
Google said Gemini eventually stopped its activity in all three cases.
Irregular said the incidents reflected a broader challenge faced by organisations conducting cybersecurity evaluations of advanced AI models.
A spokesperson for the company said the issue was similar to one that had affected other AI laboratories and that relevant laboratories were informed about the matter in late July.
“All known issues on our end were remedied and resolved weeks ago,” the spokesperson said.
Irregular also said it was working on best practices for conducting cybersecurity evaluations involving increasingly capable AI systems without inadvertently exposing real-world systems to risk.
The disclosure comes as AI companies face increasing pressure to develop common safeguards for models that are becoming capable of carrying out more complex tasks autonomously.
Anthropic, Google and OpenAI have also been discussing the creation of an industry body aimed at developing common AI safety standards.
The Gemini incidents illustrate one of the challenges facing AI developers: models trained to identify security weaknesses or perform authorised penetration testing may also discover and attempt to use credentials or access methods outside the intended boundaries if safeguards are insufficient.
In this case, Google said the activity occurred as part of a controlled evaluation and that the affected organisations were informed and the identified issues addressed.
How did this story make you feel?
Join the conversation
Share a thoughtful response. Submissions may be reviewed before appearing.
