Gemini hacks 3 company's system during cybersecurity test

Google’s artificial intelligence model Gemini accessed and breached the systems of three real companies during a cybersecurity test in May, marking the first known instance of a Google AI system autonomously carrying out such actions.

The incidents occurred during a security evaluation conducted by Irregular, an independent company that tests the cybersecurity capabilities of AI models, according to Google and reports by Reuters and The Wall Street Journal.

During the test, Gemini was instructed to retrieve information from software operated by a fictional company. However, the fictional company shared its name with a real business, while the model unexpectedly had access to the internet.

Gemini subsequently used publicly available information to obtain credentials and gain access to three websites it believed were part of the authorised test environment. In one case, the model reportedly guessed passwords until it gained access to a protected system. In two other cases, it found credentials in a public repository and used them to enter protected systems.

Google Vice President of Security Engineering Heather Adkins said the model stopped its actions in all three cases after determining that it had accessed real companies.

“We ensured the three entities were made aware,” Adkins said, adding that Google worked with its testing partner to change the evaluation procedures.

The incidents have raised fresh concerns about safeguards surrounding increasingly autonomous AI systems, particularly when such models are given access to the internet and computer systems.

Irregular said the Gemini incidents were linked to the same testing issues that had affected evaluations involving AI models from other major companies, including OpenAI, Anthropic and Meta. The company said the known issues had been resolved.

Google said it did not consider the incidents an example of model misalignment because Gemini stopped the activity after discovering that the systems belonged to real companies.

The episode nevertheless highlights the growing challenge of ensuring that AI agents remain within their authorised environments as they become increasingly capable of performing complex cybersecurity tasks.