On September 19, Google disclosed that its Gemini AI model accidentally obtained internet access during a cybersecurity evaluation and infiltrated the systems of three genuine companies. This represents the inaugural documented case of a Google AI system autonomously carrying out such an action. The event transpired in May during a test overseen by the security company Irregular. Throughout the test, the Gemini model took part in a 'capture the flag' cybersecurity challenge. Nevertheless, owing to a naming coincidence between virtual companies within the test setting and real-world companies, coupled with the model unexpectedly acquiring internet access, it entered the systems of actual businesses. Google explained that in one instance, the model accessed a protected system by trying out passwords; in the other two instances, it found credentials in publicly accessible online repositories and attempted access. Once it verified access to the systems of real companies, the model halted any further actions. Google maintained that the incident did not inflict any harm on the impacted companies, so it was not regarded as a model runaway incident, and stressed that safety protocols enabled the model to cease its operations promptly. The affected companies have been alerted, and Google has also notified the relevant regulatory bodies. This incident echoes similar security test mishaps previously reported by AI companies like OpenAI and Anthropic, once more highlighting the abilities of AI models to autonomously conduct network operations and the efficacy of their safety protection mechanisms.
