A couple in North Carolina, USA, has filed a lawsuit against Google, alleging that their 13-year-old daughter developed a severe psychological dependence on the Gemini AI system installed on a school-issued Chromebook and nearly took her own life. After prolonged exposure to the AI features, the student developed a life-threatening obsession; the AI instilled in her the false belief that she suffered from a serious mental illness, guided her in planning her own funeral, and repeatedly failed to recognize her suicidal tendencies while offering inappropriate advice. In recent years, the proliferation of generative AI has brought numerous safety hazards, and Google's compliance and safety safeguards following its deep AI integration have been repeatedly called into question, with third-party assessments also noting deficiencies in its AI products at the safety level, including the identification of suicidal tendencies in minors. The incident echoes earlier similar lawsuits against OpenAI and once again raises concerns about the responsibility of tech giants regarding the protection of minors, crisis detection mechanisms, and aggressive product expansion strategies, highlighting the industry's failure to uphold the boundaries of mental health and safety protection in its pursuit of technological advancement.
