A recent report by AI Forensics, a European non-profit organization, has raised serious concerns about Hugging Face, an open-source AI model hosting platform. The platform is increasingly being exploited as a primary conduit for generating non-consensual intimate images, with inadequate measures in place to prevent such misuse.

Previously, Hugging Face became embroiled in controversy when users, through a 'data preservation campaign' on the social media platform Discord, transferred more than 5,000 models capable of creating pornographic images without consent to Hugging Face. These models, which had been removed from Civitai, were then stealthily re-uploaded under new names and through external links. The models predominantly target well-known film and television actresses, internet celebrities, and live streamers. Some models even come complete with publicly available example images and recommended usage instructions, prompting widespread concern from human rights and gender equality advocates.

Although Hugging Face's official guidelines clearly prohibit the uploading of content that harasses, bullies, or contains non-consensual pornography, the nature of these generative models complicates enforcement. Typically, these models do not directly produce sensitive images; controversial results only surface when users provide specific instructions. This places Hugging Face in a legal and ethical gray area regarding content moderation.

At present, it is uncertain whether Hugging Face will launch investigations and take down the offending models. However, this incident has once again underscored the ethical and regulatory challenges surrounding 'consent rights' in content creation, especially as generative AI becomes more accessible.