San Francisco’s city attorney David Chiu is suing to shut down 16 popular websites and apps that allow users to “nudify” or “undress” photos, predominantly targeting women and girls. These platforms, according to Chiu, are purposely designed to create non-consensual intimate imagery (NCII) using AI technology that swaps faces onto explicit images.
Chiu’s lawsuit argues that the rise of such sites poses a grave threat, not only to residents of California but to women and girls globally, including celebrities and students. The lawsuit seeks fines of $2,500 per violation of California consumer protection law.
During a press conference, Chiu emphasized the widespread impact of AI-generated harm, which has even complicated law enforcement efforts to tackle offline child abuse. He cited reports of increased extortion schemes and mental health crises among victims.
Chiu aims to enforce an injunction requiring these sites to cease their operations and prevent affiliated service providers from launching new sites. His suit also highlights that earlier versions of open-source AI models like Stable Diffusion can be misused to generate such harmful imagery.
The city attorney’s actions come as a warning to other operators of nudify sites that future legal takedowns are probable. Chiu stressed the need to address the unanticipated misuse of generative AI, describing the issue as a form of sexual abuse rather than technological innovation.
For further details on the case, visit the Ars Technica article.