Law enforcement agencies are increasingly using artificial intelligence to generate images of fictional minors as part of operations to catch child predators online. This practice was recently highlighted in a lawsuit filed by the state of New Mexico against Snapchat. According to the complaint, the New Mexico Department of Justice (DOJ) discovered that Snapchat’s algorithm allegedly facilitated the exploitation of minors by recommending their profiles to adult predators.
The DOJ’s undercover investigation involved creating a fictitious profile for a 14-year-old girl named “Sexy14Heather.” Although the account was set to private with no initial followers, it attracted attention from explicit and predatory accounts within days. One of these interactions included conversations with an individual who initiated sexually suggestive exchanges.
Images generated through AI are argued to be a less problematic alternative compared to previous methods, which used photos of young-looking adults or actual minors. Criminal defense attorney Carrie Goldberg noted that while AI decoys may pose fewer ethical issues, the proliferation of AI-generated child abuse material presents new challenges, particularly when it involves the creation and circulation of illicit images. She warns that such practices could lead to complications in legal proceedings where entrapment defenses might be cited, asserting the government induced otherwise non-criminal behaviors.
Goldberg expressed concerns regarding the ethical implications of AI-generated child sexual abuse material (CSAM), stressing the importance of strict guidelines within law enforcement to ensure responsible use of such technologies. With the rise of AI in these sensitive contexts, the debate around its legitimate and ethical uses continues to grow.
For more details, the original article from Ars Technica explores the background and details comprehensively. You can read more here.