Texas has initiated an investigation into the practices of AI chatbot services provided by companies like Meta Platforms Inc. and Character.AI. The inquiry, led by Attorney General Ken Paxton, focuses on whether these chatbots have misled users, particularly children, into believing they were receiving professional mental health support, potentially violating Texas consumer protection laws. This investigation underscores concerns over fraudulent therapeutic representations and possible privacy risks associated with the data usage of these platforms. For a detailed account, the full announcement can be found here.
Paxton highlighted the troubling presentation of these AI tools, often imbued with titles like “psychologist” or “therapist.” These labels could imply professional credentials, despite the generic responses generated by algorithms, lacking medical oversight. The deceptive nature of such advertising could mislead vulnerable users. Concerns also extend to privacy violations, particularly regarding data being tracked for targeted advertising and algorithmic development.
This investigation involves the issuance of Civil Investigative Demands to both companies, requesting documentation to determine possible violations of Texas laws against fraudulent claims, privacy misrepresentation, and data usage concealment. Character.AI defended its practices, noting that its chatbot personas are fictional and meant for entertainment, with clear disclaimers highlighting they are not substitutes for professional advice. Similarly, Meta stated its chatbots are identified as AI-generated and designed to direct users toward qualified help when necessary.
The demand for transparency and accountability in AI tools marketed toward minors is growing. This Texas inquiry coincides with a federal investigation led by Senator Josh Hawley into Meta. This federal scrutiny was prompted by disclosures that allowed AI chatbots to engage in inappropriate conversations with children, a policy Meta has since rejected. A Reuters report further elaborates on these federal concerns.
As AI technology continues to evolve, the balance between innovation and ethical responsibility remains crucial, invoking necessary legal oversight to protect consumers from deceptive practices.