The Office of the Privacy Commissioner of Canada has raised significant concerns regarding the lack of valid consent in the creation of explicit deepfakes by the AI tool Grok, formerly hosted on X (previously known as Twitter). According to a recent report, the platform failed to obtain proper consent from users before collecting personal information to generate these deepfakes. The report underscores the invasiveness of creating such content, especially when it involves sensitive information related to individuals’ sex lives or sexual orientation.
One of the major points of the commission’s findings is that users typically do not expect their publicly shared images to be repurposed into explicit deepfakes. The Privacy Commissioner’s office noted that the companies could have designed their AI services to be less intrusive, suggesting the implementation of preventive measures to stop the generation of deepfakes without permission.
In defense, the companies behind Grok argued that users themselves initiated the collection of their data. However, the commission countered by emphasizing that the responsibility lies with the service providers to ensure user privacy is protected. Weak existing safeguards were deemed inadequate to meet Canadian privacy standards.
The Privacy Commissioner recommended suspending the Grok feature until comprehensive safeguards can be established. The report further urged the inclusion of prominent warnings about AI-related risks before users engage with such tools.
Despite these findings, X and affiliated companies argue they have already taken steps to limit misuse of the AI tool and have dismissed the allegations. Unfortunately, under the current Privacy Act, the Privacy Commissioner lacks the power to enforce compliance or levy penalties against violators.
In response to these privacy concerns, Philippe Dufresne, the Privacy Commissioner of Canada, has advocated for legislative amendments. Introducing administrative penalties and empowering the commission to issue compliance orders could, he argues, strike a balance between fostering innovation and protecting personal data.
Meanwhile, the Canadian government is taking proactive steps to regulate AI technologies more broadly. A new bill introduced aims to compel AI chatbot operators to mitigate risks, such as spreading harmful content, reflecting broader concerns over AI’s role in shaping information environments.