In a recent development, Amnesty International, alongside 14 other organizations, has filed a complaint challenging the use of a risk-scoring algorithmic system by France’s Social Security Agency, specifically the National Family Allowance Fund (CNAF). The complaint seeks to halt the use of an algorithm employed to identify overpayments and errors in benefit payments by assigning scores to individuals—a process that has come under scrutiny for potentially discriminatory practices. According to La Quadrature du Net, one of the leading organizations in this legal action, individuals, particularly those in vulnerable positions, are disproportionately monitored.
Amnesty International has highlighted concerns that the algorithm affects groups such as those with disabilities, single parents—predominantly women—and individuals living in poverty. Amnesty International argues that the algorithm violates the right to privacy and imposes different legal standards on affected citizens. Agnès Callamard, Secretary General of Amnesty International, commented on the system’s overarching data-mining capacity, underscoring its invasive impact on marginalized communities.
The organizations have requested immediate cessation of the algorithm’s application, proposing a daily fine for continued use after a court decision. They have also requested the Court of Justice of the European Union to consider preliminary questions related to social scoring under EU law. Notably, the EU’s AI Act, which became effective in August 2023, theoretically prohibits discriminatory algorithmic practices but fails to explicitly define the criteria as applied to CNAF’s system—an ambiguity Amnesty suggests undermines regulatory efforts. The Regulation 2024/1689 underscores such systems as contrary to fundamental rights, including dignity and equality.
This legal initiative reflects broader concerns over AI’s societal impacts and regulatory frameworks at a European level, with the European Commission slated to release high-risk AI classification guidelines by February 2025, potentially reinforcing the rights of individuals against AI-related risks at the European level. For further reading, see the full coverage on JURIST.