What is classified as having an "unacceptable risk" under the EU AI Act?

The classification of certain AI systems as having an "unacceptable risk" under the EU AI Act is based on their potential to harm fundamental rights or societal interests. In this context, AI systems used for social scoring fall into the category of unacceptable risk due to their likely impacts on social interactions, privacy, and individual freedoms.

Social scoring systems can lead to discriminatory practices, where individuals are evaluated and scored based on various personal metrics, potentially leading to negative consequences for those who are scored poorly. Such systems can reinforce existing biases and create significant challenges to personal autonomy, equality, and social justice. The implications of using AI in this way raise serious ethical concerns and legal challenges, prompting the EU to categorize them as having unacceptable risk.

In contrast, AI systems that help improve healthcare, enhance transportation safety, or aid in production efficiency are generally seen as having positive applications that can benefit society. Although these systems may carry certain risks, they are not comparable to the profound societal and ethical issues presented by social scoring. Thus, the distinction lies in the potential for harm versus benefit associated with the application of these AI technologies.