Artificial Intelligence Act: Risk Categories
Artificial Intelligence Act: Risk Categories
In a previous article, we launched a series on the EU Artificial Intelligence Act. We addressed its objectives, scope and entry into force. As we proceed further with our analysis, it is necessary to understand the risks associated with AI through their classification—or, at least, the way in which the Regulation classifies them. This is the subject of the present article.
Risk Levels
Adopting, in part, a risk-based approach, the AI Act identifies four levels of risk for AI systems, depending on the risks inherent in each of them. Within this framework, it classifies AI systems into four categories: prohibited risk, high risk, limited risk and minimal or no risk.
The obligations imposed on those involved at each stage of the development and use of AI systems are intrinsically linked to—and correspond with—the respective level of risk.
Prohibited-Risk AI Systems
The AI Act establishes (Article 5) a category of AI systems that are prohibited from being placed on the market, put into service or used. This is due to the serious risks they pose to fundamental rights and human dignity.
More specifically, the placing on the market, putting into service or use of an AI system is prohibited where it:
(a) Uses subliminal, purposefully manipulative or deceptive techniques. Such techniques are prohibited where their purpose or effect is to circumvent free will or materially distort a person’s behaviour. This also applies where they impair a person’s ability to make an informed decision, causing that person to make a decision they would not otherwise have made, where such a decision causes, or is reasonably likely to cause, significant harm.
(b) Exploits vulnerabilities of a natural person or a specific group of persons due to their age, disability, or social or economic situation. In order for such a system to fall within the prohibited category, its purpose and/or effect must additionally be to materially distort the behaviour of such a person, or of a person belonging to such a group, in a manner that causes, or is reasonably likely to cause, significant harm to that or another person.
(c) Evaluates or classifies natural persons or groups of persons over a certain period of time on the basis of their social behaviour or personal characteristics through social scoring, resulting in specific detrimental or unfavourable treatment. Such evaluation or classification must additionally lead to detrimental or unfavourable treatment in a social context unrelated to the context in which the relevant data were originally generated or collected and/or to detrimental or unfavourable treatment that is unjustified or disproportionate to the social behaviour of the persons or groups concerned.
(d) Performs risk assessments of natural persons in order to assess or predict the risk of a person committing a criminal offence on the basis of profiling or behaviour. Such risk assessments must additionally be based solely on profiling or on the assessment of personality traits and characteristics.
(e) Creates or expands facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage.
(f) Infers the emotions of a natural person—for example, through the recognition of facial expressions or vocal patterns to identify stress, happiness or anger—in workplaces and educational institutions. An exception applies where the AI system is intended to be placed on the market or put into service for medical or safety reasons.
(g) Performs biometric categorisation based on biometric data of natural persons in order to infer their race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation.
(h) Uses “real-time” remote biometric identification systems in publicly accessible spaces for law enforcement purposes. An exception applies, however, where the use of such systems is strictly necessary for one of the objectives specified in the AI Act—for example, searching for victims of abduction or human trafficking.
High-Risk AI Systems
A substantial part of the AI Act is devoted to high-risk AI systems. These are systems that may potentially pose serious risks to the health, safety and fundamental rights of natural persons, as well as to the environment, democracy and the rule of law.
The AI Act establishes specific obligations concerning the use of high-risk AI systems. Within this framework, the assessment and mitigation of relevant risks, record-keeping and human oversight are essential requirements.
High-risk AI systems include those (Article 6 and Annex III) used:
(a) As safety components of products. More specifically, these are AI systems intended to be used as a safety component of a product—that is, systems incorporated into machinery or other products. Examples include interactive or smart toys affecting children’s safety, lifts and medical devices under the Medical Devices Regulation (MDR).
This category also includes an AI system that is itself a product covered by EU harmonisation legislation (Annex I), including systems used in civil aviation, two- or three-wheel vehicles, agricultural and forestry vehicles, marine equipment, railway systems, motor vehicles and unmanned aircraft.
(b) For biometric purposes, provided, of course, that their use is permitted under EU or national law. These include remote biometric identification systems, emotion recognition systems and biometric categorisation systems.
As already noted, the use of biometric identification systems by law enforcement authorities is, in principle, classified among prohibited AI practices. Their use is permitted only exceptionally, where it is limited in time and geographical scope and takes place only following judicial or administrative authorisation.
(c) In critical infrastructure, such as AI systems used as safety components in the management and operation of critical digital infrastructure. Examples include systems monitoring water pressure, controlling fire alarms in cloud computing data centres, or managing road traffic, gas, heating and electricity infrastructure.
(d) In education and vocational training. These include AI systems used to determine access or admission to educational and vocational training institutions, assess learning outcomes, determine the appropriate level of education, and monitor or detect prohibited behaviour by students during examinations.
(e) In employment. These include AI tools used for recruitment and selection, publishing job advertisements, analysing and filtering job applications, evaluating candidates, making decisions concerning the terms of employment relationships, promotion and termination of contractual relationships, allocating tasks and assessing employee performance.
(f) For access to essential public and private services. These include systems used by public authorities to assess the eligibility of natural persons for essential public assistance benefits and services, including healthcare; assess creditworthiness; conduct risk assessments and pricing in relation to life and health insurance; evaluate emergency calls; and establish priorities in emergency situations.
(g) For law enforcement purposes, where such use is permitted under EU or national law. Examples include AI systems used to assess the reliability of evidence or to evaluate the risk that a person may commit or reoffend in relation to a criminal offence.
(h) For migration, asylum and border control management, where their use is permitted under the relevant EU or national law. Such systems may be intended for use by public authorities as lie detectors, for risk assessments—for example concerning security, irregular migration or health—or to assist public authorities in examining applications for asylum, visas or residence permits.
(i) In the administration of justice and democratic processes. Such systems may be intended to assist in interpreting facts and the law and in applying the law to a specific set of facts, or to perform similar functions in alternative dispute resolution.
This category also includes AI systems intended to influence the outcome of elections or referendums or the voting behaviour of natural persons when exercising their right to vote in elections or referendums.
As already noted, the majority of the provisions of the AI Act regulate matters relating to high-risk AI systems. The obligations established by the Regulation therefore primarily concern systems falling within this category.
Limited-Risk and Minimal-Risk AI Systems
As regards AI systems classified as limited risk—such as chatbots—the AI Act establishes limited obligations. These primarily concern transparency and disclosure regarding the use of AI, enabling those interacting with such systems to make informed decisions and, where they consider it necessary, discontinue their use.
AI systems classified as presenting minimal or no risk are not subject to the compliance obligations laid down by the AI Act.
As already noted, the AI Act adopts a risk-based approach: Artificial Intelligence systems are classified as prohibited-risk, high-risk, limited-risk and minimal- or no-risk systems.
Only limited obligations apply to the latter two categories. The first category, being prohibited altogether, leaves little room for further discussion. The majority of substantive compliance obligations concern high-risk AI systems. Precisely because of their importance and the regulatory burden they impose on the parties involved, we will examine them separately in a subsequent article.
Managing Partner
Koumentakis and Associates Law Firm
Note: This article forms part of a broader series published by our Law Firm on the Artificial Intelligence Act. In this series, we examine the key provisions and obligations of Regulation (EU) 2024/1689, always from a business-oriented perspective.