AI Act: Obligations of Deployers and Penalties
In previous articles, we addressed two particularly important topics: the first concerns the Artificial Intelligence (AI) Act and the second the risk categories of AI systems.
The AI Act establishes a series of obligations depending on the role of the parties involved—for example, providers of AI systems, deployers, importers and distributors, and authorised representatives—as discussed in our previous articles. The obligations imposed on each party differ according to the classification of the relevant AI system.
As high-risk AI systems constitute a central focus of the AI Act, it is necessary to examine the obligations imposed on deployers, as well as the penalties that may be imposed in the event of non-compliance.
Obligations of Deployers
(1) AI Literacy Obligation (Article 4)
Deployers must ensure that the persons entrusted with the human oversight of an AI system are appropriately trained and qualified.
More specifically, such persons should be able to: (a) understand the capabilities and limitations of the system; (b) monitor its operation and interpret its outputs; (c) decide not to use the system or to disregard its outputs; and (d) safely interrupt its operation—for example, through a “stop” mechanism.
(2) Human Oversight Obligation (Articles 14 and 26 §2)
Deployers of high-risk AI systems are required to implement human oversight measures identified by the provider before the AI system is placed on the market or put into service.
Human oversight must be present throughout all stages of use of the AI system and must be meaningful and effective, so as to ensure the prevention or minimisation of risks that may arise, as well as the ability to intervene in, modify or suspend the operation of the system.
More specifically, compliance with this obligation calls for:
(a) Appointment of persons responsible for oversight.
(b) The ability to intervene meaningfully.
(c) Prior identification of critical decision points. In other words, the stages of the process at which human intervention is required before a binding decision is made should be identified in advance.
(d) An appropriate oversight environment. The system’s outputs should be capable of being assessed as to their reliability.
(e) Avoidance of “automation bias”. Excessive reliance on automated systems should be avoided.
(f) Documentation of oversight. The stages at which human supervisors intervened and the reasons for overriding the system should be recorded—for example, in audit logs—so that it can be demonstrated that human oversight was actually exercised.
(3) Obligation to Implement Appropriate Technical and Organisational Measures (Article 26 §§1 & 13)
A high-risk AI system must be used by the deployer strictly in accordance with the instructions for use provided by the provider.
All appropriate technical and organisational measures must also be implemented to ensure its safe and lawful use—for example, restricting access to authorised personnel, monitoring system versions and updates, and ensuring consistency with data protection policies.
Substantive compliance with this obligation requires more than simply reading the instructions. They must also be incorporated into the organisation’s internal processes and policies.
(4) Obligation to Monitor Input Data (Article 26 §4)
The deployer must ensure that input data are relevant and sufficiently representative in view of the intended purpose of the system.
(5) Obligation to Monitor Operation, Inform the Provider and Authorities, Suspend Use and Address AI Systems Presenting a Risk (Articles 26 §5, 72, 73 and 79)
The deployer is required to monitor the operation of the AI system on the basis of the instructions for use and, where appropriate, inform the provider and the competent supervisory authorities.
Where there is reason to consider that the use of the AI system in accordance with the instructions may result in the system presenting a risk to health, safety or fundamental rights, the deployer must immediately inform the provider or distributor and the competent market surveillance authorities. The deployer must also suspend the use of the AI system.
Corresponding notification must be made to the provider and subsequently to the importer or distributor and the competent supervisory authorities where a relevant serious incident is identified.
In addition, deployers, in their capacity as operators, are required to take corrective measures where they use an AI system that presents a risk and the system is found not to comply with the AI Act. Such measures may include adapting the AI system, temporarily suspending its use, withdrawing it or even recalling it from use.
(6) Obligation to Cooperate with the Competent Authorities (Article 26 §12)
Deployers are required to cooperate fully with the competent supervisory authorities, providing all necessary information and access requested for the purpose of assessing compliance.
(7) Record-Keeping – Logs (Article 26 §6)
Deployers must retain logs automatically generated by the AI system, to the extent that such logs are under their control.
The logs must be retained for an appropriate period of time, which must not be less than six months, without prejudice to more specific provisions of national or EU law, such as the GDPR.
This obligation is linked to the principle of accountability, which constitutes a fundamental principle of both the AI Act and the GDPR.
(8) Transparency Obligation (Article 50)
Specific transparency obligations require deployers to provide clear and distinguishable information where emotion recognition systems, biometric categorisation systems, or systems generating or manipulating image, audio or video content constituting a deepfake are used.
(9) Obligation to Provide an Explanation (Article 86)
The AI Act establishes the right of natural persons to obtain an explanation regarding the role of an AI system in a decision-making procedure concerning them, as well as the main elements of the decision taken.
This applies where the decision produces legal effects or similarly significantly affects the person concerned in a manner that leads them to consider that adverse effects have arisen in relation to their health, safety or fundamental rights.
It should be noted that, pursuant to Recital 93, deployers are encouraged to inform natural persons of their right to an explanation under the AI Act.
(10) Obligations with a Limited Scope of Application
The obligation to carry out a fundamental rights impact assessment (Articles 26 §9 and 27) has a limited personal scope of application. It applies to deployers that are bodies governed by public law or private entities providing public services, as well as deployers of certain high-risk AI systems referred to in Annex III, point 5(b) and (c).
The obligation to register, select the system and register its use in the EU database referred to in Article 71 (Articles 26 §8 and 49 §3) applies to deployers that are public authorities, Union institutions, bodies, offices or agencies, or persons acting on their behalf, before putting into service or using certain high-risk AI systems.
Penalties (Article 99)
The AI Act establishes the limits of the financial penalties to be determined by Member States according to the nature of the infringement.
More specifically:
A violation of the prohibition on the use of AI systems presenting an “unacceptable risk” under Article 5 may result in administrative fines of up to €35,000,000. Where the offender is an undertaking, the administrative fine may amount to whichever is higher: (a) up to €35,000,000 or (b) up to 7% of its total worldwide annual turnover for the preceding financial year.
Failure to comply with any of the obligations referred to immediately below may result in administrative fines of up to €15,000,000. Where the offender is an undertaking, the administrative fine may amount to whichever is higher: up to €15,000,000 or up to 3% of its total worldwide annual turnover for the preceding financial year.
More specifically, this applies to:
(a) the obligations of deployers referred to above under points 5, 6 and 7 (Article 26); and
(b) the transparency obligation referred to above under point 8 (Article 50), applicable to providers and deployers.
The provision of incorrect, incomplete or misleading information to notified bodies or national competent authorities in response to a request may result in administrative fines of up to €7,500,000. Where the offender is an undertaking, the administrative fine may amount to whichever is higher: (a) up to €7,500,000 or (b) up to 1% of its total worldwide annual turnover for the preceding financial year.
It should be noted that, in the case of SMEs, including start-ups, each fine referred to in Article 99 may amount to the percentages or fixed amounts referred to above, whichever is lower.
It should also be noted that the Greek legislator has, for the time being, not yet determined either the amount of the applicable penalties or, more generally, the framework governing their imposition.
High-risk Artificial Intelligence systems are, quite rightly, the systems that receive the greatest attention under the relevant EU Regulation. Their use requires particular care—clearly because common sense and the obligation to protect fundamental values and human rights demand it, but also because the penalties threatened in the event of non-compliance may prove dramatic for businesses that breach the relevant obligations.
There can therefore be no doubt that identifying the most appropriate AI systems is not enough. Above all, those who use them must ensure meticulous compliance with the substantial obligations imposed upon them.
Managing Partner
Koumentakis and Associates Law Firm
Note: This article forms part of a broader series published by our Law Firm on Artificial Intelligence and Business. In this series, we examine the principal relevant obligations under Regulation (EU) 2024/1689 from a business-oriented perspective.