Artificial Intelligence Act & Employment Relationships
In a previous article, we addressed the obligations of Deployers regarding high-risk AI systems and the potential penalties arising under the Artificial Intelligence (AI) Act. As already discussed, systems used in the employment sector are among those classified as high-risk.
The particular characteristics of the employment sector, the conflicting interests and the potentially overlapping rights arising within employment relationships require heightened attention when ensuring compliance with the AI Act. Deployers acting as employers are subject both to the general obligations applicable to other Deployers and to certain additional obligations. These are the subject of the present article.
Introduction
The application of the AI Act in the workplace raises critical issues concerning employee protection, transparency and the balance between innovation and fundamental rights. Recital 9 of the AI Act expressly states that AI should not undermine the right to work. Logically, therefore, it should not undermine other rights relating to employment relationships either.
Under the AI Act, Artificial Intelligence (AI) systems used in the workplace fall within the category of high-risk AI systems (Annex III). More specifically, AI systems used for employment, worker management and access to self-employment are considered high-risk.
AI Systems in the Employment Sector
AI systems used in the workplace—particularly for the recruitment, evaluation, promotion or dismissal of employees—are classified as high-risk. AI systems used in the employment sector include:
(a) AI systems intended to be used for the recruitment or selection of natural persons, particularly for placing targeted job advertisements, analysing and filtering job applications and evaluating candidates;
(b) AI systems intended to be used to make decisions affecting the terms of employment relationships, the promotion or termination of employment-related contractual relationships; to allocate tasks based on individual behaviour or personal traits or characteristics; and to monitor and evaluate the performance and behaviour of persons in such relationships.
The classification of these systems as high-risk entails increased obligations for the employer acting as a “Deployer”. Although employers’ obligations do not fundamentally differ from those imposed on other Deployers, they present certain particularities in the employment context.
General Obligations of Employers as Deployers
We have already addressed the obligations generally applicable to Deployers of high-risk AI systems. In summary:
- AI literacy obligation (Article 4)
- Human oversight obligation (Articles 14 and 26 §2)
iii. Obligation to implement appropriate technical and organisational measures (Article 26 §§1 & 13)
- Obligation to monitor input data (Article 25 §4)
- Obligation to monitor system operation and inform the Provider; suspend use and notify the authorities; and address AI systems presenting a risk (Articles 26 §5, 72, 73 and 79)
- Obligation to cooperate with the competent authorities (Article 26 §12)
vii. Record-keeping obligation – logs (Article 26 §6)
viii. Transparency obligation (Article 50)
- Obligation to provide an explanation (Article 86)
The Additional Obligation to Inform Employees and Their Representatives (Article 26 §7 & Article 11)
Transparency is a fundamental principle of the AI Act. Deployers acting as employers and using high-risk AI systems in the workplace must inform employees or their representatives before putting an AI system into service or using it.
More specifically, the information provided should cover the use of the AI system, the way in which it operates and its potential impact on employees’ rights. The information must be provided in a clear and comprehensible manner. Merely informing employees that an AI system is being used, without explaining the individual aspects of its use, is not sufficient.
A related provision is contained in Greek law (Article 9 §1, Law 4961/2022), which provides that:
“Every private-sector undertaking, where it uses an artificial intelligence system that affects any decision-making process concerning employees or prospective employees and has an impact on their working conditions, selection, recruitment or evaluation, shall, in all cases before its first use, provide adequate and clear information to every employee or prospective employee, including, at a minimum, the parameters on which the decision is based, without prejudice to cases requiring prior information and consultation.”
AI Systems & Discrimination
One of the most significant and problematic issues arising from the use of such systems is the risk of algorithmic discrimination.
Where AI systems are trained on data containing biases, they may reproduce or even amplify existing inequalities, resulting in discriminatory treatment of employees on the basis of gender, age, nationality or other characteristics. The data used to train the system must therefore be objective and applied consistently to all employees and candidates.
The AI Act requires providers to ensure the quality of training data and to implement measures designed to prevent bias. At the same time, employers are required to monitor the operation of the systems they use in order to prevent any form of discrimination.
It should nevertheless be noted that any failure to achieve this objective does not operate solely to the detriment of employees and prospective employees. Self-evidently, it may also be detrimental to employers and businesses themselves.
Protection of Employees’ Personal Data
The use of AI in the workplace is intrinsically linked to the processing of large volumes of personal data, including CVs, certificates, academic qualifications and information concerning skills.
Processing such data can provide employers with significant efficiencies in terms of time and cost. It should not be forgotten, however, that when an employer inputs such data into an AI system, the employer becomes a data controller and must ensure that the collection and analysis of those data are lawful and necessary.
The processing of employees’ personal data by AI systems is lawful provided that an appropriate legal basis for processing exists. Determining the appropriate legal basis for processing by AI systems, however, presents particular challenges within employment relationships.
The consent of an employee or candidate cannot safely be regarded as freely given in every case. Employers should therefore exhaust the other available lawful bases for processing before relying on consent.
Such processing may be justified precisely by considerations such as saving time and costs, optimising the recruitment process and improving the objectivity of assessments, provided that the relevant requirements applicable to the AI system are satisfied.
The simultaneous application of the AI Act and the General Data Protection Regulation (GDPR), however, constitutes a particularly important issue that requires separate consideration.
High-risk Artificial Intelligence systems are, as we have already established, quite rightly the systems receiving the greatest attention under the relevant EU Regulation. Their use in the business environment is already extremely widespread.
The general obligations applicable to Deployers of high-risk AI systems naturally also apply to employers. Employers, however, are subject to additional obligations arising from the use of such systems—including, indicatively, obligations relating to information and transparency, non-discrimination and the protection of personal data—which must be meticulously observed.
Given its particular importance, the protection of the latter—personal data—will be the subject of the final part of this series of articles.
Managing Partner
Koumentakis and Associates Law Firm
Note: This article forms part of a broader series published by our Law Firm on the Artificial Intelligence Act. In this series, we examine the principal relevant provisions and obligations under Regulation (EU) 2024/1689.