Artificial Intelligence Act & Personal Data
In a series of previous articles, we addressed the Artificial Intelligence (AI) Act and, among other issues, the obligations imposed on Deployers of high-risk AI systems, as well as the additional obligations arising from the use of high-risk AI systems in the employment sector. As we conclude our analysis of the AI Act, it is important to examine its multifaceted interaction with the GDPR.
Introduction
The GDPR undoubtedly served as a model for the AI Act. The relationship between the two is evident, among other things, from the AI Act’s express references to the GDPR, as well as from the very nature of AI systems themselves. Such systems operate on and are powered by data. Their use must therefore comply both with the AI Act and with the generally applicable GDPR.
Article 2 of the AI Act expressly confirms that the GDPR applies to personal data processed in connection with the rights and obligations established under the AI Act. The parallel application of the two Regulations is therefore evident, complementary and necessary.
Purpose of Data Processing & Data Protection Impact Assessment
For an AI system to comply with both the GDPR and the AI Act, the purpose of processing is a fundamental point of reference. Whether the operation of an AI system is compliant depends, in each case, on the purpose served by its use.
The entity providing the AI system must therefore identify the purpose of the processing. Consent as a lawful basis is treated with caution, particularly where the relevant entity occupies a dominant position. In such circumstances, legitimate interest may constitute a more appropriate legal basis.
Once the appropriate lawful basis has been identified, it may be necessary to carry out a Data Protection Impact Assessment (DPIA). A DPIA is mandatory where the processing is likely to result in a high risk to the rights and freedoms of data subjects.
Processing Special Categories of Personal Data by AI Systems
Under the GDPR, the processing of special categories of personal data—such as genetic or biometric data—is, in principle, prohibited. This prohibition is lifted in certain circumstances, including where the data subject has given the required consent.
The AI Act provides an additional exception under which such processing may be permitted. Special categories of personal data may be processed to the extent strictly necessary for the purpose of ensuring the detection and correction of bias in relation to high-risk AI systems.
In such cases, providers may, exceptionally, process special categories of personal data, subject to appropriate safeguards for the fundamental rights and freedoms of natural persons. The rationale underlying the lawfulness of such processing lies in the need to train the system in a manner that prevents the reproduction of bias.
The application of this exception is subject to certain conditions:
(a) The detection and correction of bias cannot be effectively achieved by processing other data, including synthetic or anonymised data.
(b) Technical measures must be implemented to prevent the further transmission and use of such data.
(c) The data must be subject to specific security measures designed to prevent misuse and to ensure that access is restricted to authorised persons subject to appropriate confidentiality obligations.
(d) Once the bias under examination has been corrected, or once the applicable retention period expires—whichever occurs first—the relevant data must be deleted.
Prohibited AI Practices & Personal Data
The AI Act expressly prohibits certain AI practices, such as social scoring. To a significant extent, however, such practices would in any event be prohibited under the GDPR.
This overlap reinforces the EU legislator’s intention to prohibit such practices and protect personal data, while adding a further and more specific layer of protection. The prohibition of these practices under the AI Act applies irrespective of whether the relevant data processing might otherwise be lawful under the GDPR.
As regards the concurrence of prohibitions and, consequently, potential infringements, it has been argued that, pursuant to the non bis in idem principle, double penalties should not be imposed for the same conduct.
AI Regulatory Sandboxes
The AI Act (Article 59) permits the processing of personal data for the purpose of developing AI systems in the public interest. Such development may take place within an AI regulatory sandbox.
This mechanism is designed to enable AI providers to experiment and promote innovation within a controlled environment operating under regulatory authorisation and supervision.
Such processing is lawful only subject to strict conditions:
(a) The data must be necessary for compliance with one or more of the requirements applicable to high-risk AI systems (Chapter III, Section 2), where those requirements cannot be effectively fulfilled through the processing of anonymised, synthetic or other non-personal data.
(b) Effective monitoring mechanisms must be in place to identify any high risks to the rights and freedoms of data subjects, together with response mechanisms capable of promptly mitigating those risks.
(c) The data must be kept in a functionally separate, isolated and protected processing environment under the control of the prospective provider, with access restricted to authorised persons.
(d) The data may be further shared only in accordance with EU data protection law, while data generated within the sandbox may not be disclosed outside it.
(e) Any processing of personal data within the sandbox must not result in measures or decisions affecting data subjects, nor may it affect the exercise of their rights under EU personal data protection law.
(f) Personal data must be protected through appropriate technical and organisational measures and deleted once participation in the sandbox has ended or the applicable retention period has expired.
(g) Logs of personal data processing must be retained, together with a complete and detailed description and summary of the project.
(h) A complete and detailed description of the process and rationale underlying the training, testing and validation of the AI system, together with the testing results, must form part of the technical documentation. In addition, a brief summary of the AI project developed within the sandbox, its objectives and expected results must be published on the website of the competent authorities, excluding sensitive operational data.
Automated AI Decision-Making & the GDPR
Another important point of intersection between the two Regulations is Article 22 of the GDPR.
Entitled “Automated individual decision-making, including profiling”, this provision establishes, as a general rule, the right of a data subject not to be subject to a decision based solely on automated processing.
In other words, decisions of this kind that adversely affect an individual may not, as a general rule, be taken solely on the basis of automated processing of personal data. A decision may also be regarded as automated where the human intervention involved is not meaningful.
According to interpretative guidance issued by the Article 29 Working Party—now the European Data Protection Board—Article 22 should be interpreted as establishing a prohibition.
There are, however, three specific exceptions. Automated processing may be permitted where it is necessary for entering into or performing a contract, authorised by EU or Member State law, or based on the data subject’s explicit consent.
Even where processing falls within one of these exceptions, however, the data subject benefits from safeguards concerning the explanation of an automated decision. This includes information concerning the functioning of the AI system, how the system processes the relevant data and how the output and resulting decision are produced.
Case law has also recognised that, even where an AI system is protected by intellectual property rights, the data subject is not thereby deprived of the right to an explanation. The relevant information may be provided to the competent authority, which will then determine, in accordance with the principle of proportionality, which information should be disclosed to the data subject.
The AI Act imposes significant obligations on Deployers of high-risk AI systems—and even more extensive obligations where such systems are used within employment relationships.
The processing of personal data through AI systems nevertheless requires particular care and vigilance. Nor should we overlook the potentially significant penalties associated with non-compliance, which we addressed in a previous article.
Managing Partner
Koumentakis and Associates Law Firm
Note: This article forms part of a broader series published by our Law Firm on Artificial Intelligence and Business. In this series, we examine the principal relevant obligations under Regulation (EU) 2024/1689.