1 Introduction
The uptake of AI systems has the potential to bring several individual and societal benefits and foster innovation and global competitiveness. While the
This chapter considers conversations about regulating AI, focusing on the legislative initiatives that have been, or are in the process of being, undertaken within the European Union (âEUâ).1 In April 2021, the European Commission published a proposal for an EU regulatory framework on Artificial Intelligence. It was intended as future-proof legislation with flexible mechanisms that would allow it to adapt and evolve with new regulatory challenges and concerns. This proposal represents the first-ever attempt to enact a horizontal regulation of AI within the EU. The Council of the EU adopted its common position on such proposed legislation in December 2022,2 and the European Parliament (âEPâ) in June 2023.3 In December 2023, the Council and the EP reached a final political agreement on the text of the AI Act, which the EP has now passed.4
The AI Act thus has a dual concern and relative function: free trade/elimination of trade barriers and individual and societal impact/protection of health, safety and fundamental rights. It is based on Article 114 of the Treaty on the Functioning of the European Union (âTFEUâ), the âinternal market clauseâ. However, to the extent that it contains specific rules on the protection of individuals about the processing of personal data concerning restrictions of the use of AI systems for (i) remote biometric identification for law enforcement, (ii) risk assessments of natural persons for law enforcement and (iii) biometric categorisation for law enforcement, and in as far as those specific rules are concerned, the Act is based on Article 16 TFEU (the right to the protection of personal data).6
The Act will apply to private and public entities, regardless of their location within or outside the EU, as long as the AI system is available in the Union market or its use affects people in the EU. Among the Actâs most important elements are the requirement of transparency and the need for rigorous safety checks before an AI system is released into the public domain. The broad shape of things to come in Europe has been set. However, this is continually being challenged and impacted by scientific and technological developments in the sector that the law sets out to regulate, such as the emergence during the legislative process of heightened debate and controversy surrounding regulating generative AI as a result of the public availability of âlarge language modelsâ (âLLM sâ) like ChatGPT.
This chapter will provide an overview and commentary on the AI Actâs regulatory strategies and general structure. An article-by-article analysis of the Act is beyond the scope of this book chapter. Rather, we seek to shine a light on selected fundamental aspects of the legislation. The chapter proceeds as
2 Regulate What? Defining âAI systemsâ
The AI Act regulates âAI systemsâ, defining an AI system as one that is âmachine-basedâ, âdesigned to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environmentsâ.7 The Councilâs General Approach on the Act narrowed the definition to systems developed through machine learning approaches and logic â and knowledge-based approaches;8 this position was later abandoned,9 and a technologically-neutral definition made it to the final draft text.
A âgeneral-purpose AI (GPAI) modelâ means âan AI model, including where such an AI model is trained with a large amount of data using self-supervision at scale, that displays significant generality and is capable of competently performing a wide range of distinct tasks regardless of the way the model is placed on the market and that can be integrated into a variety of downstream systems or applications, except AI models that are used for research, development or prototyping activities before they are placed on the marketâ;10
A âgeneral-purpose AI (GPAI) systemâ means âan AI system which is based on a general-purpose AI model and which has the capability to serve a variety of purposes, both for direct use as well as for integration in other AI systemsâ.11
The definition of a GPAI model is broad. It is so broad that it has been argued that it could âbe interpreted in such a way as to potentially include a vast variety of technological casesâ.12 Typical examples of GPAI models are generative AI models13 such as LLM s. These models are a tool for processing and generating natural language and have many potential applications in language translation, content generation, and conversational AI. They use deep learning, which involves training a neural network on vast amounts of textual data, such as books, articles, and web pages. An LLM aims to learn the patterns and relationships between words and phrases in natural language to generate coherent and meaningful text. This involves analysing the syntax and semantics of language and understanding the context in which words are used. ChatGPT-4, developed by OpenAI and released on the public web in March 2023, is one of the most well-known examples of LLM s.
LLM s can be considered a broader category encompassing foundation models. This type serves as a starting point for building other, more specialised models and constitutes an example of a general-purpose AI system. The term
Emily M. Bender has coined the term âstochastic parrotsâ for LLM s,14 which is now commonly used when referring to such models because, while impressive in their ability to generate natural language, they ultimately do not understand the meaning of the language they are processing (they just âparrotâ). LLM s are incapable of true reasoning or understanding; they merely rely on statistical patterns in data to generate responses. As a result, they are prone to errors and biases, as they can perpetuate stereotypes and other problematic patterns in language. Furthermore, they are not always transparent about how they arrive at their responses.15 The output of âhallucinationsâ, or made-up answers, has indeed triggered a GDPR-based complaint by NOYB against OpenAI (the company that has developed ChatGPT) on account of the output of false information about people and the inability or unwillingness of OpenAI to comply with the data access request and in so doing be transparent about their training data.16
3 Why Regulate? Threats to Fundamental Rights and Values
While there is no doubt about the promise of AI to make things better, the drive for regulation stems from certain fears relating to the development and deployment of AI systems, combined with a perceived insufficiency of
A more nuanced viewpoint (embraced by these authors) holds that AI, a creation of humans, will be controlled by (some) humans, but its deployment may result in the enslavement of others. However, since (some) humans control its development, AI could be developed to protect fundamental or human rights and values âby designâ.
AI systems have the potential to unpredictably harm peopleâs life, health, and property. They can also affect fundamental values on which western societies are founded, leading to breaches of fundamental rights of people, including the rights to human dignity and self-determination, privacy and personal data protection, freedom of expression and assembly, non-discrimination, or the right to an effective judicial remedy and a fair trial, as well as consumer protection.20
The EU must ban the use of discriminatory AI systems which disproportionately affect people from marginalised communities, including
migrants, refugees and asylum seekers. Such technologies profile people and communities, claiming to âpredictâ crimes or âidentifyâ people who supposedly pose a security risk, even leading to them being denied the right to asylum. EU lawmakers must not miss this opportunity to prohibit the use of certain AI-based practices and protect the rights of migrants, refugees, and asylum seekers against harmful aspects of AI. Use of mass surveillance technologies, such as retrospective and live remote biometric identification tools must also be banned. The proposed law must also ban discriminatory social scoring systems that prevent people from accessing essential public and private services, such as child support benefits and education.21
In this vein, the AI Act does prohibit certain AI practices. It considers, among others, that âAI systems providing social scoring of natural persons by public or private actors may lead to discriminatory outcomes and the exclusion of certain groups. â¦â22
AI is broad enough to include environments many of us already interact with daily, such as technology-powered search engines or social networking sites. Manipulative and/or deceptive practices may significantly impact individuals and societies; for example, digital nudging techniques exploit human vulnerabilities and the methods used by social media to make them addictive and keep users glued to the screen for as long as possible.23
Apart from the addictive element, the potential manipulation of online platforms (âintentional manipulation of their service, including by inauthentic use or automated exploitation of the service, as well as the amplification and potentially rapid and wide dissemination of illegal content and of information that is incompatible with their terms and conditions, to use the language of
Beyond the spread of disinformation, âfilter bubblesâ â an algorithmic bias that skews or limits the information an individual user sees on the Internet â are also problematic for democratic discourse, as it means that an individualâs online experience is not open to alternative, contrasting viewpoints to those already held.25
Thus, search engines and digital platforms such as social media platforms can have a deleterious impact at both the individual and the societal levels, threatening individual autonomy and dignity, as well as the democratic fabric of society and the rule of law. These developments in our online life experiences directly result from the profit-making target of the big businesses behind them.
It is important to protect fundamental rights in all situations. It has been argued that fundamental human rights can or should not be treated and evaluated equally with business (that is, profit) interests.26 However, this is not to say that the freedom to conduct a business should not be considered.27 The matter with AI is that, like other tools, it can be used for good or for bad. It can
4 How to Regulate? A âRisk-Basedâ Approach
The AI Act adopts a ârisk-basedâ approach and lays down a âhorizontalâ (as opposed to sector-specific) legal framework for AI that aims to ensure legal certainty; this is desirable as regulating all AI systems in an equally heavy-handed manner would arguably suppress innovation and merely help to strengthen the already dominant market players.31 However, the difficulty in applying such an approach lies in deciding how or according to the criteria âhigh-riskâ to be determined. It has been suggested that initial criteria must be developed for assessing what is âhigh riskâ â thus determining whether an obligation to proceed to a full impact assessment is triggered.32
The Act does not, however, include any such criteria. It merely precludes AI systems that pose an unacceptable risk, listing several prohibited practices. Thus, it identifies and heavily regulates AI systems with high risks and briefly mentions systems with limited risks. It also includes a new Chapter V on general-purpose AI (âGPAIâ) models, which speaks of GPAI models with
4.1 Prohibited AI Practices
The AI Act prohibits eight practices deemed to present unacceptable risks. These relate to manipulation, social scoring, the assessment of the risk of an individual committing a criminal offence, the compilation of facial recognition databases through untargeted data scraping, the inference of emotions in workplaces or education institutions, biometric categorisation systems and âreal-timeâ remote biometric identification (âRBIâ).34
Thus, the Act prohibits the placing on the market, putting into service, or use of an AI system:
That âdeploys subliminal techniques beyond a personâs consciousness or purposefully manipulative or deceptive techniques, with the objective, or the effect of materially distorting the behaviour of a person or a group of persons by appreciably impairing their ability to make an informed decision, thereby causing them to take a decision that they would not have otherwise taken in a manner that causes or is reasonably likely to cause that person, another person or group of persons significant harmâ;35
That âexploits any of the vulnerabilities of a natural person or a specific group of persons due to their age, disability or a specific social or economic situation, with the objective, or the effect, of materially distorting the behaviour of that person or a person belonging to that group in a manner that causes or is reasonably likely to cause that person or another person significant harmâ;36
For âthe evaluation or classification of natural persons or groups of persons over a certain period based on their social behaviour or known, inferred or predicted personal or personality characteristicsâ, with the social score leading to the detrimental or unfavourable treatment of certain natural persons or groups of persons that is unjustified or disproportionate to their social behaviour or its gravity, and/or in social contexts that are unrelated to the contexts in which the data was originally generated or collected;37
For âmaking risk assessments of natural persons in order to assess or predict the risk of a natural person committing a criminal offence, based solely on the profiling of a natural person or on assessing their personality traits and characteristicsâ,38 although this prohibition does not apply to systems âused to support the human assessment of the involvement of a person in a criminal activity, which is already based on objective and verifiable facts directly linked to a criminal activityâ;39 That âcreate or expand facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footageâ;40
To âinfer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasonsâ;41 and
That âcategorise individually natural persons based on their biometric data to deduce or infer their race, political opinions, trade union membership, religious or political beliefs, sex life or sexual orientationâ42 (âbiometric categorisation systemsâ); this prohibition does not cover any labelling or filtering of lawfully acquired biometric datasets, such as images, based on biometric data or categorising of biometric data in the area of law enforcement.
The Act also prohibits the use of âreal-timeâ RBI systems in publicly accessible spaces for law enforcement unless and in so far as such use is strictly necessary for certain objectives as further specified by it.43 In such latter cases, RBI systems must be deployed âonly to confirm the identity of the specifically targeted individualâ, âbe authorised by national lawâ, and âcomply with necessary and proportionate safeguards and conditionsâ about their use by such national law. Furthermore, the relevant law enforcement authority must have completed a fundamental rights impact assessment concerning the RBI system and registered it in the EU database.44
Each use of an RBI system for law enforcement requires a prior authorisation granted by a judicial authority or by an independent administrative authority âwhose decision is binding of the Member State in which the use is to take
Apart from including some new prohibited practices, some changes have also been made to the wording of the original four provisions. For instance, the provision prohibiting the use of AI systems that exploit individualsâ vulnerabilities has been extended to cover persons vulnerable due to their social or economic situation, where it previously only covered âage, physical or mental disabilityâ, and the prohibition of using AI for social scoring now also covers private actors, where it previously only applied to public authorities. Likely, the provision prohibiting the compilation of facial recognition databases through the untargeted scraping of online and CCTV footage was introduced due to the heightened awareness resulting from the Clearview AI and PimEyes cases.48 The final draft text also saw an increase in the requirements to be fulfilled for the lawful use of RBI systems in publicly accessible spaces for law enforcement purposes, including having a national law authorising such use, conducting a fundamental rights impact assessment, registering such systems in an EU database, and notifying their use to additional competent authorities such as data protection authorities. This more stringent approach attempts to strike a balance between the exigencies of the law enforcement sector and the resistance of civil society, witnessed through the public consultation process,49 as
4.2 High-Risk AI Systems
âHigh-riskâ AI systems are dealt with in Chapter III of the Act, which provides for the classification of and requirements for such systems, the obligations incumbent on providers and deployers, and standards for such systems. This section focuses on the classification of AI systems as âhigh-riskâ.
An AI system is classified as high-risk under the Act if: (i) it is itself a product or a safety component of a product covered by certain Union health and safety harmonisation legislation (listed in Annex I, such as toys, machinery, lifts or medical devices) and it is required to undergo a third-party conformity assessment in order to be placed on the market or put into service according to such legislation52 or (ii) it falls within a category listed in Annex III.53 The areas set out in this Annex are the following: biometrics; critical infrastructure; education and vocational training; employment, workers management and access to self-employment; access to and enjoyment of essential private services and essential public services and benefits; law enforcement; migration, asylum and border control management; administration of justice and democratic processes.
The Commission can amend the list in Annex III through delegated acts. The Commission may include sub-areas within the existing ones if the AI system poses risks similar to those of an existing in-scope AI system, but cannot add entirely new areas.57 Under a provision introduced by the Councilâs General Approach,58 the Commission may remove high-risk AI systems from the list, provided such systems no longer pose significant risks to fundamental rights, health or safety.59
4.3 Limited Risk AI Systems
Providers shall ensure that AI systems intended to interact directly with natural persons are designed and developed in such a way that the natural persons concerned are informed that they are interacting with an AI
system, unless this is obvious from the point of view of a natural person who is reasonably well-informed, observant and circumspect, taking into account the circumstances and the context of use. (â¦)61
4.4 Minimal Risk AI Systems
The AI Act allows other types of applications to be legally developed. This could include applications such as AI-enabled video games or spam filters. Article 95, on âcodes of conduct for voluntary application of specific requirementsâ, is the only relevant provision.
4.5 General-Purpose AI Models
General-purpose AI (GPAI) models are regulated in Chapter V of the Act, distinguishing between generic GPAI models, GPAI models âwith systemic riskâ, and âfree and open-source licenceâ GPAI models.
A GPAI model is classified as one with systemic risk62 if (a) it has high-impact capabilities evaluated based on appropriate technical tools and methodologies, including indicators and benchmarks, or (b) based on a decision of the Commission, ex officio or following a qualified alert from the scientific panel, it has capabilities or an impact equivalent to those set out in point (a) having regard to the criteria set out in Annex XIII of the Act.63
GPAI models with systemic risk classified as such under (a) above must be notified to the Commission.64 If the Commission becomes aware of a GPAI model with systemic risk of which it has not been notified, it may designate it as such.65 It is also possible for the provider of a GPAI model that meets this condition to argue that, exceptionally, such a model does not, in fact, due to its specific characteristics, present systemic risks and, therefore, should not be classified as such. However, the Commission may reject these arguments.66 A designation decision may be changed upon a reasoned request by a provider and if accepted by the Commission.67 A list of GPAI models with systemic risk must be published and updated by the Commission (without prejudice
These provisions represent a change from the Councilâs position, where GPAI systems to be used as high-risk AI systems or components of such systems would have had to comply with the requirements for high-risk AI systems.69 The final version of the Act has introduced the notion of GPAI models with systemic risk, and GPAI models must comply with the new Chapter V and all other relevant obligations within the Act.
5 Regulate Who? Providers v Deployers
When regulating Artificial Intelligence, a central question is who should be subjected to legal requirements, obligations and potential future liability. The developer â or, in the language of the AI Act, âproviderâ â is the entity that will have the knowledge and practical control of the training sets, algorithms, etc. It may be argued that developers should carry legal responsibility, as the high-risk obligations mostly arise at the development stage. Such an argument is strengthened when the developer significantly profits from licensing their product or service. However, a counter-argument about GPAI notes that provider responsibility (and potential subsequent liability) may not include responsibility/liability for unforeseeable uses/risks. Nevertheless, it may not be practical (or even fair) to impose certain obligations on deployers because it may be impossible for them to fix or even audit issues of data quality, etc without access to the upstream source code, training datasets etc (which are often secret/proprietary, as with ChatGPT 4). Previous versions of the Act were also criticised for not reflecting the actors in the AI ecosystem accurately;70 the final version has clarified the taxonomy of such entities â including, for instance, replacing the previously-employed term of âuserâ with âdeployerâ â and bestows responsibilities and obligations on various actors.
providers placing on the market or putting into service AI systems or placing on the market general-purpose AI models in the Union, irrespective of whether those providers are established or located within the Union or in a third country;
deployers of AI systems that have their place of establishment or are located within the Union;
providers and deployers of AI systems that have their place of establishment or are located in a third country, where the output produced by the system is used in the Union;
importers and distributors of AI systems;
product manufacturers placing on the market or putting into service an AI system together with their product and under their name or trademark;
authorised representatives of providers, which are not established in the Union;
affected persons that are located in the Union.71
The entities listed above are defined as follows:
A âproviderâ is âa natural or legal person, public authority, agency or other body that develops an AI system or a general-purpose AI model or that has an AI system or a general-purpose AI model developed and places it on the market or puts the AI system into service under its name or trademark, whether for payment or free of chargeâ.72 A âdeployerâ is âa natural or legal person, public authority, agency or other body using an AI system under its authority, except where the AI system is used during a personal, non-professional activityâ.73 An âauthorised representativeâ is âa natural or legal person located or established in the Union who has received and accepted a written mandate from a provider of an AI system or a general-purpose AI model to respectively, perform and carry out on its behalf the obligations and procedures established by this Regulationâ.74 An âimporterâ is âany natural or legal person located or established in the Union that places on the market an AI system that bears the name or trademark of a natural or legal person established in a third countryâ.75 A âdistributorâ is âany natural or legal person in the supply chain, other
6 Regulate How? (Requirements and Obligations)
This section first describes the requirements for high-risk AI systems and the obligations of providers and deployers of such systems as laid out in Chapter III, Sections 2 and 3 and Chapter IX, Sections 1 and 2 of the AI Act. Then, it moves on to the obligations of providers of GPAI s listed in Chapter V, Sections 2 and 3. Additional specific transparency obligations, incumbent on providers and deployers and set out in Chapter IV of the Act, are discussed in a subsequent section below.
6.1 Requirements and Obligations for High-Risk AI Systems
High-risk AI systems require establishing and implementing a ârisk management systemâ, understood as a âcontinuous iterative process planned and run throughout the lifecycleâ of such systems.78 Where these systems make use of techniques involving the training of AI models with data, the âtraining, validation and testing data setsâ must be ârelevant, sufficiently representative, and to the best extent possible, free of errors and complete given the intended purposeâ and must have âthe appropriate statistical properties, including, where applicable, as regards the persons or groups of persons about whom the high-risk AI system is intended to be usedâ.79
In this regard, the AI Act provides a specific exemption from Article 9 of GDPR, which lays down an exhaustive list of legal bases for processing special personal data categories. Article 10(5) of the AI Act thus permits the processing of such data â âsubject to appropriate safeguards for the fundamental rights and freedoms of natural personsâ and âto the extent that it is strictly necessaryâ- to ensure bias detection and correctionâ.80
High-risk AI systems must be accompanied by technical documentation containing at least âthe elements set out in Annex IVâ, which prescribes âa general description of the AI systemâ and a detailed description of the elements of
High-risk AI systems must also âtechnically allow for the automatic recording of events (âlogsâ) over the systemâs lifetimeâ81 and be âdesigned and developed in such a way ⦠that natural persons can effectively oversee them during the period in which they are in useâ.82 The raison dâêtre of this provision on human oversight is stated in the provision itself: âHuman oversight shall aim to prevent or minimise the risks to health, safety or fundamental rights that may emerge when a high-risk AI system is used in accordance with its intended purpose or under the conditions of reasonably foreseeable misuseâ.83 It is also reflected in the obligation incumbent on deployers to âassign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary supportâ84 and in the additional specific requirements relating to RBI systems, where no action/decision is to be âtaken by the deployer based on the identification resulting from the system unless this has been separately verified and confirmed by at least two natural personsâ.85
Insofar as obligations are concerned, providers must inter alia ensure compliance of their systems with the requirements described above, indicate on the system or its packaging/accompanying documentation their name, registered trade name/mark and address at which they may be contacted, have a quality management system in place that ensures compliance with the AI Act and keep logs automatically generated by their systems when under their control.86 They are bound by rules related to documentation keeping,87 must
Deployers must take âappropriate technical and organisational measuresâ to ensure they use high-risk AI systems âper the instructions for useâ.97 As alluded to above, they must assign human oversight to natural persons98 and monitor the operation of high-risk AI systems based on the instructions for use.99 When they have reasons to consider that the use of the instructions may result in the system presenting a risk, they must, without undue delay, inform the provider or distributor and the relevant MSA and suspend the use of the system. They must also inform the provider first, the importer or distributor, and the relevant
Similarly to providers, deployers must keep logs automatically generated by the relevant AI system âto the extent that such logs are under their controlâ.100 Deployers who are employers must âinform workersâ representatives and affected workers that they will be subject to using a high-risk AI systemâ.101 Deployers of high-risk AI systems for post-remote biometric identification must generally request authorisation from a judicial or administrative authority,102 and deployers of high-risk systems referred to in Annex III of the Act must inform ânatural persons that they are subject to the use of the high-risk AI systemâ.103 Finally, similarly to providers, deployers must cooperate with the relevant competent authorities.104
6.2 Specific Obligations of Providers of General-Purpose AI Models
Providers of GPAI models must draw up and keep updated relevant technical documentation, including regarding such modelsâ training and testing process and the results of their evaluation;105 draw up and make available relevant information and documentation to providers of AI systems who intend to integrate the GPAI model into their systems;106 put in place a policy to comply with Union law on copyright and related rights;107 and draw up and make publicly available a sufficiently detailed summary about the content used for training the AI model.108 Providers of free and open-sourced licence GPAI models are exempt from the first two above-listed obligations; however, providers of GPAI models with systemic risks are not.109
Moreover, providers of models with systemic risk are bound by additional obligations to those described above and must also: âperform model evaluation ⦠including conducting and documenting adversarial testing of the model to identify and mitigate systemic risksâ;110 âassess and mitigate possible
The Act calls for the drawing up of codes of practice at the Union level relating to GPAI models and corresponding obligations;114 providers of such models may rely on these codes to demonstrate compliance until harmonised standards are published.115 Finally, it is also incumbent on providers of GPAI models established in third countries to appoint an authorised representative in the EU.116
6.3 Fundamental Rights Impact Assessment
The first use of âimpact assessmentsâ concerning data-driven technologies was in the field of privacy and data protection. The terminology âPrivacy Impact Assessmentâ117 was later replaced by âData Protection Impact Assessmentâ in the GDPR.118 The terminology and the intended scope of such impact assessments have been extended to a âfundamental rights impact assessmentâ119 to consider broader fundamental rights risks of AI systems. The impact assessment criteria would derive from the EU Charter of Fundamental Rights120 and the European Convention on Human Rights.121 An âimpact assessmentâ
The AI Act has previously been criticised for lacking a general fundamental rights risk assessment for all AI systems falling within its scope, not just high-risk ones.122 Moreover, Edwards has criticised the restricted scope of such ex-ante assessments, noting, in particular, the lack of systematic concern for impacts on groups, particularly algorithmically constituted groups.123 What should be legally mandated is a comprehensive ex-ante impact assessment, as well as regular ex-post audits for algorithmic systems released into society, to identify and mitigate the impact and all potential risks or harms that may result from the deployment of such systems. Moreover, such impact assessment should go beyond the traditional individual human rights approach and consider potential harms to groups and communities.
Today, however, the Act still only requires a fundamental rights impact assessment for high-risk AI systems in specific circumstances. It is only deployers of high-risk systems referred to in Annex III (except those relating to critical infrastructure) that are âbodies governed by public lawâ or âprivate entities providing public servicesâ, and deployers of high-risk systems intended to be used to evaluate the creditworthiness of natural persons or establish their credit score (except AI systems used to detect financial fraud)124 and for risk assessment and pricing concerning natural persons in the case of life and health insurance125 that must assess the impact on fundamental rights that the use of such systems may produce.126
High-risk AI systems or GPAI models that follow harmonised standards enjoy a presumption of conformity with the requirements for such systems and models set out in Chapter III, Section 2 or Chapter V, Sections 2 and 3 of the Act.127
The Commission can mandate the European Committees for Standardisation (âCENâ) and Electrotechnical Standardisation (âCENELECâ) to develop harmonised standards.128 The Commission may also adopt implementing acts establishing common specifications for the relevant requirements where it has requested a standardisation organisation to draft harmonised standards. However, the request has not been accepted, the standards have not been delivered within the deadline, fundamental rights concerns have not been addressed, or the request has not been complied with. No requirement reference has been published in the EUâs Official Journal.129 Systems and models following such specifications will also enjoy the same presumption of conformity described above to the extent that the common specifications cover the relevant requirements or obligations.130
Harmonised standards often function as a necessary point of reference for compliance. It is likely, however, that as the AI Act remains vague on how to implement the essential requirements for high-risk AI systems, the responsibility for determining the specifics of these requirements will be delegated to CEN and CENELEC. That standardisation will thus be where the real rule-making occurs. This practice of effectively privately outsourcing complex negotiations has been controversial.131 For example, Ansari and Mardais argue that the lack of representation from human rights experts or civil society organisations in these bodies raises concerns about their ability to protect fundamental rights. They state that technical standards related to data governance, transparency, security, and human oversight will directly impact fundamental rights and that
In addition to ensuring adherence to harmonised standards or common specifications, providers of high-risk AI systems must undertake a conformity assessment.133 The EDPS and EDPB have called for such assessment to be generally undertaken by third parties. However, the final draft of the Act permits a self-assessment in many cases.
Providers of high-risk AI systems falling within the Annex III category of biometrics that have applied relevant standards or specifications may opt for a self-assessment or an assessment involving a notified body. Such providers must perforce opt for the latter procedure if harmonised standards do not exist if they have not applied or have only applied part of existing standards, or if they have not applied existing specifications.134 Notified bodies are independent technical organisations to be established under the national law of a Member State135 and tasked with verifying the conformity of high-risk AI systems.136 Providers can generally opt for a notified body of their choice unless the high-risk system is intended to be put into service by law enforcement, immigration or asylum authorities or by Union institutions, bodies, offices or agencies. In this latter case, as applicable, the MSA referred to in Article 74(8) or (9) must act as a notified body.137 Providers of high-risk AI systems falling within the remaining Annex III categories shall conduct a self-assessment without needing third-party involvement,138 and providers of high-risk AI systems covered by the Union legislation in Section A of Annex I must follow the relevant conformity procedure under those legal acts.139
The Act also contains provisions for a âCE marking of conformityâ140 (or CE certification), an EU declaration of conformity. Digital CE markings shall be used for digitally provided high-risk AI systems.141
8.1 Specific Transparency Obligations
In addition to the obligations under Chapters III, V, and IX described above, the AI Act confers two specific transparency obligations on providers and four on deployers in Chapter IV.
Thus, providers must ensure that AI systems intended to interact with natural persons are designed and developed so that natural persons are informed that they are interacting with an AI system (unless this is obvious from the point of view of a natural person). This obligation does not apply to systems authorised by law to detect, prevent, investigate and prosecute criminal offences, subject to appropriate safeguards for the rights and freedoms of third parties, unless such systems are available for the public to report a criminal offence.142
Additionally, providers of AI systems, including GPAI systems, generating synthetic audio, image, video or text content must ensure that the system outputs are marked in a machine-readable format and detectable as
Deployers of biometric categorisation or emotion recognition systems must inform the natural persons exposed to such systems of their operation and process any personal data by the GDPR, even though it remains controversial under data protection law whether facial recognition technology, which processes personal data only transiently falls within the scope of the definition of âpersonal dataâ and thus, of the GDPR.144 This obligation also does not apply to such systems permitted by law to detect, prevent, and investigate criminal offences, subject to appropriate safeguards for the rights and freedoms of third parties.145
Deployers of an AI system that generates or manipulates image, audio or video content constituting a deep fake must disclose that the content has been artificially generated or manipulated. Again, this obligation does not apply where the use is authorised by law to detect, prevent, investigate and prosecute criminal offences or where the content is part of a creative, satirical, artistic or fictional work or programme subject to appropriate safeguards for the rights and freedoms of third parties.146 In such cases, there could be two persons to protect: the one depicted in the deep fake (if it concerns a person) and the audience/recipient of the deep fake. This provision seems focused on the recipient: misplaced beliefs of authenticity may present dangers, particularly fake news and disinformation in media law.
As such, Chapter IV of the Act includes the following final obligation: Deployers of an AI system that generates or manipulates image, audio or video content published to inform the public on matters of public interest shall disclose that the said content has been artificially generated or manipulated. This is also not applicable where use is authorised by law to detect, prevent, investigate or prosecute criminal offences, where the AI-generated content has undergone a process of human review or editorial control and where a natural or a legal person holds editorial responsibility for the publication of the content.147
Further specific transparency obligations for providers of GPAI models are also found within the provisions on the obligations of such providers,148 already referred to above. This is because GPAI models may form the basis for a range of downstream systems, often provided by downstream providers, defined in the Act as a provider of an AI system, including a general-purpose AI system, which integrates an AI model, regardless of whether the AI model is provided by themselves and vertically integrated or provided by another entity based on contractual relations.149 This necessitates âa good understanding of the models and their capabilities to enable the integration of such models into their products and to fulfil their obligations under [the AI Act] or other regulationsâ.150
8.2 Market Monitoring and Market Surveillance
Regarding post-marketing controls, the Act provides that each Member State must establish or designate at least one notifying authority and at least one MSA for its purposes as national competent authorities.151 These authorities are to
MSAs should have all enforcement powers under the AI Act and Regulation (EU) 2019/1020.153 The EDPS will be the competent authority for AI systems put into service or used by Union institutions, agencies, offices and bodies.154 MSAs will receive information through a chain of notification obligations. For instance, as discussed earlier, deployers who have reason to consider that the use per the instructions of a high-risk AI system may result in that system presenting a risk to the health or safety or fundamental rights of persons must inform (amongst others) the relevant MSA and both providers and deployers of high-risk AI systems are under an obligation to inform MSA s about any serious incidents.155
While it is envisaged that investigatory powers will primarily be in the hands of the national authorities, the AI Act also establishes a European Artificial Intelligence Board (AI Board)156 with specified tasks centrally focused on facilitating the âconsistent and effective applicationâ of the Act.157 During the legislative procedure, the co-rapporteurs of the Act in the EP proposed replacing the AI Board with an EU AI Office, envisaged as an independent body with its legal personality, funding, and staff.158 The AI Office has since been established by a Commission Decision,159 with the mission âto develop Union expertise and capabilities in the field of AI and to contribute to the implementation of Union law on AIâ.160 At the same time, the AI Board has been retained as an entity to be established by the AI Act.
In order to facilitate the work of the Commission and the Member States in the Artificial Intelligence field as well as to increase the transparency towards the public, providers of high-risk AI systems (other than those related to products falling within the scope of relevant existing Union harmonisation legislation) should be required to register themselves and information about their high-risk AI system in a EU database, to be established and managed by the Commission.162
The obligation of registration in the EU database has also been extended to deployers of high-risk AI systems listed in Annex III (except systems relating to critical infrastructure) that are public authorities, agencies or bodies.163
9 Regulatory Sandboxes
A regulatory sandbox allows innovators to explore and experiment with new and innovative products, services or businesses under a regulatorâs supervision. It potentially benefits all stakeholders: it allows regulators to understand the technology better, provides innovators with incentives to test their innovations in a controlled environment, and fosters consumer choice in the long run. However, regulatory sandboxes also come with a risk of being misused or abused. To succeed, an appropriate legal framework is needed.164
An âAI regulatory sandboxâ is defined in the Act as âa controlled framework set up by a competent authority which offers providers or prospective providers of AI systems the possibility to develop, train, validate and test, where appropriate in real-world conditions, an innovative AI system, under a sandbox plan for a limited time under regulatory supervisionâ.165
Member Statesâ national authorities must set up at least one AI regulatory sandbox at the national level and put a framework for governance and
An Open Loop report exploring an earlier version of this provision concluded that it is, in fact, these implementing acts, rather than the provision itself, that could create the necessary conditions for successful regulatory sandboxes. This report warned, however, that Member States should avoid creating conditions that are too favourable for participants, as this could distort the level playing field for AI development in Europe.169 National competent authorities that have established AI regulatory sandboxes will cooperate and coordinate their activities within the framework of the AI Board and the AI Office.170
SMEs, including start-ups, will be provided priority access to the AI regulatory sandboxes and, where appropriate, a dedicated channel for communication to provide advice and respond to queries about the implementation of the Regulation, particularly in AI regulatory sandboxes.171 It is envisaged that these sandboxes will allow proportionate application of the rules to the SMEs, insofar as permitted under existing legislation, and thus provide a space for experimentation under the new rules and the existing legal framework.172
The Act also regulates the processing of personal data in AI regulatory sandboxes for âdeveloping, training and testing certain AI systemsâ, which it
10 Room for National Initiatives in the EU?
In EU law, a âfull harmonisation measureâ means that it allows no scope for stricter regulation at the national level. It is unclear whether the AI Act is intended to be such a measure. A discussion of the complex relationship between EU law and national-level laws174 and the room left for national-level regulation, in particular, if not âpre-emptedâ by the AI Act, is too complex to be tackled in the space of this book chapter. Suffice it to say that while the Act is primarily concerned with regulating âhigh-riskâ AI systems, non-high-risk AI systems are barely regulated but still fall within its scope, thus arguably precluding national legislation on such systems.175 Having said this, Article 26 concerning obligations of deployers of high-risk AI systems provides that the âobligations [relating to the use of high-risk AI systems and in particular the assignment of human oversight] are without prejudice to other deployer obligations under Union or national law and to the deployerâs freedom to organise its resources and activities to implement the human oversight measures indicated by the providerâ176 thus specifically allowing room for national legislation.
11 Conclusion: Regulatory Balance
Regulation is necessary but tricky. It is important to get the regulatory balance right. Too little regulation and important rights and interests may be undermined; too much regulation and innovation, development and investment â and thus, the EUâs competitiveness on the global stage â may be stifled. Excessive regulation also tends to help further entrench the market power of the incumbent big players. When discussing regulating AI, an appropriate legal framework should enable investment and innovation while upholding
Strengths of the AI Act include the risk-based approach, the prohibition of certain AI systems, and the possibility for societal scrutiny via a public database. However, this chapter has also commented on potential weaknesses, such as the role of the standardisation bodies that lack the participation of human rights experts and/or civil society organisations and that are likely to write the real rules against which providers will self-assess.
The AI Act acknowledges the risks to individuals and society posed by AI technologies and seeks to prevent them without stifling innovation. Nevertheless, earlier versions of the Act were criticised for being âgenerally less encouraging of the development of new AI technologiesâ and not going far enough to protect fundamental rights.177 It remains to be seen whether the Act will lead to a balanced model between fundamental rights and technological development or whether Europeâs zealousness to regulate this space will mean that innovative businesses will base themselves elsewhere and choose not to offer their services within the EU. Beyond the internal market, the potential global implications of the EU AI Act should also not be ignored, considering the EUâs capacity to set de facto international standards for AI regulation due to its substantial market size and regulatory influence. In this sense, the EUâs influence may extend beyond hard law to a form of soft extraterritorialityâtermed the âBrussels effectââwhich may encourage non-EU countries and multinational corporations to align their AI practices with EU standards. This and other complex issues will undoubtedly be the subject of several further studies in future.
Bibliography
Ansari M, and V Marda, âOpinion. AI Act â leaving oversight to the techies will not protect rightsâ (EUobserver, 5 May 2023) <https://euobserver.com/opinion/156992#:~:text=In%20May%2C%20the%20European%20Parliament,regulate%20the%20use%20of%20AI.> accessed 8 May 2023.
Bostrom N, Superintelligence: Paths, Dangers, Strategies (OUP 2014).
Earls Davis P A, âFacial Detection and Smart Billboards: Analysing the âIdentifiedâ Criterion of Personal Data in the GDPRâ (2020) 6(3) European Data Protection Law Review 365.
Ebers M, Standardizing AI â The Case of the European Commissionâs Proposal for an Artificial Intelligence Act (August 6, 2021). The Cambridge Handbook of Artificial Intelligence: Global Perspectives on Law and Ethics, Available at SSRN: https://ssrn.com/abstract=3900378 or http://dx.doi.org/10.2139/ssrn.3900378
George D and K Reutimann, âGDPR Bypass by Design? Transient Processing of Data under the GDPRâ (2019) 9 International Data Privacy Law 14.
Höppner T and L Streatfeild, âChatGPT, Bard & Co.: An Introduction to AI for Competition and Regulatory Lawyersâ (2023) 9 Hausfeld Competition Bulletin (1/2023), Article 1, <http://dx.doi.org/10.2139/ssrn.4371681> accessed 20 May 2024.
Johanson P, Online Filter Bubbles (Greenhaven Publishing, 2017).
Pariser E, The filter bubble: How the new personalised web is changing what we read and how we think (Penguin, 2011).
Purtova N, âFrom knowing by name to targeting: the meaning of identification under the GDPRâ (2022) 12(3) International Data Privacy Law 163.
Raposo V L, âEx Machina: Preliminary Critical Assessment Of The European Draft Act On Artificial Intelligenceâ (2022) 30(1) International Journal of Law and Information Technology 88.
Truby J, R D Brown, I A Ibrahim and O C Parellada, âA Sandbox Approach To Regulating High-Risk Artificial Intelligence Applicationsâ (2022) 13(2) European Journal of Risk Regulation 270.
Veale M, and F Z Borgesius, âDemystifying the Draft EU Artificial Intelligence Actâ (2021) 22(4) Computer Law Review International 97.
Weatherill S, âThe Fundamental Question of Minimum or Maximum Harmonisationâ, in S Garben and I Govaere (eds), The Internal Market 2.0 (Hart Publishing 2020) 261â284.
Legislative Train
Proposal for a Regulation laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts COM/2021/206 final https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A52021PC0206
Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts â EU Council General approach â Brussels, 25 November 2022 https://data.consilium.europa.eu/doc/document/ST-14954-2022-INIT/en/pdf
Artificial Intelligence Act: Amendments adopted by the European Parliament on 14 June 2023 on the proposal for a regulation of the European Parliament and of the Council on laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts (COM(2021)0206 â C9-0146/2021 â 2021/0106(COD))
CORRIGENDUM to the position of the European Parliament adopted at first reading on 13 March 2024 with a view to the adoption of Regulation (EU) 2024/ ⦠of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act), 19.4.2024 <https://www.europarl.europa.eu/doceo/document/TA-9-2024-0138-FNL-COR01_EN.pdf>
Legislation
Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market For Digital Services and amending Directive 2000/31/EC (Digital Services Act) OJ L 277/1
Official Publications
European Commission Impact Assessment of the Regulation on Artificial Intelligence, 21 April 2021, SWD(2021) 84 final
EDPB-EDPS Joint Opinion 5/2021 on the proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) 18 June 2021 <https://edps.europa.eu/system/files/2021-06/2021-06-18-edpb-edps_joint_opinion_ai_regulation_en.pdf>
The Council of Europe recently adopted the first-ever international treaty on artificial intelligence, which will not, however, be considered in this chapter. For more information, see: <https://www.coe.int/en/web/portal/-/council-of-europe-adopts-first-international-treaty-on-artificial-intelligence> accessed 20 May 2024.
The Council adopted its common position (âProposed AI Act (Council General Approachâ) on the AI Act on 6 December 2022: Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts â EU Council General approach - Brussels, 25 November 2022 <https://data.consilium.europa.eu/doc/document/ST-14954-2022-INIT/en/pdf> accessed 20 May 2024. See also Artificial Intelligence Act: Council calls for promoting safe AI that respects fundamental rights, Press release 6 December 2022, <https://www.consilium.europa.eu/en/press/press-releases/2022/12/06/artificial-intelligence-act-council-calls-for-promoting-safe-ai-that-respects-fundamental-rights/> accessed 20 May 2024.
Amendments adopted by the EP on 14 June 2023 on the proposal for a regulation of the EP and of the Council on laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts (COM(2021)0206 â C9-0146/2021 â 2021/0106(COD)) <https://www.europarl.europa.eu/doceo/document/TA-9-2023-0236_EN.pdf> accessed 17 May 2024.
The text of the final draft (European Parliament âCorrigendumâ of 16th April 2024) is available here: <https://www.europarl.europa.eu/doceo/document/TA-9-2024-0138-FNL-COR01_EN.pdf> accessed 17 May 2024. The Regulation is expected to be finally adopted before the end of the legislature and also needs to be formally endorsed by the Council.
Recital (1) AI Act.
Recital (3) AI Act.
Article 3(1) AI Act.
Article 3(1) Proposed AI Act (Council General Approach).
EP DRAFT Compromise Amendments 16 May 2023.
Article 3(63) AI Act.
Article 3(66) AI Act.
Innocenzi Genna, âFoundation Models: how they are regulated in the AI ACTâ (radiobruxelleslibera, 5 February 2024) <https://radiobruxelleslibera.com/2024/02/05/foundation-models-how-they-are-regulated-in-the-ai-act/#:~:text=Foundation%20Models%20are%20now%20defined,components%20into%20an%20AI%20system> accessed 20 May 2024.
Recital (99) AI Act.
Emily M. Bender and others, âOn the Dangers of Stochastic Parrots: Can Language Models Be Too Big?â (2021) Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency 610 <https://doi.org/10.1145/3442188.3445922> accessed 20 May 2024.
Explained simply here: Muhammad Saad Uddin, âStochastic Parrots: A Novel Look at Large Language Models and Their Limitationsâ (Towards AI, 20 April 2023) <https://towardsai.net/p/machine-learning/stochastic-parrots-a-novel-look-at-large-language-models-and-their-limitations#:~:text=At%20its%20core%2C%20the%20term,the%20language%20they%20are%20processing> accessed 20 May 2024.
NOYB, âChatGPT provides false information about people, and OpenAI canât correct itâ (noyb, 29 April 2024) <https://noyb.eu/en/chatgpt-provides-false-information-about-people-and-openai-cant-correct-it> accessed 20 May 2024.
As reflected in science fiction in the Terminator series of films or in Isaac Asimovâs I, Robot. See also: Nick Bostrom, Superintelligence: Paths, Dangers, Strategies (OUP 2014); Chris Vallance, âArtificial Intelligence could lead to extinction, experts warnâ (BBC News, 31 May 2023) <https://www.bbc.com/news/uk-65746524> accessed 20 May 2024.
Andrew Murray, âWhen machines become sentient, we will have to consider them an intelligent life formâ (LSE, 10 August 2018) <https://blogs.lse.ac.uk/businessreview/2018/08/10/when-machines-become-sentient-we-will-have-to-consider-them-an-intelligent-life-form/> accessed 20 May 2024.
Article 7(1)(b) AI Act.
Martin Ebers and others, âThe European Commissionâs Proposal For An Artificial Intelligence ActâA Critical Assessment By Members Of The Robotics And AI Law Society (RAILS)â (2021) 4(4) Multidisciplinary Scientific Journal (2021) <https://www.mdpi.com/2571-8800/4/4/43> accessed 5 May 2023.
Mher Hakobyan, Advocacy Advisor on Artificial Intelligence Regulation at Amnesty International, quoted in âEU: European Union must protect human rights in upcoming AI Act voteâ 26 April 2023 <https://www.commondreams.org/newswire/eu-european-union-must-protect-human-rights-in-upcoming-ai-act-vote> accessed 27 April 2023.
Recital (31) AI Act.
Tina van der Linden, âRegulating Artificial Intelligence: Please Apply Existing Regulationâ (Amsterdam Law Forum 2021) <https://www.amsterdamlawforum.org/articles/abstract/432>. Action against addictive design is part of formal proceedings opened by the Commission against TikTok in February 2024: European Commission, âCommission opens formal proceedings against TikTok under the Digital Services Actâ (European Commission, 19 February 2024) <https://ec.europa.eu/commission/presscorner/detail/en/ip_24_926> accessed 15 May 2024.
Article 34(2) Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market For Digital Services and amending Directive 2000/31/EC (Digital Services Act) OJ L 277/1
Eli Pariser, The filter bubble: How the new personalized web is changing what we read and how we think (Penguin, 2011); Paula Johanson, Online Filter Bubbles (Greenhaven Publishing, 2017).
Fanny Hidvegi, Daniel Leafier and Estelle Massey, âThe EU should regulate AI on the basis of rights, not risks (Access Now, 17 February 2021) <https://www.accessnow.org/eu-regulation-ai-risk-based-approach/> accessed 22 May 2023.
Article 16 EU Charter; Peter Craddock Op-Ed: Who dares question the primacy of data protection? 7 May 2024 <https://www.linkedin.com/pulse/op-ed-who-dares-question-primacy-data-protection-peter-craddock-kfkxe/?trackingId=xYW%2BEJ2RRjyEbnNgn8Lfsw%3D%3D> accessed 17 May 2024.
See for example UNEP, âHow Artificial Intelligence is helping tackle environmental challengesâ (UNEP, 7 November 2022) <https://www.unep.org/news-and-stories/story/how-artificial-intelligence-helping-tackle-environmental-challenges> accessed 29 May 2023.
See for example Annette Ekin, âAI can help us fight climate change. But it has an energy problem, tooâ (Horizon, 12 September 2019) <https://ec.europa.eu/research-and-innovation/en/horizon-magazine/ai-can-help-us-fight-climate-change-it-has-energy-problem-too> accessed 29 May 2023.
European Union Agency for Fundamental Rights, Getting the Future Right: Artificial Intelligence and Fundamental Rights, 2020 <https://fra.europa.eu/sites/default/files/fra_uploads/fra-2020-artificial-intelligence_en.pdf> accessed 29 May 2023.
Cf. Thomas Höppner and Luke Streatfeild, âChatGPT, Bard & Co.: An Introduction to AI for Competition and Regulatory Lawyersâ (2023) 9 Hausfeld Competition Bulletin (1/2023), Article 1, Available at SSRN <https://ssrn.com/abstract=4371681> or <http://dx.doi.org/10.2139/ssrn.4371681> Accessed 5 June 2023.
Lilian Edwards, Expert Opinion: Regulating AI in Europe â Four problems and four solutions, 31 March 2022 Ada Lovelace Institute <https://www.adalovelaceinstitute.org/wp-content/uploads/2022/03/Expert-opinion-Lilian-Edwards-Regulating-AI-in-Europe.pdf> accessed 31 May 2023.
Article 53(2) AI Act.
Article 5 AI Act.
Article 5(1)(a) AI Act.
Article 5(1)(b) AI Act.
Article 5(1)(c) AI Act.
Article 5(1)(d) AI Act.
Ibid.
Article 5(1)(e) AI Act.
Article 5(1)(f) AI Act.
Article 5(1)(g) AI Act.
Article 5(1)(h)(I) â (III) AI Act.
Article 5(2) AI Act.
Article 5(3) AI Act.
Article 5(4) AI Act.
Article 5(6) AI Act.
See for instance, Kashmir Hill, âA Face Search Engine Anyone Can Use is Alarmingly Accurateâ The New York Times (New York, 26 May 2022) <https://www.nytimes.com/2022/05/26/technology/pimeyes-facial-recognition-search.html> accessed 20 May 2024.
See for instance Amnesty International âEurope: Proposed legislation too weak to protect us from dangerous AI systemsâ (21 April 2021) <https://www.amnesty.org/en/latest/press-release/2021/04/eu-legislation-to-ban-dangerous-ai-may-not-stop-law-enforcement-abuse/> accessed 20 May 2024 and SHERPA, âFeedback to the European Commission on its Proposal for a legal act of the European Parliament and the Council laying down requirements for Artificial Intelligenceâ (6 August 2020) <https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/12527-Artificial-intelligence-ethical-and-legal-requirements/F2665582_en> accessed 20 May 2024. All documentation submitted to the Commission pursuant to the public consultation period are available at <https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/12527-Artificial-intelligence-ethical-and-legal-requirements/feedback_en?p_id=24212003> accessed 20 May 2024.
EDPB-EDPS Joint Opinion 5/2021 on the proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) 18 June 2021.
Michael Veale and Frederik Zuiderveen Borgesius, âDemystifying the Draft EU Artificial Intelligence Actâ (2021) 22(4) Computer Law Review International 101.
Article 6(1) AI Act.
Article 6(2) AI Act.
Article 6(3) AI Act.
Article 6(3) Proposed AI Act (Council General Approach).
Article 6(3) AI Act.
Article 7 AI Act.
Article 7(3) Proposed AI Act (Council General Approach).
Article 7(3) AI Act.
Article 50 AI Act.
Article 50(1) AI Act.
Defined in Art. 3(65) AI Act.
Article 51(1) AI Act.
Article 52 AI Act.
Ibid.
Article 52(3) AI Act.
Article 52(5) AI Act.
Article 52(6) AI Act.
Article 4b Proposed AI Act (Council General Approach).
See for instance Norberto Nuno Gomes de Andrade, Laura Galindo and Antonella Zarra, âArtificial Intelligence Act: A Policy Prototyping Experiment. Revisiting the Taxonomy of AI Actorsâ (Open Loop, April 2023) <https://openloop.org/programs/open-loop-eu-ai-act-program/?utm_source=substack&utm_medium=email> accessed 8 May 2023.
Article 2(1) AI Act.
Article 3(2) AI Act.
Article 3(4) AI Act.
Article 3(5) AI Act.
Article 3(6) AI Act.
Article 3(7) AI Act.
Article 3(8) AI Act.
Article 9(1) and (2) AI Act.
Article 10(3) AI Act.
Article 10(5) AI Act.
Article 12(1) AI Act.
Article 14 AI Act.
Article 14(2) AI Act.
Article 26(2) AI Act.
Article 14(5) AI Act.
Articles 16(1)(a)-(c) and (e), 17 and 19 AI Act.
Article 18 AI Act.
Article 16(f) AI Act.
Article 16(h) AI Act.
Article 20(1) AI Act.
Article 79(1) AI Act.
Article 20(2) AI Act.
Article 21 AI Act.
Article 22 AI Act.
Article 72 AI Act.
Article 73 AI Act.
Article 26(1) AI Act.
Article 26(2) AI Act.
Article 26(5) AI Act.
Article 26(6) AI Act.
Article 26(7) AI Act.
Article 26(10) AI Act.
Article 26(11) AI Act
Article 26(12) AI Act.
Article 53(1)(a) AI Act.
Article 53(1)(b) AI Act.
Article 53(1)(c) AI Act.
Article 53(1)(d) AI Act.
Article 53(2) AI Act.
Article 55(1)(a) AI Act.
Article 55(1)(b) AI Act.
Article 55(1)(c) AI Act.
Article 55(1)(d) AI Act.
Article 56 AI Act.
Article 55(2) AI Act.
Article 54 AI Act.
See for example the work of then Information and Privacy Commissioner/Ontario Anne Cavoukian, Privacy Impact Assessment Guidelines for the Ontario Personal Health Information Protection Act (October 2005) <https://www.ipc.on.ca/wp-content/uploads/resources/phipa_pia-e.pdf> accessed 31 May 2023; see also Roger Clarke, âPrivacy impact assessment: Its origins and developmentâ (2009) 25(2) Computer Law & Security Review 123.
Article 35 GDPR.
Heleen Janssen, Michelle Seng Ah Lee, Jatinder Singh, âPractical fundamental rights impact assessmentsâ (2022) 30 International Journal of Law and Information Technology 200.
Charter of Fundamental Rights and Freedoms of the European Union [2012] OJ C 326/02.
Council of Europe Convention for the Protection of Human Rights and Fundamental Freedoms [1953]. On the relationship between these instruments, see Article 52(3) EU Charter: âIn so far as this Charter contains rights which correspond to rights guaranteed by the Convention for the Protection of Human Rights and Fundamental Freedoms, the meaning and scope of those rights shall be the same as those laid down by the said Convention. This provision shall not prevent Union law providing more extensive protectionâ.
Edwards (n 32).
Ibid.
Annex III 5(b).
Annex III 5(c).
Article 27 AI Act.
Article 40 AI Act.
Regulation (EU) No 1025/2012 of the European Parliament and of the Council of 25 October 2012 on European standardisation, amending Council Directives 89/686/EEC and 93/15/EEC and Directives 94/9/EC, 94/25/EC, 95/16/EC, 97/23/EC, 98/34/EC, 2004/22/EC, 2007/23/EC, 2009/23/EC and 2009/105/EC of the European Parliament and of the Council and repealing Council Decision 87/95/EEC and Decision No 1673/2006/EC of the European Parliament and of the Council OJ L 316/12. Amended by Directive (EU) 2015/1535 of the European Parliament and of the Council of 9 September 2015 laying down a procedure for the provision of information in the field of technical regulations and of rules on Information Society services (codification) OJ L 241/1.
Article 41(1) AI Act.
Article 41(3) AI Act.
Veale and Borgesius (n 51) 105â106.
Mehwish Ansari and Vidushi Marda, âOpinion. AI Act â leaving oversight to the techies will not protect rightsâ (EU Observer, 5 May 2023) <https://euobserver.com/opinion/156992#:~:text=In%20May%2C%20the%20European%20Parliament,regulate%20the%20use%20of%20AI.> accessed 8 May 2023.
Article 43 AI Act.
Article 43(1) AI Act.
Article 31(1) AI Act.
Article 34 AI Act.
Article 43(1) AI Act.
Article 43(2) AI Act.
Article 43(3) AI Act.
Article 48 AI Act.
Article 48(2) AI Act.
Article 50(1) AI Act.
Article 50(2) AI Act.
Peter Alexander Earls Davis, âFacial Detection and Smart Billboards: Analysing the âIdentifiedâ Criterion of Personal Data in the GDPRâ (2020) 6(3) European Data Protection Law Review 365; Damian George and Kento Reutimann, âGDPR Bypass by Design? Transient Processing of Data under the GDPRâ (2019) 9 International Data Privacy Law 14; Nadezhda Purtova, âFrom knowing by name to targeting: the meaning of identification under the GDPRâ (2022) 12(3) International Data Privacy Law 163.
Article 50(3) AI Act.
Article 50(4) AI Act.
Article 50(4) AI Act.
Article 53(1)(b) AI Act.
Article 4(68) AI Act.
Recital (101).
Article 3(48) and Article 70 (1) AI Act.
Article 70 AI Act.
Recital (156) AI Act. This is also reflected in the definition of an MSA in Art. 3(26) AI Act.
Article 70(9) AI Act.
Articles 26(5) and 73 AI Act.
Article 65 AI Act.
Article 66 AI Act.
Title VI âGovernanceâ, Chapter 1 âEuropean Artificial Intelligence Officeâ, Articles 56â58, Text adopted by the leading parliamentary Committees of the European Parliament on 11 May 2023 <https://www.europarl.europa.eu/resources/library/media/20230516RES90302/20230516RES90302.pdf> accessed 10 June 2023.
Commission Decision of 24.1.2024 establishing the European Artificial Intelligence Office C(2024) 390.
Recital (148) AI Act.
Article 71 AI Act.
Recital (131) AI Act.
Article 49(3) AI Act.
Tambiama Madiega with Anne Louise Van De Pol, European Parliament Briefing Artificial Intelligence Act and regulatory sandboxes, June 2022 <https://www.europarl.europa.eu/RegData/etudes/BRIE/2022/733544/EPRS_BRI(2022)733544_EN.pdf> accessed 30 May 2023.
Article 3(55) AI Act.
Article 57(6) AI Act.
Article 57(12) AI Act.
Article 58 AI Act.
Norberto De Andrade, Laura Galindo and Antonella Parra, âArtificial Intelligence Act: A Policy Prototyping Experiment. EU AI Regulatory Sandboxesâ (Open Loop, April 2023) <https://openloop.org/programs/open-loop-eu-ai-act-program/?utm_source=substack&utm_medium=email> accessed 8 May 2023.
Article 57(15) AI Act.
Article 62(1)(a) and (c) AI Act.
European Commission Impact Assessment of the Regulation on Artificial Intelligence, 21 April 2021, SWD(2021) 84 final, p.71.
Article 59 AI Act.
Discussed in: Stephen Weatherill, The Fundamental Question of Minimum or Maximum Harmonisation, in S Garben and I Govaere (eds), The Internal Market 2.0, Oxford: Hart Publishing, 2020, 261â284.
Veale and Borgesius (n 51) 110.
Article 26(3) AI Act.
See Edwards (n 32) and Veale and Borgesius (n 51).