“Serious concerns” on the draft EU AI regulation

On 18 June 2021, the European Data Protection Board (EDPB) and the European Data Protection Supervisor (EDPS) published their joint opinion (the Opinion) on the European Commission's (EC) proposed artificial intelligence (AI) Regulation (the Proposal).

Background

By way of quick background, the EDPB is responsible for ensuring a harmonised application of the EU's privacy regulations and rules. The EDPS oversees EU institutions' own compliance with data protection law and also provides guidance to the EC. The EC announced the Proposal on 21 April 2021, which aims to implement harmonised rules on AI systems and integrate them into existing EU legislative frameworks.

Key Points from the Opinion

Whilst the EDPB and EDPS welcomed the fact that the EC is addressing the use of AI within the EU and stressed that the Proposal has important data protection implications, they call for the Proposal to go further. Some of the key points from the Opinion are summarised below:

• Scope of the Proposal: The EDPB and EDPS have welcomed that the Proposal extends to the use of AI systems by EU institutions, bodies or agencies noting the impact that these institutions' use of AI might have a significant impact of the fundamental rights of individuals.

• Exclusion of international law enforcement cooperation: This is an exclusion from the scope set out in Article 2(4) of the Proposal. The EDPB and EDPS cite that this exclusion creates a significant risk of circumvention, for example by third countries or international organisations operating high-risk applications.

• Risk-based approach: The Opinion welcomes the Proposal's risk based approach but notes that societal/group risks posed by AI systems should be assessed and mitigated. In addition, the EDPB and EDPS believe that the concept of "risk to fundamental rights" should be aligned with the GDPR where personal data is involved.

• Data Protection Legislation: The EDPB and EDPS believe that the Proposal should be clarified to state that existing EU data protection legislation should apply to any processing of personal data that fall within the scope of the Proposal, including adding a recital which states that the Proposal does not seek to affect the application of existing EU laws governing the processing of personal data.

• Risk-assessments: The Proposal requires providers, rather than users, of an AI system to perform a risk assessment of the relevant system, but the Opinion points out that in most cases the users of the AI system will be the data controller rather than the providers. In addition, the EDPB and EDPS believe that it will not always be possible for providers to assess all uses for the AI system. The EDPB and EDPS suggest that the Proposal be updated to state providers should do the initial risk assessment of the AI system, considering the use cases, and that the user of the AI system should perform a data protection impact assessment, as required under the GDPR, factoring in technical characteristics, the use case and the specific context in which the AI system will operate.

• Prohibited uses of AI: The EDPB and EDPS think "intrusive forms of AI" should be prohibited and suggest the Proposal should go further. They call for a ban on AI use for "social scoring" (as foreseen in Art 5(1) of the Proposal) and "automated recognition of human features in publicly accessible spaces", regardless of context. They consider the approach taken by the Proposal on biometric identification, which provides a list of exceptional cases where 'real time' remote biometric identification in publicly accessible spaces for law enforcement, as "flawed".

• More autonomy for the EAIB: The Opinion states that the European Artificial Intelligence Board (EAIB), which the Proposal establishes, should be given more autonomy to ensure the consistent application of the regulation across the single market as there will be input from the EC. The EDPB and EDPS also believe that the EAIB needs to have its legal status clarified.

• Data Protection Authorities: The Opinion states that data protection authorities should be designated as the national supervisory authorities pursuant to Art 59 of the Proposal, and this would ensure a more harmonized regulatory approach, and contribute to the consistent interpretation of data processing provisions and avoid contradictions in its enforcement among Member States.

• CE markings: The EDPB and EDPS believe that the certification system outlined in the Proposal is missing a clear link to EU data protection law as well as to other EU and Member States' law applicable to each 'area' of any high-risk AI systems. They suggest the Proposal does not take into account the principles of data minimization and data protection by design as one of the aspects consider before obtaining a "CE" marking. Therefore, the Opinion recommends amending the Proposal to clarify the relationship between certificates issued under law and data protection certifications, seals and marks, citing that Data Protection Authorities should be involved in the preparation and establishment of harmonized standards and common specifications.

Next steps

Businesses should continue to monitor the development of the EU’s proposal on Artificial Intelligence regulation – the Opinion expresses some key concerns that the Proposal does not address or that it considers are not extensive enough. The conclusion here is that the Proposal may be reviewed and become more restrictive on both providers of AI systems and also the users.