ICO's fourth call for evidence on generative AI

The focus of this fourth consultation is on how organisations developing, training and fine-tuning generative AI models can enable individuals to exercise their data protection rights.  

As before, the ICO is seeking views from various people, ranging from developers and users of generative AI to legal advisors and consultants working in this area. The results of this and previous consultations will be used to shape the ICO's policy position on generative AI. This call for evidence is open until 10 June 2024 and can be responded to via this link.  

In this blog post, we explore the ICO's analysis and the input it is seeking.

Individual's rights  Under data protection law, individuals have the following rights (restrictions and exemptions can sometimes apply) over their personal data when it is processed:

• the right to be informed about how their data is being used;
• the right to access a copy of their personal data;
• the right to have information about them rectified, or deleted;
• the right to object to processing of their data;
• the right to portability of their data;
• the right to not be subject to solely automated decisions; and
• the right to restrict or stop the use of their information.

Across the AI lifecycle, organisations must have processes in place to enable and record the exercise of these rights.  
 

Individual rights in AI development

Right to be informed  

The right to be informed is a prerequisite for exercising other rights under data protection law as individuals can only exercise them if they know if their information is being processed. The ICO stipulated that when generative AI developers collect personal data directly from individuals, for example to train or fine-tune a model, they must provide individuals with clear information about how this data is used and how they can exercise their rights according to Article 13 UK GDPR. Where the data is supplied by a third party, this obligation then lies with the third party.  However, where developers collect personal data from other sources, for example through web scraping, they must still provide this information as set out in Article 14 UK GDPR. There are exceptions to this, as sometimes it is impossible or would require disproportionate effort to fulfil this obligation. Nonetheless, developers are still expected to take appropriate measures to protect individuals' rights and freedoms.   

For example, developers must:

• make privacy information publicly available;
• publish information on the sources, types and categories of personal data used to develop the model (avoid using vague statements about data sources such as "publicly accessible information");
• publish explanations of the purposes for which personal data is being processed and the lawful basis for the processing; and
• provide mechanisms for individuals to exercise their rights. 
   

Right of access  

The ICO noted that it expects developers to have clear and easily accessible methods to help them respond to requests by individuals to access a copy of their data, regardless of the part of the AI lifecycle the data relates to. If developers claim they cannot fulfil these requests due to an inability to identify individuals, they must provide an explanation to the requestor, demonstrating why identification is not possible. The individual then has the option to provide additional information to aid in their identification, as outlined in Article 11(2) UK GDPR. 
 

Right to erasure, restriction of and to object to processing  

In its analysis, the ICO recognised that developers face challenges applying the right to erasure, restriction of and to object to processing due to memorisation issues inherent in AI models. AI models are prone to unintentionally outputting sections of the training data they have 'memorised' despite not being explicitly asked. To prevent this, developers often employ input and output filters. Input filters identify and modify user prompts, while output filters detect and adjust model outputs.  Developers are also expected to consider the implications for the fairness and statistical accuracy of generative AI models where groups of individuals exercise these rights (such as a specific community). 
 

Individual rights in AI deployment  

Individuals' rights need to be respected throughout the AI lifecycle and supply chain, including during deployment. This also relates to the personal data that is submitted to the model once deployed and any outputs that can constitute personal data. Responsibility for fulfilling these rights lies with the organisations who are controllers or joint controllers at the various stages involved in the development and deployment of these generative AI models.
 

What is the ICO requesting?

Based on its analysis, the ICO is seeking the following:

• views on what further measures generative AI developers should take to protect individuals' rights and freedoms;
• evidence on how requests to exercise the right to object are being respected in practice;
• views on whether input and output filters are a sufficient mechanism for enacting people's rights; and
• views on what mitigation measures should be in place where groups of individuals exercise their rights.  

The ICO is interested in receiving evidence on tested, verifiable and effective methods that organisations are developing or using to meet their obligations in this area.  
 

NOYB complaint against OpenAI

On 29 April 2024, NOYB (the organisation behind complaints against Meta's EU-US data transfers and more recently the "Consent or Pay" model) filed a complaint regarding OpenAI with the Austrian supervisory authority alleging that ChatGPT provides false information and OpenAI does not have any ability to correct it.  

When asked about the complainant's (a public figure) birthday, ChatGPT repeatedly provided the wrong date of birth. However, OpenAI refused the complainant's request to correct or delete the data, claiming that it wasn't possible to do so. The only solution was for OpenAI to filter or block data on certain prompts (such as the name of the complainant), but it would be challenging to selectively filter out only specific details while retaining the rest of the context related to the complainant. OpenAI also allegedly failed to adequately respond to the complainant's access request.
 

Our takeaway thoughts

Overall, it is clear that generative AI presents many challenges to ensuring individuals are informed about how their personal data is being processed and how to respond to data subject rights requests. It also highlights that data subject rights must not be an afterthought, but that rights mechanisms need to be considered early and built into the design of the AI system to respond to these requests.