A new GDPR complaint is addressing ChatGPT’s tendency to “hallucinate” information about individuals and present it as if it is fact, something that parent company OpenAI has already been taken to court over in private lawsuits.
Filed by data privacy crusader Max Schrems and his group “noyb,” the GDPR complaint asserts that OpenAI refuses to correct ChatGPT output about individuals and will simply try to filter or block requests tied to that name. The complaint also accuses OpenAI of failing to live up to their subject access request (SAR) responsibilities under EU rules, in some cases not telling requesters where it obtained their personal information or who it has been shared with.
GDPR complaint filed in Austria, follows similar 2023 complaint lodged in Poland
As noyb’s GDPR complaint points out, Articles 5 and 16 of the regulation specifically protect EU residents in this area. Personal information that is reported publicly must be accurate, and individuals have a right to rectify incorrect data of this sort or request its deletion. Article 15 also specifies that data holders must be able to identify where personal data came from and inventory what they are sitting on, upon request.
OpenAI’s take thus far appears to be that ChatGPT is something different, and operates in such a way that these regulations cannot apply to it. The company says that it cannot go into training data to correct the chatbot’s output, nor can it inventory what the model has on a particular person or where it got this information from. It also cannot guarantee that “hallucinations” will not happen and that it will not spit out completely untrue information in response to a query about someone.
The current GDPR complaint centers on inability to correct a subject’s birthday, which ChatGPT consistently gets wrong when asked. OpenAI has said that they cannot change the birthday, and that in cases like this they will simply filter or block queries involving the subject. Complicating the issue somewhat is that OpenAI’s privacy policy does offer users “correction requests” meant to address factually inaccurate information, though it also includes a disclaimer that data may not be corrected in some cases due to “technical complexities.” It is not clear in which cases, if any, OpenAI is able to correct personal information that is being reported in error.
A GDPR complaint that was filed in Poland in August 2023 is very similar. That case saw the country’s Office for Personal Data Protection (UODO) open an investigation after privacy researcher Lukasz Olejnik asked ChatGPT to create a biography and found it to be full of inaccurate information. As with the current complaint in Austria, OpenAI refused requests to correct these inaccuracies. The central points of GDPR criticism are the same, but the complaint in Poland does not involve noyb and cites a broader range of Article violations. That investigation is still open, as authorities there have cautioned that it is likely to be a slow process.
noyb has asked Austria’s data protection authority to open its own similar investigation, order OpenAI to comply with the personal issues raised in the GDPR complaint, and impose a fine to ensure future compliance.
Chatbot hallucinations continue to cause legal issues
The issue highlights the tightrope that many nations are presently walking with AI tools. Everyone sees them as the “next big thing” and an economic necessity, and fears being left behind if they are not given room to develop. But they are also being governed by outdated rule sets that, in many cases, did not anticipate something like ChatGPT existing. In the case of this string of GDPR complaints, strictly applying the rules as they exist could potentially put the model out of business.
The central issue is that personal information held by AI systems can only be inventoried by examining the training data, but these companies necessarily hold that training data close to their vest. And once a model has been trained, the errant information essentially can’t be removed; a new model has to be trained, which is a process that can take months and is not viable for every correction request. Some researchers are working on “machine unlearning” techniques to address this issue, but the technology is still in its very early stages. This method would also likely not be able to clear hallucinated data from a trained model.
The primary strength of LLMs like ChatGPT is in quickly and neatly summarizing facts, essentially creating a personalized Wikipedia article in seconds in response to a request. But recent research has found that these models have worryingly high rates of adding incorrect hallucinations to what otherwise appears to be factual information, ranging from 3% to 27% of the time depending on the model. Sometimes the model will come up with different incorrect answers at different times, as Schrems demonstrated in his recent GDPR complaint by asking ChatGPT to find his birthday: it came up with three different answers, all incorrect.
