How to protect data privacy when using and training language models
The other day I asked a question out loud and the answer I got was “I don't know, ask ChatGPT”.
Generative artificial intelligence has captured the attention of the tech industry and of the general public. From automatic text generation to creating images and video, advances in large-scale language models — or LLMs, for Large Language Models — have transformed how we interact with technology. Using and training these models, however, raises new data privacy challenges, a subject that keeps rising on the global regulatory agenda.
We'll go through the main privacy risks that come with training and using LLMs, and the best practices companies can adopt to protect personal data and comply with emerging regulations such as the new data protection law in Chile.
The new personal data protection law and its impact on using LLMs
Data protection laws, such as the reform of the Chilean law that takes effect in a few months and the General Data Protection Regulation (GDPR) in Europe, are shaping a new regulatory environment for companies that build and use AI models.
A key feature of the new Chilean law is that it gives users the right to object to their data being used to make automated decisions, including decisions based on profiling. That is especially relevant for LLMs, which among other things can build profiles from large volumes of information. In this context, companies deploying these technologies should make sure they are ready to meet those regulatory requirements and protect users' rights.
Where do the risks come from?
Training large-scale language models requires enormous amounts of data. That data comes from varied sources, including websites, books and public databases, but it can also contain sensitive or personal information.
The chance of pulling in personal or private data goes up when companies adapt models by retraining them on internal company data or on information their users provided directly. This kind of adjustment, known as fine-tuning, can put privacy at risk and expose confidential data in later interactions with the model.
The risks that emerge are therefore the following:
Collecting data without consent: much of the data used to train LLMs comes from scraping websites, which can include copyrighted content or personal information gathered without the owners' knowledge or authorization.
Exposure of sensitive data: because language models generate text from prior patterns, they could produce output that inadvertently reveals personal data. This is also known as "regurgitating" personal or sensitive information captured during training when the right prompt is entered, which compromises users' privacy.
No way to rectify or erase data: users have the right to request the erasure or correction of their data, but that is particularly complicated with LLMs. That's because models don't store data directly; they learn patterns, which makes it hard to delete or change specific information after training.
How to protect privacy while training LLMs
As we saw, training large-scale language models carries several risks, especially where personal data is involved. To comply with privacy regulations and avoid problems later on, companies have to follow a set of good practices from the very start of the training process. Some key recommendations:
Choosing your data sources carefully is essential. Not all available data is suitable: some may be protected by copyright or contain personal information obtained without consent. This is where companies have to be very selective and make sure the data complies with the regulations in force. At Soyio we build technology to manage users' consent and keep an up-to-date record of which data is authorized for AI training. If you want to know more, get in touch at hola@soyio.id
Another crucial piece is anonymizing and cleaning the data before using it for training. Removing personal identifiers and making sure the processed information is free of sensitive data significantly reduces the risk of exposure. Cleaning the data doesn't only protect users, it also improves the quality of the model by avoiding biases and errors. If companies decide to retrain models on internal data, they have to be even more careful. When you use your own data or data provided by users, it is very important to have mechanisms that minimize how much personal data gets processed, and to make sure it isn't integrated in a way that lets it be reused or leaked later.
Finally, a good practice is to restrict access to this data. Only the people who strictly need it should have access to the information during training. Access controls such as multi-factor authentication and role-based permissions keep sensitive information protected from unauthorized access. One way to secure this by design is to use federated learning: this technique lets you train models on the devices that generate the data, so the data never has to be transferred to central servers.
What if the LLMs are already in use?
Once models have been trained and start being used in technology products, the risks change but they don't go away. In fact, this is the phase where companies have to be especially careful with the information they handle, because users could provide sensitive data in their interactions with the models.
Some measures you can put in place:
Control the information users provide, especially if it will be used to retrain and refine the model. It is very easy for a user to reveal personal data without realizing it during an interaction with an LLM. To prevent that, you can use filters that automatically detect and anonymize this kind of information before it is stored for that purpose.
Even when the data isn't used to retrain the model, it is still important to guarantee proper separation between the information each user submits when interacting with the model. This has more to do with how the software is built than with the model itself, but it is essential to keep any personal data shared by a user out of reach of unauthorized people.
It is also important to have systems that constantly monitor the model's behavior. That helps detect possible information leaks or unauthorized access, and lets you react quickly to any suspicious activity, such as prompt manipulation that could trigger the generation of sensitive information.
Along the same lines, it is also worth taking measures to keep the model from reproducing sensitive data that was used during its training. This is a technical challenge, but there are ways to tune the algorithms to reduce the chance that the model "regurgitates" personal information.
Finally, running regular audits of the models' security and privacy is a practice that confirms the measures in place are working correctly. These audits surface possible weak points and let you reinforce the areas that need it.
Users' rights in the face of AI challenges
One of the main challenges raised by AI models like LLMs is how hard it becomes to exercise traditional data protection rights. Rights such as access, rectification or erasure become complicated to exercise, because LLMs don't store information in its original form. Instead of keeping the data as it was submitted, models turn it into mathematical parameters and patterns, which makes it practically impossible to trace and delete specific personal data once it has been used in the training process.
This technical limitation becomes a considerable challenge for fully exercising the rights set out in regulations like the GDPR or the new data protection law in Chile. Even so, there are measures companies can adopt while the problem remains an open question.
On the one hand, it means doing everything possible to keep models from being trained on personal data, using the measures described above.
On the other hand, one of the most relevant actions they can take is to guarantee the right to object to automated decisions. This right, required by the new Chilean law, lets users request that their data not be used to make decisions that could significantly affect them without human intervention. Not every data right can be applied to AI models, but companies can implement mechanisms that make sure users aren't subject to automated decisions without their consent or without proper review.
To do this, it is very important that companies are ready to handle these requests efficiently: keeping a record of the consents their users have granted, and being able to switch these features off granularly for the users who ask, making sure they aren't turned back on without their consent.
At Soyio we build technology that makes it easier for companies to manage the consents their users have granted, changed or revoked, granularly and in simpler language. That increases your users' trust and makes complying with the regulation easier for every internal team. If you'd like more information, get in touch.
Exercising these rights is a challenge that is still evolving, especially with LLMs, and while more advanced solutions are being developed, companies should focus on meeting today's regulations and offering transparency and control over how data is used in automated processes.
