ChatGPT data leak from your own apps: how it happens

When people search for a ChatGPT data leak they usually picture a colleague pasting a client letter into a chat window. That still happens, but there is a quieter route that IT teams own: the internal applications that call the OpenAI API with the firm’s key. A case management add-on, a CV screener, a tool that drafts replies to tenants. Each one builds prompts from your data, automatically and in volume.

Nonimo Gateway runs on your own servers. See plans and get in touch.

Personal data escapes those apps in five places: the prompt, the logs, the documents fetched for retrieval, the tool calls, and the copies the provider keeps. The fix is the same for all five: send less, replace what you must send with placeholders before it leaves, keep the key at home, and restore the real values in the answer.

How a ChatGPT data leak happens through the API

None of these routes needs a hacker. They are the normal working of an app built in a hurry.

Route	What leaves	Where it can stay
The prompt	Whatever the code concatenates: whole records, email threads, notes fields	The provider, for its retention period
Logs	Full requests and responses written for debugging	Your app logs, your gateway, your monitoring service
Retrieval (RAG)	Chunks pulled from your document store because they looked relevant	The provider, plus any vector store hosted outside
Tool calls	Arguments the model passes to your functions and the results you send back	The provider, plus the logs of every tool in the chain
Provider retention	Copies of requests, files and stored conversations	The provider’s systems, under its policies

The prompt is the obvious one. The other four are why firms that “just send summaries” are often surprised. A retrieval step may pull a neighbouring paragraph with a client’s medical history. A tool call to look up a payment can return a full bank record that is then sent back to the model. A debug flag left on in production writes all of it to a log that a monitoring vendor stores for months.

What the provider keeps

OpenAI’s own page on API data is the place to check, and as of 20 October 2026 it says four things a UK firm should know.

• Data sent to the API is not used for training by default, unless you opt in.
• Abuse monitoring logs are generated for all API usage and kept for up to 30 days by default, longer where the law requires it.
• Zero Data Retention removes customer content from those logs, but it is subject to OpenAI’s prior approval and additional requirements.
• Files, vector stores and conversations you create through the API are kept until you delete them. Responses API state is kept for 30 days by default.

There is also a UK-specific point. OpenAI lists the United Kingdom for data residency storage, but not for processing, so every request is processed abroad and the ICO’s rules on restricted transfers apply. Whatever your app sends is the thing that crosses the border.

For the consumer side of the same question, what ChatGPT itself shares and keeps, see does ChatGPT share your data.

Is it a personal data breach under UK GDPR?

Not every API call carrying personal data is a breach. The ICO defines a personal data breach as a breach of security leading to the accidental or unlawful destruction, loss, alteration, unauthorised disclosure of, or access to, personal data. A use you planned, documented, contracted and told people about is processing. A leak is what happens outside that plan.

In practice, a breach looks like this: an HR app sends whole personnel files when it was designed to send job titles, a retrieval step pulls documents from a matter the user has no access to, or a log containing client data is exposed to a supplier with no contract. If the breach is notifiable, you must tell the ICO no later than 72 hours after becoming aware of it.

The easier position is not to have the identities in the request at all. Article 25 of the UK GDPR names pseudonymisation as a data protection by design measure, and Article 32 lists it among the security measures. A placeholder that leaks is a much smaller incident than a name.

How to stop it, in four steps

1. Minimise. Decide, per app, which fields the model needs. A tenancy reply tool needs the tenant’s issue and the relevant clause, not their date of birth or their National Insurance number. Remove the rest in the code, and strip hidden metadata from Word and PDF files before they become retrieval chunks.
2. Pseudonymise before the call. Replace names, addresses, emails, phone numbers and identifiers with consistent placeholders inside your network, so the provider, its logs and any tool beyond your boundary see “PERSON_1” instead of a client.
3. Keep the key. The mapping between placeholders and real values stays on your servers. The ICO is clear that pseudonymised data is still personal data for whoever holds that additional information, so guard it as you would the original.
4. Restore in the answer. Put the real values back into the model’s response before your user sees it, including while the answer streams, so the tool stays useful.

A worked example

An invented one. Pennine Talent, a 30-person recruitment agency in Manchester, built a screening tool that sends each CV with a job description to the API and asks for a shortlist note. The CV of a candidate called Priya Mistry includes her home address in Didsbury, her mobile number, her date of birth, and a line about a career break for health reasons.

Without changes, all of that reaches the provider, sits in its abuse monitoring logs for up to 30 days, and is written in full to the agency’s own application log. With minimisation, the date of birth and the health line are dropped by the code, since the shortlist does not need them. With pseudonymisation, the rest becomes “PERSON_1, ADDRESS_1, PHONE_1”. The note comes back as “PERSON_1 meets four of five criteria”, and the proxy restores her name before the consultant reads it. The agency’s own log keeps the placeholder version.

The code change on the app side is one line, because the proxy speaks the same API:

from openai import OpenAI

client = OpenAI(base_url="https://ai-proxy.internal.example/v1")

Nonimo Gateway

Nonimo Gateway does steps two to four. It runs on your own servers, and the content does not pass through Nonimo. It is OpenAI-compatible, so you change base_url and keep your own OpenAI key. It replaces personal data before the request leaves and puts the real data back in the response, and it comes as a plugin for LiteLLM if your AI gateway is LiteLLM. Plans and prices are on the Nonimo Gateway page.

Step one stays with your developers, and the decision on lawful basis stays with you. The Gateway helps with pseudonymisation; it does not make an app lawful on its own.

Closing the leaks

Treat every app that calls a model as a data flow with five exits, and check each one: what the prompt carries, what the logs keep, what retrieval pulls, what tools return, and what the provider retains. If you are tempted to avoid the whole question by running a model in-house, weigh it against the proxy route in private LLM or privacy proxy. For tools that clean text in other ways, see our PII redaction software overview and the page on LiteLLM PII guardrails. And for the person pasting into a chat window rather than an app, what personal data to remove before AI covers the manual side.

See Nonimo Gateway’s plans and get in touch.

Sources

Checked 20 October 2026.

• OpenAI, Data controls in the OpenAI platform. No training on API data by default; abuse monitoring logs kept for up to 30 days by default; Zero Data Retention subject to prior approval; files and vector stores kept until deleted; Responses API state kept for 30 days by default; the United Kingdom listed for storage but not for regional processing.
• ICO, Personal data breaches: a guide. The definition of a personal data breach, including unauthorised disclosure, and the 72 hour reporting rule.
• ICO, A brief guide to international transfers, updated 15 January 2026. Restricted transfers outside the UK.
• ICO, Pseudonymisation. Pseudonymised data remains personal data for whoever holds the additional information.
• UK GDPR, Article 25 and Article 32, legislation.gov.uk. Pseudonymisation named as a data protection by design measure and as a security measure.
• UK GDPR, Article 5, legislation.gov.uk. Data minimisation, Article 5(1)(c).

---

This page is for information and is not legal advice. Product details of other vendors were checked on their official pages on 20 October 2026 and can change.