Private LLM vs a privacy proxy: do you need your own model?

A private LLM is a model that runs on infrastructure you control, so prompts never reach a shared public API. It is the right answer when the content itself must not leave your network. It is a costly answer when the real worry is narrower: the names, emails and ID numbers inside otherwise ordinary requests.

Nonimo Gateway runs on your own servers. See plans and get in touch.

For that narrower worry there is a lighter option: keep the hosted model you already use and put a privacy proxy in front of it, which replaces personal data before the call and restores it in the answer. This guide sets out what each option costs in general terms, where each one fits and how to tell which problem you actually have.

What “private LLM” means in practice

The phrase is used for three quite different set-ups, and vendors do not always say which one they mean.

Set-up	Where the model runs	Who sees your prompts
Self-hosted open-weight model	Your servers or your own cloud account, on GPUs or other accelerators	You
Dedicated deployment	A provider’s infrastructure, reserved for you	You and that provider, under your contract
Business plan of a public provider	Shared provider infrastructure, with contractual terms on training and retention	You and the provider, under its data terms

The self-hosted model is the one that keeps prompts entirely inside your walls. The other two are hosted models with contracts on top, which can be perfectly adequate, but they are not what most people picture when they say “private”.

What a private LLM costs, without the sales figures

Prices for hardware and cloud GPUs move too fast for any number here to stay true, so it is more useful to list what you are signing up for.

• Hardware sized for the peak. A model that answers staff during office hours needs enough capacity for the busiest hour, and that capacity sits idle at night. Larger models need more memory on the accelerator, and longer documents need more again.
• A model choice you have to keep making. Open-weight models improve quickly. Someone has to test new releases on your own tasks and decide when to switch.
• Evaluation. Hosted providers test their models at scale. On your own, you need a set of real tasks and someone to check that answers stay good after each change.
• Security and operations. Patching, access control, monitoring, backups and an answer to “who is on call when it stops”.
• People. This is usually the largest line. Running a model well is a skill set that a small IT team may not have, and it competes with everything else they do.

None of this makes self-hosting wrong. It makes it a project, rather than a setting.

When a private LLM makes sense

Self-hosting earns its cost in a few clear situations:

• The content cannot leave, not just the names. Defence work, unreleased financial results, source code under strict licence or a client contract that forbids any external processing.
• You need to work offline or in a network with no route to the internet.
• Volume is high and steady, so dedicated hardware stays busy around the clock.
• You want to fine-tune on your own data. Be careful here: in its December 2024 opinion on AI models, the European Data Protection Board said whether a model trained on personal data is anonymous must be assessed case by case. A model tuned on client files may itself hold personal data.

When keeping the API and pseudonymising the request is enough

Most requests from law firms, accountancies, clinics and HR teams are sensitive because of who they are about, not because of what they ask. “Summarise this letter”, “draft a reply”, “list the deadlines in this contract” all work just as well when the client is called PERSON_1. In those cases:

• the model does not need the real name, address or account number to do the task;
• your team wants the quality of a frontier hosted model without running one;
• you have no GPU operations skills in-house and do not plan to hire them;
• what you must show a regulator, a client or an insurer is that identifiable data was minimised before it went out.

A privacy proxy fits that profile. It sits between your code and the provider, replaces personal data with stand-ins, and swaps the real values back into the answer. Your staff see a normal answer; the provider sees stand-ins.

Two cautions keep this honest. Pseudonymised data is still personal data for whoever holds the key; the ICO says so, and the definition in Article 4(5) of the GDPR depends on that extra information being kept separately. That is why the mapping table should stay on your servers. And the provider still processes what you send: OpenAI states that abuse monitoring logs are kept for up to 30 days by default, so you still need its data terms in place, just with less in the request.

A side-by-side decision table

Question	Points to a private LLM	Points to a privacy proxy
What is sensitive?	The content itself	The identities in it
Can any of it leave your network?	No	Yes, once personal data is replaced
Do you need the strongest hosted models?	Not essential	Yes
Do you have people to run GPUs and models?	Yes	No
Is usage high and steady?	Yes	Varies
Do you need to fine-tune on client data?	Possibly	No

If your answers split, the combination works too: routine requests through a hosted model behind a proxy, and a small self-hosted model for the documents that may not leave.

An example: a clinic group that nearly bought GPUs

A group of four physiotherapy clinics, 25 staff, wants AI to turn session notes into referral letters. The IT lead’s instinct is a self-hosted model, because the notes are health data. On a closer look, the notes read like this (invented):

Tom Hadley, DOB 14/02/1981, tom.hadley@example.com. Right ACL reconstruction 2025, now full range of motion. Refer back to Mr K. Osei at Riverside Orthopaedics.

The clinical content is ordinary for a physiotherapy practice; what makes it sensitive is that it is tied to Tom. With the name, date of birth, email address and the surgeon’s name replaced before the call, the hosted model writes the same letter, and the real details are put back before it reaches the clinician. The group drops the GPU plan, saves the project for later and keeps one rule: notes about rare conditions, where the story could identify the patient alone, stay out of the tool.

Nonimo Gateway

Nonimo Gateway is the privacy proxy for that second path. It runs on your own servers. The API is OpenAI-compatible, so you change base_url and keep your code and your own OpenAI key. It replaces personal data before the request reaches the model and restores the real values in the answer. It recognises the national documents and numbers of each country it serves, such as the Spanish NIE, the Italian codice fiscale and the Mexican CURP. A LiteLLM plugin is available.

Plans and prices are on the Nonimo Gateway page.

Choose the problem before the product

A private LLM answers “nothing may leave”. A privacy proxy answers “nothing that identifies a person may leave”. Work out which sentence describes your data, then pick. For how the proxy sits next to routing and logging, see what is an AI gateway; for the routes data takes out of an app, AI data leakage. If you are comparing self-hosted detection tools, start with the Presidio alternative and the PII redaction software overview. Our guide to what to redact before you paste a document into an AI covers which details count, and if your sector is health or HR in the US, PHI vs PII adds the American terms.

If the identities are your problem, not the content, see Nonimo Gateway’s plans and get in touch before you buy hardware.

Sources

• EDPB, opinion on AI models, 18 December 2024. Whether an AI model is anonymous is assessed case by case.
• ICO, pseudonymisation. Pseudonymised data stays personal data for whoever holds the additional information.
• GDPR, Regulation (EU) 2016/679. Article 4(5), additional information kept separately.
• OpenAI, data controls in the OpenAI platform. Abuse monitoring logs for up to 30 days; API data not used for training by default.

All checked 20 October 2026.

---

This page is for information and is not legal advice. For your own situation, ask your regulator or a qualified adviser.