Private LLM or privacy proxy: do you need your own model?

You need your own model less often than the pitch suggests. A private LLM, meaning an open-weight model served on hardware you control, keeps every prompt inside your walls, and the OAIC itself says local or on-premises deployment is likely to be more privacy-preserving. It also makes you the operator of a GPU stack, with the hardware, the patching and the model upgrades that come with it.

Nonimo Gateway runs on your own servers. See plans and get in touch.

For most Australian practices of 10 to 100 people, the question underneath is narrower: does the model need to see who the client is? If the answer is no, you can keep a hosted model through its API and send it a request with the personal data replaced. This guide sets out both paths, honestly, and how to tell which one your work calls for.

Private LLM vs privacy proxy: the two architectures

A private LLM moves the model to the data. You choose an open-weight model, serve it on GPUs in your server room or your own cloud account, and point your apps at it. Nothing reaches a model provider because there is no provider in the loop.

A privacy proxy moves the data away from the model. Your apps keep calling a hosted model such as OpenAI’s, but every request passes through a proxy on your servers that swaps names, addresses and other identifiers for placeholders, and swaps them back in the answer. The provider sees the work, not the people.

	Private LLM	Privacy proxy in front of a hosted model
Where the model runs	Your hardware or cloud tenancy	The provider’s infrastructure
What leaves your network	Nothing	The request, with personal data replaced
Hardware	GPUs sized to the model and the users	An ordinary server or container
People	Someone to serve, patch, evaluate and upgrade models	Someone to deploy and update one service
Model quality	Whatever open-weight model you can run	The provider’s current models
APP 8 question	Mostly falls away	Reduced, and still worth a written assessment

What the OAIC actually says about hosting your own

The OAIC’s guidance on commercially available AI products makes two points that matter here. One: where an organisation uses a proprietary system with protections against information leaving the organisation, putting personal information into it is a use rather than a disclosure. The other: it says that deploying a system locally or on premises is likely to be more privacy-preserving, because it limits the risk of outside parties reaching the data.

Neither point is a requirement to self-host. The same guidance recommends, as a general step, minimising the personal information that goes into an AI system at all. A request with the names taken out is minimisation, whichever model reads it.

There is a catch on the proxy side, and it should be stated plainly. The Privacy Act defines personal information and de-identified information, and nothing between them. Text you have pseudonymised, while keeping the way back, is still personal information in your hands. De-identified vs pseudonymised walks through why. The proxy does not take you outside the Act; it changes what the provider receives and what sits in its logs.

When hosting your own model makes sense

Self-hosting is the right call more often in some sectors than others. It tends to pay off when several of these hold:

• A contract or a client says the data cannot leave. Some government, defence and health work comes with terms that rule out any external processing, pseudonymised or not.
• The identity is the task. Matching records across systems, checking a person against a list, or reasoning about relationships between named parties needs the names.
• The volume is high and steady. Hardware you own earns its keep when it is busy, and costs the same when it sits idle.
• You need a model trained on your own material. Fine-tuning on internal documents is easier when the weights are yours.
• You already have the people. A team that runs GPU servers and evaluates models is the largest cost, and some firms have it already.

The costs are real and mostly about people. Hardware is a purchase or a monthly rental. Keeping a model current, measuring whether it still gives good answers, securing the server and handling the next model release is a job that does not finish. We are not putting a dollar figure on it here, because it varies too much with model size and usage to be honest about in general.

When keeping the API and pseudonymising is enough

For most professional services work, the model needs the structure of the problem and not the identity of the person. A conveyancing summary, a draft letter of advice, a list of deadlines from a lease, a triage note for an intake form: all of them read the same with “Client A” in place of a real name.

In those cases a privacy proxy gives you most of what a private LLM would, for this specific risk, without the GPU stack:

1. The provider never receives the identifiers, so its retention and logging matter less.
2. Your own logs can hold the pseudonymised version.
3. You keep using the hosted models your staff already rely on.
4. The real names come back in the answer, so the output is usable straight away.

Hosted providers have also moved on data location. OpenAI’s data controls page, for example, lists Australia as a data residency region for storage, while marking regional processing as unavailable there. That helps with storage questions and leaves the processing question to your APP 8 assessment, which is another reason to send less in the request. Which assistants suit which uses under the APPs is in Australian Privacy Principles and AI tools.

A worked example: a family law practice in Parramatta

An invented practice, Harbourside Family Lawyers, has 25 staff. Its IT contractor is asked to let solicitors summarise affidavits and draft chronologies with AI, and the partners start by asking for a private LLM.

Question	Their answer	What it points to
Does the model need to know who the parties are?	No, it needs dates, events and the order they happened in	Proxy
Does any client contract forbid external processing?	No	Proxy
Is there someone to run GPU servers?	No, one contractor two days a week	Proxy
Is any work so sensitive it must not leave at all?	Children’s matters with a court order in place	Keep those out of AI, or on a local model

The practice keeps OpenAI through the API, puts a privacy proxy in front of it, and writes a rule that matters involving suppression or non-publication orders stay out of AI entirely. A litigation practice also has the court’s own rules to follow, covered in the NSW Supreme Court AI practice note.

Nonimo Gateway, in short

Nonimo Gateway is the proxy option. It replaces personal data before the request reaches the model API and restores the real data in the response. It runs on your own servers or in your own cloud and accepts OpenAI-style calls, so developers change base_url and nothing else. It is also available as a LiteLLM plugin, and you call OpenAI with your own key.

Plans and prices are on the Nonimo Gateway page.

It helps you minimise what reaches the provider; it does not make your use of AI lawful on its own. For the wider market of detection tools, see PII redaction software for LLM apps and alternatives to Presidio. For the traffic layer around it, what an AI gateway is, and for the ways data escapes from your own code, ChatGPT data leak from your own apps.

Making the call

Ask whether the model needs the identity, whether anything forbids external processing, and who would run the servers. If the answers are no, no and nobody, a privacy proxy in front of a hosted model is the proportionate step, and a private LLM can wait until a workload truly needs one. See Nonimo Gateway’s plans and get in touch.

Sources

Checked on 21 October 2026 unless stated.

• OAIC, Guidance on privacy and the use of commercially available AI products, published 21 October 2024, updated 17 January 2025. Use rather than disclosure for protected proprietary systems, local or on-premises deployment as more privacy-preserving, minimisation.
• OAIC, APP Guidelines chapter 8, updated 3 October 2025. Overseas recipients, effective control and section 16C.
• OAIC, De-identification and the Privacy Act. The definition of de-identified information.
• Privacy Act 1988 (Cth). Section 6(1) definitions of personal information and de-identified.
• OpenAI, data controls in the API platform. Data residency regions, including Australia for storage.

The practice, its staff and its matters are invented.