AI gateway in Australia: when your firm needs a private one
· Written and maintained by Nonimo
An AI gateway is a server that sits between your applications and the model APIs they call. It gives your developers a single endpoint, holds the provider keys so they stop living in a dozen config files, routes and retries requests, caps spending and logs every call. For most teams that log is the point: who called which model, how often, and at what cost.
Nonimo Gateway runs on your own servers. See plans and get in touch.
For an Australian business the log is also where the Privacy Act starts to bite. A gateway that records full prompts holds whatever your staff and your software put in them: client names, matter details, a Medicare number pasted where it was never needed. This guide explains what a gateway does, when you need one you run yourself, and where a privacy proxy fits beside it.
What an AI gateway does, and what each feature touches
Gateways differ in features, but the core set is consistent across open source and hosted products.
| Feature | What it gives you | What it touches |
|---|---|---|
| A single endpoint | One address for every app, whatever the model behind it | Every request passes through it |
| Central keys | Provider keys held in one place, with per-team virtual keys | Nothing personal, but it is the choke point |
| Routing and fallback | A second provider or model when the main one fails | Data may go to a provider you did not plan for |
| Rate limits and budgets | Spending caps per team or app | Metadata, as a rule |
| Caching | Repeated questions answered without a new call | Stores prompts and answers to match them |
| Request logs | A record of every call for audit and debugging | Stores prompts and answers, often in full |
The last two rows are the ones a privacy officer should read twice. A cache and a log are both stores of the text that went to the model, and that text is often the most sensitive thing your business handles all day.
Where the prompts end up: the gateway log under APP 11
If your business is an APP entity, the log is personal information it holds, and APP 11 asks for reasonable steps to protect it from misuse, interference, loss and unauthorised access. Since the Privacy and Other Legislation Amendment Act 2024, APP 11.3 states that those steps include technical and organisational measures. The OAIC notes that this applies to personal information held from 11 December 2024, whenever it was acquired or created.
APP 11.2 then asks you to destroy or de-identify information you no longer need. A gateway that keeps every prompt for a year because nobody changed the default is the kind of store that clause is written for. The OAIC’s APP 11 guidance also expects your steps to cover third party providers, cloud included, and to verify that de-identification or destruction you asked a provider for has actually happened.
Two further questions decide how heavy this gets:
- Where the gateway runs. A hosted gateway in another country, run by someone else, is a third party holding your clients’ prompts. The OAIC’s APP 8 guidelines treat sending data to an overseas cloud provider as a use, not a disclosure, where a binding contract limits the provider to storage and access, subcontractors are bound the same way, and your business keeps effective control. Otherwise APP 8 applies, and section 16C makes you accountable for what the overseas recipient does.
- Whether a breach of it is notifiable. Unauthorised access to a log full of client details is a candidate for the Notifiable Data Breaches scheme, which applies where serious harm is likely and you cannot prevent it with remedial action. You then have 30 days to assess a suspected breach.
The OAIC’s guidance on commercially available AI products adds a plain recommendation: minimise the personal information you put into an AI system at all. A gateway is a good place to measure that. It is a poor place to keep it.
When an Australian firm needs a private AI gateway
A hosted gateway is a sensible choice for many teams. A private one, running on your own servers or in your own cloud account, is worth the effort when several of these are true:
- The Privacy Act reaches you. Turnover above $3 million, or an exception that catches you regardless: health service providers, AML/CTF reporting entities for those activities, contractors to the Commonwealth, and others on the OAIC’s list.
- Your apps send client data, not just public text. A tool that drafts replies to client emails, summarises file notes or reads uploaded documents sends personal information on every call.
- Sensitive information is in the mix. Health, criminal record and other sensitive categories carry a higher standard, and a clinic or an NDIS provider will meet them daily. See sensitive information under the Privacy Act.
- Clients ask where their data goes. If a client writes it into the contract, you need an answer you can show, not a vendor’s default setting.
- You want the numbers without the content. Usage per team and per app, with no prompt text sitting in a log.
If you tick none of these, a hosted gateway with logging switched to metadata may well be enough.
Where a privacy proxy fits beside the gateway
A gateway decides where traffic goes. It does not, by default, change what the traffic says. That is the job of a privacy proxy: it reads the outgoing request, replaces personal data with placeholders, sends the cleaned request to the model, and puts the real values back in the answer before your app sees it.
Some gateways can call a privacy filter as a plugin, and they differ in what they do with a hit. LiteLLM runs guardrails in several modes, including a logging_only mode that masks data before it reaches the logs, per its guardrails documentation. Cloudflare’s AI Gateway can flag or block requests that match a DLP profile, per its DLP page, rather than masking them. Kong’s AI PII Sanitizer, an Enterprise plugin, can replace data and optionally restore it in the response, per its plugin page. The detail is in our comparisons of LiteLLM PII guardrails, Cloudflare AI Gateway and PII and Kong AI Gateway and PII.
The practical order is simple: your app, then the privacy proxy, then the gateway (or the gateway calling the proxy), then the model. Whatever the gateway logs after that point is already pseudonymised.
A worked example: an accounting practice in Launceston
An invented practice, Tamar Valley Accounting, builds an internal tool that drafts replies to client emails using the OpenAI API. Staff paste the client’s message; the tool sends it with a short instruction. One client writes:
Hi Priya, it’s Daniel Okafor. My new address is 14 Wattle Grove, Prospect Vale. Can you lodge my return before I start at the hospital in March? My email is d.okafor@example.com.
| Stage | Without a privacy proxy | With a privacy proxy in front |
|---|---|---|
| Sent to the model | The full message as written | Names, street address and email replaced with placeholders |
| Kept in the gateway log | The full message, for as long as the log lives | The placeholder version |
| What the staff member sees | A draft reply | The same draft, with the real names and details back in |
Nothing about the drafting changes. What changes is that the log and the provider never held Daniel’s name, street address or email. The tool’s own code should also strip what the task does not need at all: a tax file number or Medicare number belongs in the practice software, not in a prompt. How to redact a TFN before AI covers the manual side.
Nonimo Gateway, in short
Nonimo Gateway is a private proxy that replaces personal data before a request reaches the model API and puts the real data back in the response. It runs on your own servers, and the content does not pass through Nonimo. It speaks the OpenAI API, so your developers change base_url and keep their code, and it runs as a plugin for LiteLLM if that is already your gateway. You keep your own OpenAI key.
Plans and prices are on the Nonimo Gateway page.
It helps with the minimisation the OAIC asks for, and it does not replace your privacy policy or your APP 8 assessment. If you are still deciding whether to host a model yourself instead, read private LLM or privacy proxy, and for the ways data escapes from your own code, ChatGPT data leak from your own apps.
Before you choose
Write down what the gateway will log, for how long, and in which country, before anyone switches it on. Decide whether personal data should reach the log at all. Then decide whether it should reach the model. A private AI gateway with a privacy proxy in front answers both questions on your own servers. See Nonimo Gateway’s plans and get in touch.
Sources
Checked on 21 October 2026 unless stated.
- OAIC, APP Guidelines chapter 11, updated 3 October 2025. APP 11.1, APP 11.2, APP 11.3 from 11 December 2024, third party and cloud providers.
- OAIC, APP Guidelines chapter 8, updated 3 October 2025. Use or disclosure with overseas cloud providers, effective control, section 16C.
- OAIC, Data breach preparation and response, Part 4. The three criteria of an eligible data breach, the 30 day assessment in section 26WH(2), remedial action.
- OAIC, Guidance on privacy and the use of commercially available AI products, published 21 October 2024, updated 17 January 2025. Minimising personal information input into AI systems.
- OAIC, Small business. The $3 million threshold and the exceptions.
- LiteLLM, guardrails quick start, checked 20 October 2026. Guardrail modes, including logging_only.
- Cloudflare, AI Gateway DLP, checked 20 October 2026. Scanning of prompts and responses, flag and block actions.
- Kong, AI PII Sanitizer plugin, checked 20 October 2026. Enterprise plugin, placeholder replacement, optional restoration.
The accounting practice, its staff and its client are invented, and the email address uses a domain reserved for examples.
Nonimo is the software that does this on your own computer: it redacts client names and IDs before your text reaches ChatGPT. No account needed, and the app does it without your files leaving your machine.
Common questions
What is an AI gateway?
An AI gateway is a server that sits between your applications and the model APIs they call. It gives your developers one endpoint, keeps the provider keys in one place, routes and retries requests, applies rate limits and records each call, so IT can see who used which model and what it cost.
Does an AI gateway store my prompts?
Many do, because request logging is one of the main reasons to run one. If those prompts carry a client's name, email address or case details, the log is a record of personal information, and APP 11 applies to it like any other system your business holds.
Is a hosted AI gateway an overseas disclosure under APP 8?
It can be. The OAIC's APP 8 guidelines say handing data to an overseas cloud provider is a use, not a disclosure, in limited circumstances: a binding contract that restricts the provider to storage and access, and your business keeping effective control. Where the provider can do more with the data, APP 8 and section 16C accountability come into play.
When does a small Australian business need a private AI gateway?
When the Privacy Act reaches it and its own software sends client data to a model API. Businesses with annual turnover of $3 million or less are generally exempt, but health service providers, AML/CTF reporting entities and several other groups are covered whatever their turnover.
What is the difference between an AI gateway and a privacy proxy?
A gateway manages traffic and a privacy proxy changes its content. The gateway decides where a request goes and records it; the privacy proxy replaces personal data in the request before it leaves your network and puts the real values back in the answer. The two can run together, one behind the other.
Can I keep using my own OpenAI key with a private gateway?
Yes. A private gateway runs on your infrastructure and calls the provider with the key you already have, so your contract and billing with OpenAI stay as they are. What changes is that the requests pass through a server you control before they leave.