What Is Your AI App Sending? Watching LLM Traffic

Where does your text go?

Most AI features in apps work the same way: the text you type, paste or record leaves your phone, travels to a server, and comes back as a reply. Which server, who runs it, and who can see the text on the way are questions most apps never answer on their face. There is a piece of plumbing in the middle of this that developers argue about, the LLM proxy. Understanding it makes it much easier to judge what an AI app is doing with your words.

What an LLM proxy is

An LLM proxy is a service that sits between an application and an AI provider’s API. It receives each request, passes it on, and returns the answer. Along the way it can cache replies, limit usage, manage keys and keep logs. Developers like that because it gives them one place to see cost and behaviour. The trade-off is that a proxy adds delay, and a badly configured one adds security and privacy risk, because everything flows through it.

That last point is the one that matters to you as a user: whoever runs the proxy can, in principle, see every prompt and every reply that passes through it.

Three ways an AI app can be wired

• Direct to the provider. The app talks straight to the AI company’s servers, often with a key you supply. The app’s developer may never see your text. The provider does.
• Through the developer’s server. The app sends your text to the developer’s backend, which forwards it to an AI provider using the developer’s own key. This is convenient and common, and it means the developer’s server can see, and could log, what you send.
• On the device. The model runs on your phone or Mac and nothing leaves it. The trade-off is smaller models and less capability.

Real apps often mix these. An app might translate on-device by default and offer a cloud option as an extra, which is why “does it use AI?” is the wrong question. “Where does my text go?” is the right one.

How to check what your app sends

You can see this for yourself, on your own phone and your own account. We covered the full method in our guide to checking whether an iPhone app uploads your data. The short version:

1. Install a traffic inspector such as Proxyman or mitmproxy and route your phone’s Wi-Fi traffic through it. You will need to trust its certificate so it can read encrypted traffic.
2. Open the app and use one AI feature, ideally with a harmless test sentence.
3. Read the hostnames first. A request to a provider such as api.openai.com, api.anthropic.com or generativelanguage.googleapis.com means the app is calling the provider directly. A request to the developer’s own domain means your text is going through their server.
4. Then read the request. Is it only the text you chose, or does it include extra context, identifiers or other fields? Check the headers too, to see where a key is being sent.
5. Note the other destinations. Analytics and advertising calls tell you as much as the AI call does.

Two limits. Some apps pin their certificates, which makes them refuse to work behind an inspector, so a blank result is not proof of nothing. And this only works on traffic from your own device. Don’t inspect other people’s.

Why people run their own proxy, and what it costs

The case for running your own is visibility: you can see every request, track cost, rotate keys in one place and swap providers without touching the app. Open-source gateways such as LiteLLM exist for this. The costs are real, though. You add latency, you become responsible for securing the keys and the logs, and you now hold a record of everything that passes through, which is itself something that has to be protected. A proxy you run gives you visibility. It doesn’t remove the provider from the picture.

Bring your own key: the honest trade-off

Some apps let you paste in your own API key. That can be a good arrangement. The app’s developer isn’t paying for your usage or sitting in the middle of your text, and you can check the traffic yourself. But you take on the provider’s terms, and a key is a credential: anyone who gets it can spend on your account, so use a key with a spending limit and treat the app’s handling of it as something to verify.

We make Offgrid Locale, an App Store listing translator with this option. Its full version can use your own key with Claude, ChatGPT, Gemini or a custom provider, and when you use those, the text you translate goes to the service you chose. It also has an on-device mode and a copy-and-paste mode. We are not offering that as proof of anything; it’s an example of the three wirings above. See the Offgrid Locale page for what it does.

Before you paste something private into an AI app

• Ask where it goes. Does the app say whether text goes direct to a provider, via its own server, or stays on the device?
• Check the provider’s data terms. Retention and training policies differ and change. Read the current ones for the service you are using.
• Prefer on-device for the sensitive stuff. If a smaller local model is good enough, nothing leaves your phone.
• Test with a harmless sentence first. Inspect the traffic once. Ten minutes of checking tells you more than a privacy page does.

Sources

• Langfuse: should you use an LLM proxy?
• CostHawk glossary: LLM proxy
• MintMCP: LLM proxy overview

The Hacker News discussion that prompted this piece asked how people know what is going on if they don’t run their own proxy. We have not relied on that thread’s contents here. We are not affiliated with any tool named above, and this is general information, not security advice for your specific setup.

More from Offgrid Studio

How to Check If an iPhone App Is Secretly Uploading Your Data (Free, 15 Minutes)
AI Cloned Photoshop in Days. Can Adobe Sue?
Running an AI Model Bigger Than Your RAM: What Works