Where should the AI actually run?

I was discussing a fairly ordinary AI problem this week: what to do when someone types personal information into an AI application.

Suppose someone writes their name, address or phone number into a chat. You may not want that information to reach the large language model. So you put something in between that recognises personal information and removes it.

Something like this:

User → browser → application server → PII filter → LLM

Perfectly reasonable.

Except that the personal information may not reach the LLM, but it has already reached us. Our application server has received the original message.

So I wondered: can you move the filter?

User → browser → local PII filter → application server → LLM

Almost the same picture. One component has moved.

But architecturally, something important has changed. The original personal information may now never leave the user's device.

Then I happened to find Desert Ant

Later that day, quite by coincidence, I saw a post by Paul Veugen announcing a new company called Desert Ant Labs.

I have followed Paul for a while, so I clicked out of curiosity. Not because I was looking for a PII solution.

Desert Ant grew out of work behind Detail, where the team had been using machine learning for things such as transcription, audio enhancement and video processing. Instead of relying on a cloud API for every task, they started developing small models that could run locally.

While looking through the models, I found one called Redact.

Which was almost exactly the thing I had been thinking about a few hours earlier.

Desert Ant Labs: Little brains in every product

The fastest on-device models for the job. Add them to your app in a few lines of code, at no token cost.

Desert Ant Labs

A small model with one job

Redact is not an LLM in miniature.

It has a particular job: recognise personally identifiable information in text and replace it with placeholders. Desert Ant provides it for native applications, but also for JavaScript applications running on the web.

That made the earlier conversation concrete.

A web application does not necessarily have to send the original text to its server and then decide what should go to an LLM. It can perform some computation locally first.

So this:

User → browser → application server → PII filter → LLM

can become this:

User → browser + PII filter → application server → LLM

The important distinction is the trust boundary. In the second version, the application server may never receive the original identifying information.

That does not make the application anonymous or privacy-proof. Detection can fail, context can still identify someone, and server-side safeguards may remain necessary.

But as a way of minimising what data leaves a device in the first place, it is an interesting building block.

Apparently, you can build applications this way

This is where my little PII question became more interesting.

I am not an application developer. I spend enough time around software and AI development to understand quite a bit of what is happening, but I do not spend my days deciding which process runs where.

What clicked for me was that AI can become one of the components inside an application, rather than something the whole application simply sends work to.

A native app can run models on the phone. A web application can do computation locally in the browser. A backend can still handle the things that belong on the server. And a large external model can be called when the task actually needs one.

Google's work with Gemini Nano and Chrome's built-in AI APIs is another, quite different, example of local rather than cloud AI: there the browser platform itself supplies local AI capabilities rather than an application incorporating a specialised model.

That is a different mental model of "adding AI". It also connects to something I wrote about earlier in Looking Past the Model: the model itself is only one part of the system we are actually building.

Not every AI task needs a large model

Desert Ant makes that visible because its models are deliberately narrow: language detection, transcription, audio enhancement, PII detection and similar tasks.

These are useful jobs, but they do not require a model that can write essays, explain quantum mechanics and discuss philosophy.

That is probably what stayed with me from the whole discovery.

I started with the question: how do we stop personal information from reaching the LLM?

A few hours later, I had a different one:

Where should this particular piece of intelligence run in the first place?

Sometimes the answer will still be a large model somewhere in the cloud.

But apparently, quite a lot can happen before you ever get there.