Blog · November 11, 2026

Should your prompts ship inside your app binary?

Where the API key lives is a settled question: on a server, never in the app. Where the prompt lives gets far less thought, and most mobile AI features still build theirs on the device and send the finished text to the model. That choice decides how fast you can fix a bad answer, and whether every old version of your app keeps running a prompt you’d rather forget.

What a shipped binary commits you to

A web app has one version: the one you deployed last. A mobile app has every version anyone still has installed. Auto-update is on by default, but people turn it off or stay on an old OS your new build no longer supports. Each of those devices keeps running the prompt it shipped with, unchanged, for as long as it’s installed.

So a prompt fix that would take five minutes on a server takes a release cycle in an app. You change the string, build, wait for review (Apple says most submissions are reviewed within a day, and some take far longer), then wait again while the update spreads. Even once it has, a tail of older installs still sends the old wording.

The model name has the same problem with a sharper edge. Providers retire models on a schedule. Google shut down gemini-1.5-flash-002 on September 24, 2025, and requests to a retired model ID simply fail. A binary that hardcodes that name doesn’t get worse answers after the date; it gets none. (We wrote about the fallback side of this in a hardcoded model name is a single point of failure.)

Four things, four possible homes

“The AI call” is really four separate decisions, and they don’t have to live together:

The key. Server only. No exceptions, including keys fetched at runtime from remote config, because a key the app can read is a key anyone with the app can read.
The model choice. Which model, with which settings: temperature, output limit, thinking budget.
The prompt. The instructions, the examples, the schema you ask for.
The parser. The code that turns the response into something your UI can draw.

The parser almost always stays in the app, because the app is what renders the result. The other three can move. Here is where ours sit today, honestly. ShotCanvas’s mobile app sends every AI request through our own signed-in proxy, which holds the Gemini key. The app lists the models it would like, newest first, but the first entry is Google’s gemini-flash-lite-latest alias rather than a pinned version, so when Gemini 3.5 Flash-Lite shipped on July 21, 2026, every installed copy started using it the same day with no release. The server can also route each feature (design, headlines, image work) to a different model on its own.

The prompt, though, is still assembled on the phone. That has a real cost: when we want to tighten the instructions for headline copy, the change waits for the next build like any other code change, and older installs keep the old wording until they update.

The case for moving the prompt to the server

You can fix it today. Bad output from a prompt is a bug users see right now, and the fix is usually a sentence. On the server it ships in minutes, to everyone at once, and rolls back just as fast if it makes things worse.

You can test it properly. With one live prompt per feature, you can run it against a saved set of inputs before deploying, and compare versions with real traffic. With five prompt versions spread across five app versions, nobody can say which one produced a complaint.

Your proxy stops being a free chatbot. This is the one people miss. If the app sends a complete request body and the proxy forwards it, any signed-in user can replace that body with anything they like and you pay for the answer. Authentication tells you who is calling, not what they’re asking. If the app instead sends “run the headline task with these inputs”, the server builds the prompt, and the only thing a caller can change is the inputs. Pair that with a cap on input length and per-user rate limits and the endpoint does exactly one job.

Secrecy is the weakest reason. Most prompts aren’t worth stealing, and the outputs reveal much of the approach anyway. Move it for speed and control, not to hide it.

The catch: the prompt and the parser are one contract

Moving the prompt splits a contract across two places that update on different schedules. The prompt decides the shape of the answer; the parser in the app expects that shape. Change the prompt to return a new field, rename one, or switch a number to a string, and every installed app that hasn’t updated starts failing to read the response.

The rule that keeps this safe is the same one that keeps any public API safe. The server may change wording freely: tone, examples, instructions, which model runs it. It may not change the output shape in a way old clients can’t read. In practice:

Have the app send a contract version with each task, such as headline@2.
Keep a prompt per contract version on the server, and leave old ones running until the app versions that use them are a negligible share of your traffic.
Make changes additive where you can. A new optional field is invisible to an old parser that ignores unknown keys; a removed field breaks it.
Enforce the shape with structured output, so wording changes can’t drift the format by accident. (A schema guarantees the shape, not the content, which is why the parser still validates.)

When the prompt can stay in the app

Some prompts have no server to move to. If you run an on-device model, such as the one Apple’s Foundation Models framework exposes, the prompt goes wherever the model goes. In that case treat prompt changes as app changes: test them against a fixed set of inputs before each release, and keep the tasks narrow enough that a release cycle is a tolerable fix time.

An early prototype is fine too. Move the prompt once the feature has users who would notice a bad answer, which is usually sooner than it feels.

One habit is worth keeping either way: separate the data from the instructions. If the app builds a prompt by gluing a dozen pieces of local state into a paragraph, start sending the state as named fields and let one function build the paragraph. That function is the thing you will later move to the server, and it’s much easier to move when it’s already one function.

How to move one without a rewrite

Do it a feature at a time. Pick the feature whose prompt you change most often, and:

Give the task a name and a version, and list the inputs it needs.
Add a server route that accepts only that name, version and those inputs, builds the prompt, calls the model, and returns the response unchanged.
Ship an app version that calls the new route. Leave the old path in place for installs that haven’t updated.
Once the old path’s traffic is gone, delete it from the proxy so arbitrary bodies are no longer accepted.

If you’re on Firebase, you may not need to write the route at all. Firebase AI Logic added server prompt templates in late 2025: the template (instructions, schema and model settings) lives on Firebase’s servers, and the app calls it by template ID with a map of input values. It’s the same design, with the template ID as the task name.

Four questions for any AI feature in a shipped app: if the output is wrong tonight, how long until every user gets the fix? If the model is retired, what does an old install do? If a signed-in user rewrites the request, what can they make your proxy pay for? If you change the prompt, which installed versions can still read the answer? A good setup has a short answer to each.

Keys belong on the server because they’re secret. Prompts belong there for a plainer reason: they’re the part of an AI feature you’ll edit most, and an app binary is the slowest place to edit anything.

Try AI screenshot design → When a model name breaks your app