The 90% discount hiding in your API bill.

Every serious LLM provider now offers a way to reuse tokens you've already paid to process, at a fraction of the price — up to 90% off, on both Anthropic and Google's numbers. Most indie developers calling these APIs have never turned it on. Here's what prompt caching actually does, why the saving is real and not a marketing ceiling, and the one structural rule that decides whether you get it.

The cost you're probably not thinking about

When people estimate what an AI feature will cost, they usually count the output — the answer coming back. In most real apps that's backwards. A chat app resends the entire conversation history on every turn. A coding agent resends the system prompt, the tool definitions, and a slab of file context on every single tool call. An app with a long, carefully-tuned system prompt pays for that prompt in full, every request, even though the text hasn't changed since the last one. Input tokens, not output tokens, are where the volume — and the waste — actually lives.

What prompt caching actually does

Prompt caching lets a provider remember the exact token sequence of a prompt prefix it processed recently, and charge you a fraction of the price to reuse it instead of running it through the model again from scratch. On Anthropic's API, a cache hit costs roughly 10% of the standard input rate — a model priced at $3 per million input tokens drops to about $0.30 per million on the cached portion. Gemini's context caching works the same way: cached reads run at roughly 10% of the base input price, so a $2-per-million model's cached tokens cost about $0.20. Both companies describe the saving as "up to 90%," and on a prompt where most of the tokens are unchanged boilerplate, that's close to the real number, not an upper bound you'll never hit.

Where the 90% actually shows up

The discount only applies to the part of a prompt that's byte-identical to something processed recently, so the shape of your app decides how much of it you actually collect. Three shapes benefit enormously:

• A long, static system prompt — instructions, style guide, few-shot examples — sent with every request. Pay full price once, then roughly 10% on every call after.
• Multi-turn agent loops where each step resends a growing transcript plus tool results. The chunk that was cached on the previous turn stays cheap on every turn after it.
• RAG or codebase-context apps that paste the same reference document into dozens of queries in a session — cache the document once, vary only the question.

A one-shot tool that sends a short, unique prompt every time — a translator, say, where the input text is different on every call — gets almost nothing from caching, because there's nothing repeated to hit.

The catch: caching isn't free, and it isn't permanent

Two costs work against the headline number. First, writing to the cache costs more than a normal input token, not less — Anthropic charges 25% above the standard rate for a 5-minute cache, or double for a 1-hour cache, because the provider is storing model state, not just re-reading text. You only come out ahead once that write gets reused more than once or twice.

Second, caches expire. Anthropic's options are a 5-minute or 1-hour time-to-live; on Gemini, explicit caches carry their own configurable TTL, while implicit caching — on by default for paid Gemini projects — runs on a much shorter, provider-managed window. Leave twenty minutes of silence between calls and a 5-minute cache is gone. You'll fall back to paying full price with nothing in the response telling you that happened.

The one rule that actually matters: order your prompt static-first

Caching works on prefixes. The provider walks the prompt from the beginning and finds the point where it stops matching what it's seen before — everything after that point is a cache miss, even if the divergence is one token. That means prompt structure, not prompt content, is what decides whether caching helps you at all.

Put system instructions, few-shot examples, and any large static reference material first. Put the part that changes every call — the user's actual message, this turn's tool output — last. A prompt built the other way around, with a timestamp or a session ID stitched into the opening line "for logging," silently breaks the cache on every single request, because nothing after that point can ever match again.

How this looks in a real app

If you're building anything with an LLM in the loop, the checklist is short:

1. Move anything static — system prompt, tool schemas, style guide, few-shot examples — to the front of the request, and don't let anything dynamic sneak in ahead of it.
2. Mark the cache breakpoint explicitly if your provider requires one (Anthropic's API needs a cache_control block; Gemini's implicit caching needs nothing, explicit caching needs a cache object).
3. Check your usage dashboard for a "cached input tokens" or "cache read" line item. If it reads zero, caching isn't happening — no error will flag that for you.
4. Batch calls inside a cache's TTL window when you control the timing: a burst of processing beats an all-day drip through the same cache.

Where this matters for a design tool

ShotCanvas's own AI features — the headline writer, the screenshot picker, the backdrop generator — call Gemini on narrow, single-purpose prompts rather than one long carrying conversation, partly for the reason we wrote about in why your AI keeps forgetting, and partly because a tight, consistent prompt is exactly the shape that benefits from caching in the first place. If you're shipping your own AI feature and haven't checked whether your input tokens are actually hitting the cache, that's worth five minutes before you optimize anything else.