Most of your AI bill is the part you send every time
The instructions at the top of your prompt do not change between calls. You are paying full price to send them again on every single request, and you probably do not have to.
A working prompt usually has two parts. A long stable part — the instructions, the rules, the examples of good output — and a short variable part, which is the actual request. The stable part is often ten times longer than the variable one.
Without caching you pay for every token of that stable part on every call. Run a job a hundred times a day and you have paid for the same instructions a hundred times.
What caching actually changes
Prompt caching lets the provider keep that stable prefix and reuse it. The first call pays a premium to write the cache. Every call after that reads it at a steep discount instead of paying the normal input rate.
The shape of the saving is what matters: it gets better the more often you call, and it only works if the stable part is genuinely identical and genuinely at the front. Move one character near the top and you have a different prefix and a fresh cache write.
Count the write, not just the read
The common mistake in cost reporting is to count cached reads and forget that writing the cache costs more than ordinary input, not less. If your first call of every cycle pays that premium and you never record it, your reported spend is lower than your invoice.
- Put everything that does not change at the very front, in a fixed order
- Keep the variable part — the actual request — at the end
- Record four numbers per call, not two: input, output, cache read, cache write
- Re-check after any prompt edit; a change near the top invalidates the cache for every later call
If you only track two numbers you cannot tell a cheap month from an expensive one until the invoice arrives. Four numbers per call is the difference between an estimate and a guess.
When it is not worth it
If you call a model a few times a day, caching saves you very little and adds a thing to get wrong. It earns its keep on volume and on long stable instructions. Measure before you restructure a prompt around it.