How AI Works
$defineprompt-caching--plain-english
TLDRPay once to read it, reuse it cheaply.
Watch a kitchen during the rush. The ticket lands and the cook does not hunt for an onion. It is already diced. Stock is simmering. Sauces sit in little containers. That prep has a name: mise en place.
Prompt caching is the model's mise en place.
Every turn, the model re-reads its entire context window from scratch. Your long system prompt. The docs you pasted. The project rules. The stack you picked in the builder. Hit send again and it reads all of it again, top to bottom, like the first time. That is slow — and if you are paying per token through an API, it is expensive. You are re-chopping the same onion on every message.
Caching preps the parts that do not change and holds them ready.
The stable prefix gets processed once and set aside. Next turn, instead of re-reading all of it, the model grabs the prep that is already done and only chews on what is new — your latest question, the fresh error paste, the one line you added.
You feel this in two ways:
Replies come back faster. Less to re-read means less time staring at the thinking dots.
It is cheaper. Reused prep costs a fraction of fresh text — often roughly a tenth of the price for the cached chunk, depending on the provider.
The catch: the prep does not sit out forever. Go idle long enough and the kitchen tosses it. Come back later and the onion gets chopped fresh again, at full price. A fast back-and-forth in one session keeps reusing the bowl.
The lesson if you are building or prompting:
Put the big, stable material at the top and leave it alone.
Why does order matter? The prep only holds while nothing above it moves. Change one word near the top and everything below has to be re-chopped, because the prefix is not the same prefix anymore. Keep the steady stuff steady and you keep the discount.
Most hosts do this quietly. Cursor, API providers, agent harnesses — you rarely flip a switch. Everything you leave unchanged is prep you do not pay to redo.
