# Inference (/academy/inference)



Training already happened. You were not in the room.

Someone spent weeks and a warehouse of chips teaching a model to predict the next [token](/academy/token). That bill is not on your card. **Inference** is the live pass: you send a prompt, GPUs walk those tokens, new tokens come back. Hit send in Cursor. Call an [API](/academy/api). Let an [agent](/academy/agent) take a step. That is inference. You pay for it. You wait for it. "The model is down" means nobody is running that pass.

This is why the words stream. You are watching tokens land one at a time, not a paragraph appearing from a drawer. A longer answer is a longer pass. A fatter prompt is a fatter pass. That is also why [latency](/academy/latency) feels like "thinking" even when the model is not doing extra [reasoning](/academy/reasoning) — it is just cooking tonight's plate.

Once you see the live pass, a few product choices stop being vibes:

* **Loops have a price.** "Call this a thousand times" is a thousand plates. Demo-night agents that retry forever are a surprise invoice.
* **[Prompt caching](/academy/prompt-caching)** is leftover prep. The stable prefix gets paid once, then reused.
* A [rate limit](/academy/rate-limit) is the kitchen capping how many tickets it will take this minute.
* Reasoning models spend extra inference on a private draft before the sentence you asked for. Same kitchen, longer ticket.

The [runtime](/academy/runtime) on your laptop is not the same bill. Local models still do inference — they just burn your fan instead of your API key.

**What this unlocks**

You budget the thing you actually buy. Not "AI" as a lump. Turns, tokens, retries, tool calls. A hackathon agent that looks smart in one chat can go broke in a loop. Design the loop like it has a meter.