Prompt caching
Paying a reduced rate for a prefix the vendor has already processed, instead of full price for sending it again.
Also written: prompt cache, cached input, cache hit.
Because nothing persists between calls, an agent re-sends the same system prompt (Standing instructions sent ahead of every conversation, setting the role, rules and format.) and the same files on every single turn. Prompt caching lets the vendor keep the processed form of that prefix for a short while and charge you far less — often a tenth of the input rate — for reading it back.
The saving is real and large. The part that catches people out is that putting something into the cache is not free, and usually costs more than sending it normally would have.