The Hidden Cache You’re Paying For But Not Using
This title could be clearer and more informative.Sign up to try Clickbait Shieldfor free.
Prompt caching stores the computed key/value attention state for a prefix of your input and bills re-reads at roughly one-tenth the normal input price. Agent loops are the ideal workload because they re-send a growing transcript every turn, making costs quadratic without caching. The correct pattern uses two cache_control breakpoints: one static on tools + system, one rolling on the last message. Anthropic requires explicit breakpoints while OpenRouter caches OpenAI/DeepSeek/Gemini automatically but still requires explicit markers for Anthropic-backed routes. The failure mode is silent — a plain string in the system field or a client that only reads prompt_tokens means you pay full price while believing you're optimized. A worked cost model shows a ~78% reduction in input costs for a 12-turn agent run. Key observability advice: instrument cache_read_input_tokens, keep your prefix byte-stable, and verify the number increases on turn two.
Table of contents
Anthropic: explicit breakpoints and a strict hierarchyThe prefix hierarchyThe rules that bite youPricing: the whole pointHow you know it workedWhat invalidates a cacheShare this post