Skip to main content
When consecutive requests share a long identical prefix — a system prompt, a reference document, a growing conversation — the provider can cache that prefix and charge a cache read rate for it instead of the full input rate. Reads are typically around a tenth of the input price. Two things decide whether it happens:
  • Most providers cache automatically. Send the same prefix twice and the second request is cheaper. Nothing to configure.
  • Anthropic caches only where you mark a breakpoint. Without one, a Claude model never caches, no matter how long or how repetitive the prompt is.

​
Inspecting cache usage

The OpenAI-shaped usage object does not report cached tokens, so a Chat Completions or Responses response looks identical whether or not the cache was hit. Read the request log instead:
Each row carries cached_input_tokens (read from the cache, billed at the cache-read rate) and cache_write_tokens (written to it, billed at the write rate). On /v1/messages the same numbers also appear inline as cache_read_input_tokens and cache_creation_input_tokens.
A cache write costs more than an uncached call — 1.25x the input rate on Anthropic. You come out ahead from the second request onward, not the first. A single probe run once will always look like caching made things worse.

​
Anthropic Claude

Caching is opt-in. Set cache_control and everything up to and including the marked point is cached.

​
Request-level breakpoint

Set cache_control once at the top level of the request. The gateway places the breakpoint on the last cacheable block and advances it as the conversation grows, so you never have to move it yourself:
This is the simplest option for an agentic loop, where each turn appends to the conversation and the cacheable prefix keeps growing.

​
Explicit breakpoints

For fine-grained control, put cache_control directly on a content block. Use this when you want to cache a stable prefix and deliberately leave a volatile tail uncached:
Anthropic allows at most four explicit breakpoints. An explicit breakpoint anywhere in the request takes precedence and the request-level field is ignored — so use one style or the other, not both. Both forms work on Chat Completions, the Responses API and Messages, streaming and non-streaming alike. cache_control is an extension on the OpenAI-shaped surfaces, which have no equivalent of their own; non-Anthropic routes ignore it.

​
Cache lifetime

A cache entry lives five minutes by default, and every read pushes that window out again. Add a ttl to keep it for an hour instead:
5m and 1h are the only accepted values; anything else is a 400, so a typo can never silently downgrade you to the default. The field goes wherever cache_control goes — request level or on a block — and works on all three surfaces. The longer window is not free: a one-hour write costs 2x the input rate against 1.25x for the five-minute default, so it needs three reads to pay for itself where 5m needs two. Reach for it when traffic is bursty with gaps longer than five minutes. Continuous traffic keeps the default cache warm on its own and should stay on 5m. Both rates are published per model by GET /v1/models, as cache_write_per_mtok_usd and cache_write_1h_per_mtok_usd.

​
What has to be true

  • Minimum length. Anthropic will not cache a short prefix. The floor depends on the model — roughly 1,024 tokens for Sonnet, higher for Opus and Haiku. Below it you get no cache and no error.
  • Byte-identical prefix. Any change before the breakpoint, including whitespace, starts a new cache entry.
  • Roughly a five-minute TTL, refreshed on each hit. A longer idle gap means paying the write again.
Some backends need a larger prefix than Anthropic’s own API before they return a hit. If a model caches on one route and not another with the same prompt, try a substantially longer prefix before concluding it does not cache.

​
OpenAI, xAI, DeepSeek, Google Gemini and others

Caching is automatic and needs no configuration — repeat the prefix and the provider handles it. cache_control is accepted and ignored on these routes, so a single request body can target both an Anthropic and a non-Anthropic model. Not every model that publishes a cache-read price delivers a hit on every request; some providers cache opportunistically. Check cached_input_tokens in the request log rather than assuming.

​
Billing

You are billed the gateway’s published rates for the model, whichever backend serves the request:
  • Cache reads at the model’s cached_input_per_mtok_usd.
  • Cache writes at cache_write_per_mtok_usd where the provider charges a premium (Anthropic: 1.25x input). Where no write rate is published, writes bill at the normal input rate.
  • Uncached tokens in the same request bill at the standard input rate.
Both rates are published per model on /v1/models.