Cut your API bill 75%: a prompt-caching tutorial

If you're paying full input price on every API call, you're probably overpaying. Most production prompts resend the same material over and over — system instructions, tool definitions, retrieved documents, conversation history — and every major provider now sells you a way to stop paying full price for it. That way is prompt caching: a few lines of code, zero quality change, and 75–90% off the repeated portion of your bill.

The headline number behind this tutorial: on Anthropic's Claude Fable 5.1, cache reads now cost $0.25 per million tokens against a $10/M base input price — a 97.5% discount on any token served from cache. Across all three big APIs, cache reads run at roughly 10% of base input pricing. Here's how to capture it.

What prompt caching actually does#

Prompt caching stores the internal representation (the KV cache) of a stable prompt prefix so repeat requests skip recomputing it. Two effects: cost — repeated prefix tokens are billed at the cache-read rate (~10% of input price); and latency — time-to-first-token drops because the prefix isn't recomputed.

The catch: caching only covers stable prefixes — content that is byte-identical across requests and sits at the start of the prompt. A change mid-prefix invalidates everything after it. That single fact dictates all strategy below.

The pricing math that matters#

A few things to note. These numbers move — check the vendor pricing pages before deploying. Writes are not free on Anthropic: the first cache write costs 1.25× the base input rate, so you need roughly 3+ reads before you're in the black on a 5-minute TTL. OpenAI is the outlier in your favor: caching is automatic and writes are free on most current models, so every hit is pure savings.

A worked example#

Take an agent workload: 10,000 requests/day, each carrying a 5,000-token system prompt + tools + context, a 500-token user message, and a 1,000-token response, with an 80% cache hit rate. On Claude Sonnet 4.6 ($3/M input, $15/M output):

• Cache reads: 40M tokens × $0.30/M = $12.00
• Cache writes: 10M tokens × $3.75/M = $37.50
• Uncached input + output: $15 + $150 = $165.00
• Total: $214.50/day

Without caching, the same workload costs $315/day. That's 32% off the total bill — and the savings scale with how input-heavy your workload is. Anthropic reports up to ~45% effective savings on heavily agentic Fable 5.1 runs, entirely from the cache-read line item.

Anthropic: explicit breakpoints#

Anthropic gives you the most control — and demands the most from you. You place cache_control markers on the content blocks you want cached, and caching only happens at those breakpoints. There are two modes:

• Automatic caching (simplest): add one top-level cache_control field; the breakpoint lands on the last cacheable block and moves forward as conversations grow.
• Explicit breakpoints: up to 4 markers, for caching sections that change at different frequencies (e.g., rarely-changing tool definitions vs. daily-refreshed context).

import anthropic
client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    # Option A: automatic caching — one line, breakpoint follows the conversation
    cache_control={"type": "ephemeral"},
    system="You are a support agent for Acme Corp. [long static instructions...]",
    messages=[{"role": "user", "content": "Where is my order?"}],
)

For explicit control over multiple stable sections:

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    system=[
        {"type": "text", "text": STATIC_SYSTEM_PROMPT,
         "cache_control": {"type": "ephemeral"}},          # breakpoint 1
    ],
    tools=[TOOLS_WITH_BREAKPOINT],                          # breakpoint 2
    messages=[
        {"role": "user", "content": [
            {"type": "text", "text": STATIC_KB_CONTEXT,
             "cache_control": {"type": "ephemeral"}},       # breakpoint 3
            {"type": "text", "text": user_question},        # changes per request
        ]}
    ],
)

Rules that matter: put static content first and dynamic content (timestamps, user messages) after your last breakpoint, since Anthropic's hierarchy (tools → system → messages) invalidates everything after a change. Prefixes below the model's minimum (1,024 tokens for Sonnet 4.6, up to 4,096 for Haiku 4.5) are silently not cached — check cache_creation_input_tokens and cache_read_input_tokens in the response usage: if both are 0, nothing was cached. And watch the TTL math: a 5-minute TTL with one request every 10 minutes means every request pays the 1.25× write premium and never reads.

OpenAI: it's already on#

OpenAI's approach is the opposite philosophy: caching is automatic for any prompt prefix of ≥1,024 tokens that is byte-identical to a recent request. No markers, no code changes — you just keep the prefix stable:

from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
    model="gpt-5-5",
    messages=[
        {"role": "system", "content": STATIC_SYSTEM_PROMPT},   # identical every call → auto-cached
        {"role": "system", "content": TOOL_DEFINITIONS_JSON},   # also static
        {"role": "user", "content": user_message},              # the only thing that changes
    ],
)
usage = response.usage
print(usage.prompt_tokens_details.cached_tokens)  # tokens served from cache

What to know: cache hits arrive in 128-token increments and show up in usage.prompt_tokens_details.cached_tokens — monitor it to verify your hit rate. The discount varies by model: current GPT-5.x rate cards price cached input around 0.1× (e.g., $0.50/M cached vs. $5/M input on GPT-5.5), while some older models were 0.5×. And prefix stability is the whole game: if cached_tokens reads 0 despite a static system prompt, something upstream is mutating the prefix — non-deterministic serialization, per-request content ahead of the static block — so fix ordering first.

Google Gemini: explicit caches + free implicit ones#

Gemini has two tiers, and they're easy to confuse:

• Implicit caching (automatic on Gemini 2.5+): if your prompt matches a recent request, you get the discounted read rate automatically. Same per-token discount as explicit, no storage cost — but no savings guarantee.
• Explicit context caching: you create a named CachedContent object holding your static content, then reference it by ID. Guaranteed, but you pay storage (~$0.50/M tokens/hour through 2026, rising toward $1.00) plus the discounted read rate.

from google import genai
from google.genai import types
client = genai.Client()

# Create the cache once for your static content
cache = client.caches.create(
    model="gemini-3-7-flash",
    config=types.CreateCachedContentConfig(
        contents=[STATIC_REFERENCE_DOC],
        system_instruction=STATIC_SYSTEM_PROMPT,
        ttl="3600s",
    ),
)

# Reference it on every request
response = client.models.generate_content(
    model="gemini-3-7-flash",
    contents=user_message,
    config=types.GenerateContentConfig(cached_content=cache.name),
)
print(response.usage_metadata.cached_content_token_count)

What to know: a request referencing a cache cannot carry its own system_instruction, tools, or tool_config — those must live in the CachedContent object. The cache's model and the request's model must match, and minimums are roughly 1,024 tokens (Flash) / 4,096 (Pro). Explicit caching only pays off when storage is dwarfed by read savings — it's built for large hot contexts; for small prefixes, implicit caching is the better deal.

The five pitfalls that kill your savings#

1. Varying content before static content. A timestamp, request ID, or the user's message placed above your static blocks invalidates the prefix. Always: static first, dynamic last.
2. Non-deterministic serialization. If your tool definitions serialize with random key order or your few-shot examples shuffle, the prefix is never byte-identical and the cache never hits. Freeze them as constants.
3. TTL expiry on sparse traffic. On Anthropic, a 5-minute TTL with one request every 10 minutes means every request pays the 1.25× write premium and never reads. Match TTL to your arrival rate — or don't cache at all.
4. Parallel fan-out. A cache entry only exists after the first response begins. N workers firing the same prefix in parallel all miss; only later waves benefit. Warm the cache with one request first, or stagger.
5. Caching tiny prompts. Below the minimum token threshold, cache_control markers are silently ignored. A 300-token system prompt gets you nothing — pad the cached content or skip it.

Takeaway#

Prompt caching is the rare optimization with no downside to quality: identical outputs, lower latency, lower cost. The implementation differs — explicit breakpoints on Anthropic, automatic on OpenAI, explicit cache objects on Gemini — but the discipline is identical: identify what's static, put it first, keep it byte-stable, and monitor your hit rate.

Start today: on OpenAI, you may already be saving money — check cached_tokens in your usage. On Anthropic, add one top-level cache_control field to your highest-volume multi-turn endpoint. On Gemini, let implicit caching work and reach for explicit caches only for large, hot contexts. Then look at next month's bill — for most teams, this single change beats months of model-switching and prompt-trimming.

Pricing figures are per vendor documentation as of September 2026 and drift over time — verify current rates on the Anthropic, OpenAI, and Google pricing pages before deploying.