Most cost-optimization advice for LLM apps focuses on trimming tokens or shrinking context. But if you’re sending the same system prompt, document, or tool schema with every request, you’re paying full input price for content the model has already processed. Amazon Bedrock’s prompt caching attacks that specific waste: cache the static prefix, and subsequent requests read from cache at up to 90% lower input cost, with a faster time-to-first-token (TTFT).
The mechanics: cache points and TTLs
Prompt caching in Bedrock works through a cachePoint marker in the Converse API. Everything before that marker gets snapshotted after the first request. On a cache hit, the model skips reprocessing those tokens and starts generation from the cached state. On a miss, it processes the full content and writes to cache.
Four constraints shape how you use it:
- Scope: Cache entries are per AWS account and Region.
- Token thresholds: Each checkpoint needs a minimum token count to activate. Claude Sonnet 4.5 and 4.6 require at least 1,024 tokens; Opus models need 4,096.
- TTL: Entries expire based on the TTL you set. Default is 5 minutes; select models support up to 1 hour.
- Syntax: The
cachePointsyntax is identical across supported model families, including Anthropic Claude and Amazon Nova.
Pricing splits into three token categories: cacheWriteInputTokens cost 25% more than standard input (or 2x for 1-hour TTL), while cacheReadInputTokens cost 90% less. For a 10,000-token document queried with 10 different questions, the first request pays the write premium, and the next nine read at the discount—netting roughly 75% savings on input tokens, assuming all hits fall within the TTL window.
Where the savings actually show up
The AWS Machine Learning Blog walks through six scenarios, from basic to advanced. The first three cover the most common patterns:
- Message content caching: Put a
cachePointbetween a static document and the dynamic user question. This is the RAG pattern—same reference content, many queries. - System prompt caching: Cache persona definitions and instructions across conversations. If your assistant’s system prompt is stable, this is free money.
- Tool definition caching: Cache tool schemas for agentic workflows. Tool definitions rarely change between turns, so they’re a natural cache target.
More advanced scenarios include mixed TTL caching (assigning different lifetimes to different content tiers), tenant isolation for multi-tenant apps, and LangChain integration.
The implementation is straightforward. In Python with boto3, you structure content blocks like this:
content = [
{"text": "<static document content>"},
{"cachePoint": {"type": "default"}},
{"text": "<user question>"}
]The cache point sits between the stable prefix and the changing suffix. Everything above it gets cached; everything below it is processed normally each time.
The tradeoff you’re signing up for
Prompt caching isn’t free. The cache write costs 25% more than a standard input token, so you need repeated hits within the TTL to come out ahead. If your traffic pattern is one-shot queries with no shared prefix, caching will cost you more, not less.
TTL is the other lever. The default 5-minute window works for rapid-fire question sequences on the same document. The 1-hour option doubles the write cost but extends the hit window—useful for slower, session-based interactions. The AWS post notes that cross-Region inference profiles can occasionally increase cache write frequency, since requests may route to different Regions where the cache entry doesn’t exist.
This is the same infrastructure-level thinking behind model caching on HyperPod: move repeated computation to a cache layer, and the per-request cost drops. But prompt caching operates at the token level, not the model-weight level, so the integration point is your prompt structure, not your serving stack.
What to do next
Start by identifying the largest stable prefix in your requests. For most apps, that’s the system prompt or a long context document. Measure how often that prefix repeats within a 5-minute window. If the answer is “often,” add a cachePoint and watch the cacheReadInputTokens metric.
If your prefix is under the model’s minimum token threshold, caching won’t activate—you’ll pay the write premium with no read discount. The supplied AWS post uses Claude Sonnet 4.5 with a 1,024-token minimum, so a short system prompt won’t qualify. In that case, consider consolidating instructions or bundling tool definitions to cross the threshold.
Prompt caching is a structural optimization, not a prompt-engineering trick. It rewards you for knowing exactly which part of your request is static and which part changes. Get that split right, and the cost curve bends in your favor without touching model quality.
Sources
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.