Long-running agents make many API calls that carry forward the same instructions, tool definitions, and context. OpenAI says GPT-6 caches that shared context to reuse computation, cutting response times and giving discounts of up to 90% on cached input tokens. The awkward part has always been the misses: when a prefix stops being reused, you often find out from the bill rather than from a clear signal.
On 2026-09-22, OpenAI described an improved caching system for the GPT-6 family, plus tooling aimed squarely at that gap. The supplied material covers three things: a dashboard, a diagnostics tool, and controls for choosing what gets cached.
What the dashboard and diagnostics actually tell you
The Prompt Caching Dashboard shows how much of an application’s input is served from cache. You can track hit rates over time and use an input composition chart to compare cached and uncached tokens. That is the monitoring layer: spot a drop, then check whether a change to your app caused it.
For a specific miss, the prompt caching diagnostics tool compares a request against a recent response to identify what prevented reuse — model, tools, settings, or input. It returns an estimated count of affected tokens. The example in the source shows a tools_changed reason with 5,629 reusable tokens missed:
{
"prompt_cache_diagnostics": {
"type": "cache_miss",
"reason": "tools_changed",
"comparison_reusable_tokens": 5629,
"cache_missed_tokens": 5629
}
}That number is the useful part. It turns a vague “caching got worse” into a size estimate you can weigh against the cost of restructuring your request.
The controls that keep a prefix reusable
OpenAI lists several optional controls that sit on top of default caching behavior. Explicit cache breakpoints let you choose which prompt prefixes to reuse. The refreshed prompt caching guide covers how long cached prefixes stay eligible and how tool and input changes affect reuse.
Two details matter for agent design. First, on GPT-6 models you can change reasoning effort between responses without breaking cache, by appending a configuration_update while leaving request-level reasoning effort unchanged. Second, when tool needs change, keep definitions, schemas, and ordering stable. Use allowed_tools to limit which tools are callable, or set tool_choice to none, rather than removing definitions. New instructions go in new developer messages appended toward the end of context.
There is also prewarming: preparing known context ahead of time so the model can start sooner when a request arrives. Shared instructions or reference material can be prepared during startup, moving that processing out of the user’s wait.
Why this is a product decision, not just an API setting
Cache eligibility now has a 30-minute window for eligible shared prefixes. That is a design constraint. If your agent pauses longer than that between turns, or rebuilds its tool list per request, the discount quietly disappears.
The practical move is to treat cache hit rate as a metric you watch the way you watch latency. This is the same discipline behind tiering models and caching context deliberately, which we covered in GPT-6 Sol and Luna on Bedrock: pick the right model, then make the context reusable. The new diagnostics close the loop by naming the reason a prefix stopped being reused.
One limitation: the supplied material does not specify pricing beyond the up-to-90% cached-input discount, nor how the dashboard behaves across multiple projects. If your agent’s tool set changes often, start by measuring miss size before rewriting your prompt structure — the diagnostics output is the cheapest way to find out whether the problem is worth fixing.
To get started, OpenAI points to the dashboard, the diagnostics tool, and the caching guide, and notes that Codex can review your code, apply improvements, and measure results.
Sources
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.