OpenAI Announces Improved Prompt Caching for GPT-6

OpenAI announced improved prompt caching for the GPT-6 family on September 22. The feature reuses already-computed state when several requests share the same opening portion, and the update raises default cache hit rates and sets a 30-minute window within which reused prefixes earn a cache discount. A dashboard for watching cache performance and a diagnostics tool that explains why the cache missed arrived at the same time.

The backdrop is agents that run for hours. According to OpenAI, GPT-6 enables agents that keep working for hours on long tasks such as refactoring codebases or producing research documents and presentations, and the applications behind them send a chain of API requests that carry forward the instructions, tool definitions and context from earlier turns. When that shared opening portion hits the cache, responses come back faster and cached input tokens get a discount of up to 90%. METAL previously reported that OpenAI previewed the cache discount and new caching tools when it released GPT-6 Sol and Luna the same day, and this announcement is the follow-up that lays out, alongside developer documentation, how the machinery actually works.

What the cache stores is not text. According to the developer documentation, the model produces intermediate values called key-value states as it processes input tokens, and prompt caching keeps those states for the unchanged opening portion of a prompt so the next request can reuse them. OpenAI's hidden system instructions, tool definitions and schemas, developer messages and conversation history stack up in that order, and if anything changes near the front, everything after it has to be computed again.

The rules changed starting with GPT-5.6. The minimum cacheable length is 1,024 visible input tokens, and a new explicit mode lets developers place cache breakpoints themselves. On lookup, the system walks back from the longest prefix through the first 2 and latest 50 explicit breakpoints, and implicit mode adds the end of the latest user message and up to 20 earlier message endings. GPT-5.5 and GPT-5.5 Pro placed boundaries at 2,048-token intervals and rounded reported cached tokens down to a multiple of 128, while the new approach reports the exact boundary.

The billing structure changed too. On GPT-5.6 and later models, cache reads are charged at 0.1 times the uncached input rate and cache writes at 1.25 times, and a cache entry lives at least 30 minutes after its latest write or reuse. If a prefix written to the cache is not read again within 30 minutes, the developer effectively pays 25% more than without caching, which makes the hit rate the cost.

The cache lives on individual machines. According to the documentation, traffic above 15 requests per minute can trigger overflow routing to other machines, in which case a request may not reach the machine holding the cache. Caches are not shared across organizations and are not reused across regional processing boundaries.

The new tools make that hit rate visible. The Prompt Caching Dashboard shows over time how much of an application's input is served from cache, and an input composition chart separates cached from uncached tokens. When an unexpected miss occurs, the diagnostics tool compares the current request with a recent response to pinpoint whether the model, tools, settings or input changed, and estimates how many tokens were affected.

Diagnostics return nine reasons: model changed, cache key changed, service tier changed, tools changed, output format changed, reasoning effort changed, verbosity changed, context compacted and input changed. In the documentation's example, simply renaming a function tool from get_time to get_date caused 5,629 tokens to miss the cache entirely. Diagnostics carry no extra cost and do not count separately toward rate limits, and because they keep only configuration metadata, token estimates and hashes for a short period instead of raw prompts, they work with Zero Data Retention. They report only the first classified reason at a time, so a developer has to fix it and compare again to see the next cause.

Optimization options fall into four groups. Explicit breakpoints choose how much to cache, and on GPT-6 models developers can raise or lower reasoning effort between responses without breaking the cache by appending a configuration_update while leaving request-level reasoning effort unchanged. When tools are not needed, developers narrow callable tools with allowed_tools or set tool_choice to none instead of deleting definitions, and they append new instructions as developer messages at the end of the context. Prewarming, which processes shared instructions, tool definitions and reference material before the user asks the first question, is also included.

METAL confirmed that the reason table in the diagnostics documentation still lists a reasoning effort change as a cause of cache misses. The promise of changing reasoning effort freely applies only to the new path of appending a configuration_update without touching the request settings, and changing the request's reasoning effort value itself still forces the prefix to be recomputed.

Customers shared numbers as well. Mario Rodriguez, Chief Product Officer at GitHub, said "over the past several months, we've reduced by more than 50% the share of prompt tokens requiring fresh processing across billions of requests to OpenAI models, relative to our previous baseline." Arian Hanifi, Chief Technology Officer of Strawberry, which builds an agentic browser, said "OpenAI's prompt caching diagnostics and dashboard helped us improve cache hit rates by a few percentage points, reducing costs by 20%." He added that his team gets alerts when caching breaks unexpectedly and uses Codex agents to find the root cause, and that caching stable context with explicit breakpoints while keeping frequently changing content at the end of the prompt has made it economically viable to fork conversations for background tasks.

Seen through the eyes of agent engineering, this announcement is less a discount feature than a design discipline. In an hours-long task, most of the input is a prefix that has already been computed once, and whether that prefix is preserved or broken decides both latency and the bill. OpenAI has started returning reason codes and token counts for the fact that small habits, such as putting a timestamp at the top of the instructions or reordering tools, force thousands of tokens to be recomputed. The first job for teams building agent applications is not switching models but writing down what changes in their own prompts on every request.