Summary: On October 31, 2026, Pydantic announced on X that changing a dynamic system prompt no longer has to rewrite the start of your conversation. Pydantic AI can append the change to message history instead, keeping the earlier prompt prefix available for provider-side caching. For anyone running agents with per-user or per-request instructions, that’s the difference between a cache hit and paying full input price on every turn.
The problem: dynamic prompts break prefix caching
Most agent frameworks put the system prompt at the very front of the request. Provider prompt caches, like Anthropic’s, work on exact prefix matches — the model reads tokens left to right, and caching discounts only apply to the longest unchanged opening block. So the moment you inject something dynamic into the system prompt (today’s date, the user’s account tier, a retrieved doc), the entire prefix changes and the cache misses.
For long conversations that’s expensive. Instructions, tool definitions, and the whole prior history get re-billed at full input price every single turn, purely because one line near the top moved.
What Pydantic AI does instead
According to Pydantic’s announcement on X, the framework can now append a changed dynamic system prompt to the conversation history rather than rewriting the beginning. The old prompt stays where it is, the new instructions arrive as a later message, and the provider’s cache still matches on the stable prefix.
It’s a small structural trick with a familiar logic: treat instructions like append-only event log entries instead of a mutable header. The tradeoff is that your context grows slightly and the model sees stale instructions unless the appended change explicitly supersedes them — worth checking that your prompts are written to tolerate that.
Caching is opt-in
The Pydantic AI agents documentation adds an important caveat: several providers, including Anthropic, cache nothing unless the request explicitly asks for it. You enable this by passing the Caching capability to your agent — for example capabilities=[Caching()] — and the docs recommend deciding whether to cache the full request or only the stable prefix of instructions, tools, and conversation.
Without that capability, every request pays full price for the instructions, tools, and history regardless of whether anything changed. That makes this update a two-part fix: the append-on-change behavior keeps the prefix stable, and the Caching capability actually tells the provider to use it.
Why this matters for agent builders
If your agents carry per-request context — user preferences, tenant config, live data — you’ve probably faced a blunt choice: freeze the system prompt for cache efficiency, or personalize it and eat the cost. Pydantic AI’s approach removes the choice by making personalization cache-compatible.
The same prefix-stability logic shows up elsewhere in the stack. We covered how prompt caching savings actually accrue on Amazon Bedrock, and the principle is identical: cache hits are won or lost in the first few hundred tokens of a request. Frameworks that respect this — by structuring prompts so stable content leads and dynamic content trails — turn caching from a nice-to-have into a default cost posture.
One limitation to keep in mind: appending changes grows your context over a long session, and context growth has its own token cost. For short conversations the rewrite penalty may be trivial. As always, measure against your actual session lengths rather than assuming the append strategy wins everywhere.
The practical next step if you’re on Pydantic AI: turn on the Caching capability, review which parts of your system prompt are genuinely dynamic, and restructure them so stable instructions lead and per-request details arrive as appended updates.
Sources
- Pydantic on X: Changing a dynamic system prompt no longer has to rewrite the start of your conv
- Agents
View the original post on X