In-depth analysis · About 6 min read
AWS explains prompt caching for Bedrock agents: stable prefixes and separate cost examples
The AWS guide combines explicit shared prefixes with implicit growing history, while distinguishing tool changes, helper reuse, modeled savings and endpoint monitoring.
- Published here
- Source published
- Source checked
Claudio Mazzoni’s October 26 AWS engineering guide explains how agents using OpenAI models on Amazon Bedrock can reuse prompt computation. It is a workflow tutorial, not a new model announcement. Our analysis follows the complete article and its cost charts; AISubHub has not run its code or independently measured cost, latency or model availability.
Listen to the news
Read this article aloud or keep listening to the news.
Preparing the reader…
Device voice
Speech is handled by your browser or device. AISubHub does not send the text to an audio API. Voice availability and playback with a locked screen vary by device.
Resuming may repeat a short phrase.
At a glance
- Cacheable prefixes need at least 1,024 tokens. AWS describes a minimum 30-minute lifetime refreshed by successful reuse, and cache writes billed at 1.25 times normal input rates.
- Figure 4 reports relative input costs of 0.38 for one implicit setup and 0.39 for rolling breakpoints, versus 1.00 without caching; these workload results are not universal savings.
- A separate modeled six-turn example starts with 8,000 prefix tokens and adds 2,000 tokens per turn: $0.87 without caching, $0.50 with static caching and $0.32 with rolling breakpoints.
1. Reuse computation, not an agent’s permanent memory
An agent normally resends its tools, instructions and earlier exchanges on each request. During prefill, the model builds key/value state from those input tokens. AWS’s mechanism reuses a saved matching prefix, then processes the new suffix. It requires the same input tokens; consistent tool serialization and ordering matter, while timestamps, names and request identifiers belong after the reusable prefix.
Explicit markers identify a shared endpoint after tools and instructions. Implicit mode follows eligible conversation endings; both can operate together. The prefix must reach 1,024 tokens. The examples use store=False, which leaves conversation history with the application and does not disable prompt caching. This computation optimization is not persistent agent memory or permission to omit the history the request still needs.
2. Preserve earlier boundaries as the conversation grows
The loop appends model output and matching tool results, including encrypted reasoning, while preserving earlier messages. In explicit-only mode, the newest result batch gains a marker and earlier markers stay. A static marker alone reuses startup context but repeatedly processes later history; moving a single marker to the newest turn can also lose useful reuse. An implicit endpoint through one task does not necessarily save a shorter shared endpoint for a different task.
Figure 4 concerns a six-turn GPT-6 Astra tool loop with roughly 2,000 tokens of tool output per turn. Relative input cost is 0.38 for the shown implicit configuration, 0.39 for rolling breakpoints, 0.61 for static-only caching and 0.72 for newest-turn-only marking, against 1.00 uncached. The guide’s roughly 62% and 61% reductions belong to these configurations. They must not be substituted for the different modeled example in Figure 9.
3. Change callable tools without rewriting the initial catalog
For known tools, AWS keeps the same sorted TOOLS definitions and changes allowed_tools through tool_choice. Its auto mode permits a decision about calling an allowed tool; required demands at least one allowed call. This changes call selection without rebuilding the cached catalog. AISubHub’s analysis treats this as prompt organization, not proof that application authorization or tool execution has been secured.
For tools added mid-session, the guide says both Bedrock Runtime and Mantle accept additional_tools for GPT-6 Astra, GPT-6.1 Sol and GPT-5.6 Luna. Append the schema to history and keep the original catalog unchanged; retain that item on later requests. The additional_tools item cannot itself hold a breakpoint: explicit-only examples add one to a following input_text block and preserve older markers. Different agent roles should start with separate stable catalogs. This is not a claim of availability for every model or Region.
4. Warm shared context before helpers, within the same reuse scope
The helper pattern sends an ordinary warm-up request and waits for completion before parallel fan-out. Both warm-up and helpers mark the shared prefix, even with implicit mode, and use the same tools, model and key. Usage counters verify cache activity first. A summarizer can copy the completed parent history and append its own instruction without changing the parent; that appended instruction does not automatically create a new cached endpoint.
Reuse requires the same model, AWS account, Region, key and matching prefix, with the entry still available. A session-specific key separates otherwise matching sessions; a stable group key enables sharing. AWS states at least 30 minutes after the latest write or successful reuse, potentially longer. A refresh request itself costs money. Keys and lifetime should therefore follow intended sharing boundaries, rather than promising permanent retention or treating accounting separation as a security guarantee.
5. Separate modeled input savings from a production bill
Figure 9 models a GPT-6 Astra support agent with an 8,000-token startup prefix and six turns adding 2,000 tokens each. Its cumulative input cost reaches $0.87 without caching, $0.50 with a static breakpoint and $0.32 with rolling breakpoints, using the guide’s on-demand price assumptions. The stated 64% modeled reduction belongs to this example, not Figure 4 or an independently tested deployment.
Writes cost 1.25 times the normal input rate; read rates depend on the model. A short or frequently changing workload may not amortize writes and warm-up requests. The author recommends tracking read savings alongside write expense. Output tokens must be excluded when comparing shares of input cost, and separately included in the overall bill. AISubHub does not present these historical illustrative amounts as a current quote or a guaranteed reduction in total spending.
6. Verify the right counters and retain governance boundaries
Response usage exposes cached_tokens and cache_write_tokens. Bedrock Runtime also provides CloudWatch CacheReadInputTokenCount and CacheWriteInputTokenCount; invocation logging can support request-level inspection. The guide says Mantle’s CloudWatch namespace currently lacks separate cache counters, so its response usage should be recorded by the application. Cost Explorer can group Bedrock entries by Usage type to separate reads, writes and uncached input.
The author also warns that tool definitions and system prompts can be leaked or exploited and calls for governance covering caching. Our interpretation is that reuse scope, permissions and monitoring require their own review; the article does not establish a specific breach or a complete leakage defense. We have not independently read the linked documentation or executed the examples. Teams still need to verify their model, endpoint and Region and measure their own costs and latency.
Why it matters
The useful design distinction is between a stable shared prefix, an append-only conversation and intentional sharing among helpers. Savings depend on preserving those boundaries and observing real reads and writes, rather than enabling a cache option alone.
Limits and availability
This analysis is based on the complete official guide and its necessary charts. Test results and modeled costs remain attributed to AWS and are different evidence types. No code, supplementary documentation, current pricing or deployment was independently tested here.
Sources and evidence
Source published: · Source checked:
AI-assisted editorial articles checked against linked primary sources. Vendor claims are attributed; inclusion is not endorsement.