
PAU BARRENA/AFP via Getty Images
Amazon Web Services published a developer guide Monday at 10:42 a.m. ET showing that explicit prompt caching on Amazon Bedrock's open-weight models stores nothing unless developers set two request fields together, and the guide names no model that supports the feature. The AWS Machine Learning Blog guide leaves that to model cards. AWS documentation lists explicit caching for Moonshot AI's Kimi K3 and Kimi K3.1 Pro and for Z.ai's GLM 5.3 model card, and both labs are based in Beijing, so the cache settings are only part of the buying decision.
The guide is a how-to, not a launch. Kimi K3 became the first open-weight model on Bedrock to support explicit caching when AWS announced Kimi K3 September 18, and Monday's post tells developers how to use the feature with open-weight models while using a placeholder model ID in every sample and reporting no measured savings. The model cards, the pricing page and Bedrock's documentation fill some of those gaps, as covered below.
Developers turn on explicit caching in two steps. First, they set a Bedrock-specific field named prompt_cache_options to explicit mode, which the OpenAI Python SDK passes through its extra_body parameter. Second, they attach a prompt_cache_breakpoint to a text content part, and everything up to and including that part becomes eligible for caching.
Skipping either step changes the outcome. Explicit mode without a breakpoint caches nothing, while a breakpoint without explicit mode behaves like implicit caching. Developers can place several breakpoints, up to a per-model maximum, to cache layers that change at different rates, and an optional ttl field sets entry lifetime. Content after the last breakpoint is billed at the standard input rate and never written to cache.
Implicit mode needs no request changes but is best-effort, and AWS says it can write content that later requests never read. In the authors' test, a prompt that put the product manual and the customer question in one user message got no implicit hits, so the samples keep static material in a system message and the question in a separate user message.
Caching targets the input-processing stage of inference, not output generation. On the first request the model processes the marked prefix and stores the internal state associated with it, and a later request with a matching prefix skips that recomputation, according to AWS's guide to effective prompt caching. The match must be exact, so static content such as instructions and tool definitions goes first and the changing question goes last.
A conversation or coding-agent loop produces this pattern naturally, because each request's prefix is the previous request. AWS says implicit mode suits those stable-prefix loops, while explicit mode suits a fixed shared prefix followed by a changing question.
Two engineering constraints shape real hit rates. Bedrock's documentation says cache entries live on one server, so a request served by a different server, such as one in another Region, or sent before an identical request finishes, writes a new entry instead of reading one. The same documentation says minimum prefix sizes run from 512 to 4,096 tokens depending on the model, and a checkpoint below the minimum succeeds without caching anything. A basic caching principle applies: benefit depends on hit ratio, not on the feature being switched on.
For Kimi K3, AWS's launch post specified a breakpoint after a reusable prefix of at least 1,024 tokens, with cached tokens kept at least 30 minutes, and said cache-hit tokens skip quota counts for input tokens per minute. The Kimi K3 model card now lists a 256-token minimum instead, so the card, not the launch post, is the figure to check. LetsDataScience reported the launch-post figures of 1,024 tokens and 30 minutes.
The GLM 5.3 model card lists explicit caching on the Responses and Chat Completions APIs with a 1,024-token minimum and at least 30 minutes of retention, and it recommends explicit caching over the default implicit mode. The card does not state a breakpoint maximum, supported ttl values or cache-write pricing. A third model, Kimi K3.1 Pro on Bedrock, is documented with a 256-token minimum.
AWS's launch post describes explicit caching as a Kimi K3 feature, and it says Bedrock has added dozens of open-weight models since 2025 from DeepSeek, Google, MiniMax, Mistral AI, Moonshot AI, NVIDIA, OpenAI and Qwen. Monday's guide does not say which of those models support explicit caching and sends readers to each model card for modes, minimum prefix lengths and TTL values. Bedrock has offered caching for proprietary models since a December 2024 limited preview for select customers on Claude 3.5 and Nova models.
Monday's guide gives no rates and says cache reads cost less than standard input while writes can cost more, depending on the model. AWS's pricing page lists rates for the models it labels "Kimi 3," GLM 5.3 and "Kimi K3.1 Pro - Highspeed." On the global cross-Region profile, standard-tier Kimi 3 input costs $3.00 per million tokens, a cache read $0.30 and a 30-minute cache write $3.75. GLM 5.3 lists $1.68, $0.312 and $2.10. Kimi K3.1 Pro Highspeed lists $4.00, $0.20 and $5.00. The US profile adds about 10 percent, with Kimi 3 at $3.30, $0.33 and $4.125.
Applying those rates, a Kimi 3 read costs 10 percent of the input rate and a write costs 125 percent. One write plus one read costs $4.05 per million prefix tokens against $6.00 for two uncached sends, a 32.5 percent saving, and ten sends cost $6.45 against $30. For GLM 5.3, a read costs about 18.6 percent of input and a write 125 percent, so one write plus one read costs $2.412 against $3.36, a 28.2 percent saving. In both cases a prefix written and never reread costs 25 percent more than not caching. Applied to the roughly 4,000-token manual in AWS's sample, three questions would cost about $0.017 with explicit caching on Kimi 3 versus $0.036 uncached, and about $0.011 versus $0.020 on GLM 5.3. AWS names no model in that sample, so the figures are an illustration, not an AWS result.
The structure matches OpenAI's GPT-5.6 on Bedrock, where Bedrock documentation puts cache writes at 1.25 times the uncached input rate and an AWS news aggregator relayed a 90 percent discount on cached input. Rates differ by model, so one model's rates do not transfer to another.
Bedrock's documentation says support for caching does not guarantee a cache hit on any request. AI engineer Haris Farooq published measurements of one failure mode on Oct. 6 using Claude Sonnet 4.6 on Bedrock in us-west-2: a cache hit at 21 content blocks added after a checkpoint and a miss at 22, every time. The request succeeds with no error, and the cached text is billed at full input price plus the write premium. Farooq also found silent resets from changing thinking or effort settings, switching tool choice, and placing dynamic content before the cache point. That test did not cover open-weight models, but it shows why AWS tells developers to confirm hits in the usage fields and to watch the CacheReadInputTokenCount and CacheWriteInputTokenCount runtime metrics.
Cross-Region routing adds a second cost wrinkle. AWS says cross-Region inference can produce extra cache writes during high demand, and the GLM 5.3 card lists only US-geography and global cross-Region profiles, with in-Region on-demand inference not supported. Bedrock's documentation also says caching does not work with the batch inference API.
AWS's coding-agent demo uses OpenCode, a free coding agent from the Anomaly team, configured to reach Bedrock through an OpenAI-compatible provider. The three-turn session covers a bug review, a proposed fix and a one-sentence question about a low-stock method in a small Python repository. Each turn resends the earlier context plus tool calls and results, and Bedrock reads the earlier request from cache and processes only the new tail, with no caching configuration.
The agent's reporting has a blind spot. OpenCode 1.18.35 does not record cache-write tokens for OpenAI-compatible providers, so its cache-write counter stays at zero even when writes occur. AWS labels its figures illustrative and gives no benchmark numbers, saying only that savings grow with the amount of reused context and how often it is reused.
Caching changes the input bill and input latency, not answer quality, because it optimizes only the input-processing stage. The independent evidence on the two original models is thin and mixed. On the First Proof benchmark of unpublished research-level math problems, TechTimes reported Humanize's Kimi K3 results: the system running on Kimi K3 failed all ten submissions in Batch 3, and reviewers judged its work roughly equal to GPT 5.2 Pro, about ten months behind frontier tools.
Artificial Analysis's Kimi K3 model page bases its headline speed and latency figures on Kimi's own API, lists Amazon Bedrock among 11 providers, and marks the model deprecated in favor of Kimi K3.1 Pro. Its provider-level timing page exists, but figures vary by snapshot, so published latency numbers cannot be assumed to describe Bedrock. For GLM 5.3, a LetsDataScience write-up of the GDPx benchmark of 206 expert tasks across 13 models put the model at 2.7 percent overall.
Several details a developer needs sit outside the guide. The GLM 5.3 card omits breakpoint limits, ttl values and write pricing, the pricing page labels the Moonshot model "Kimi 3" and carries no explicit-caching note for open-weight models, and the guide's samples never name a model.
The ttl field is a trap for developers who assume uniform support. Bedrock's documentation says the default explicit TTL is five minutes and resets on each hit, some models support a one-hour option, and adding ttl to a model that supports only five minutes can trigger a ValidationException. For the documented open-weight models, AWS gives a floor of 30 minutes instead, and after an idle gap longer than the TTL the next request writes the prefix again.
Moonshot AI is headquartered in Beijing, China, and Z.ai's legal entity, Knowledge Atlas Technology, is also based in Beijing. China's National Intelligence Law, passed June 27, 2017, and in force from June 28, 2017, requires organizations and citizens to cooperate with national intelligence work in accordance with law. TechTimes reported in July that these obligations apply to the company regardless of where its servers sit, and that the Cybersecurity Law (2017) and Data Security Law (2021) provide separate government-access bases.
How far Article 7 reaches is contested. Legal scholar Jeremy Daum argued in 2024 that "intelligence" is undefined and the article lacks a clear enforcement mechanism, while noting that direct requests from security agencies would still be hard to resist. Huawei submitted a 2018 legal opinion arguing its overseas subsidiaries fall outside the law's territorial jurisdiction. The U.S. Commerce Department added Zhipu to its Entity List in January 2025, citing national security concerns.
AWS's answer is architectural. It says Kimi K3 prompts are processed within the AWS data boundary, are not shared with the model provider and are not used for training, and that zero data retention is always on for inference requests. Monday's guide makes the same no-sharing statement for open-weight models generally, and the US geographic profile keeps Kimi K3 processing inside the US geography. No independent audit of those data-handling claims appeared in the sources reviewed, and the GLM 5.3 model card makes no data-handling statement.
One question remains open for cached prompts specifically. AWS's guide does not say whether cache entries are isolated per customer. A research team including Chenchen Gu, Percy Liang and Tatsunori Hashimoto reported cache sharing across users in seven API providers in 2025, a condition that lets an attacker infer other users' prompts from response timing, though the findings do not describe Bedrock. Teams handling sensitive text can keep secrets out of cached prefixes, because anything placed before a breakpoint is stored by design. No setting removes the legal condition attached to the model provider, and the practical lever is what the provider can receive.
Treat Bedrock's open-weight caching as five separate checks instead of one cost lever. On mechanics, explicit mode needs both the prompt_cache_options field and a breakpoint, prefixes must clear the model's minimum (1,024 tokens on the GLM 5.3 card, 256 on the current Kimi K3 card), and entries live at least 30 minutes. On economics, Kimi 3 and GLM 5.3 listed rates break even on the second request that reads a cached prefix but cost 25 percent more if a prefix is never reread. On reliability, silent misses are documented on a Claude model and unmeasured on open-weight ones, so hits must be verified in usage fields and runtime metrics.
On performance, independent evidence is limited and sometimes poor, and published latency figures do not describe Bedrock. On law and data, the documented models come from Beijing-based labs bound by China's National Intelligence Law, AWS says prompts never reach those labs, no independent audit of that claim was found, and the guide leaves cache isolation unstated. A team that reads only the price table will miss four of those five checks.
AWS's guide does not list any. AWS documentation confirms Kimi K3, the first open-weight model with explicit caching, announced Sept. 18, plus Kimi K3.1 Pro and GLM 5.3, whose model card lists support. AWS says each model card is the authority on modes, minimums and TTL, and support varies by Region.
Read the usage details in the response: cached_tokens counts cache reads and cache_write_tokens counts writes. A write-heavy pattern with few reads signals misses, and the Bedrock runtime metrics CacheReadInputTokenCount and CacheWriteInputTokenCount show the same trend over time, as Farooq's measurements also showed.
AWS says prompts and outputs on Bedrock open-weight models are not shared with the model provider and are not used to train models, and says zero data retention is always on for Kimi K3 inference. No independent audit of those statements appeared in the sources reviewed, and AWS has not said whether cache entries are isolated per customer.
AWS documents at least 30 minutes for Kimi K3 and GLM 5.3, and the TTL resets on every hit. After an idle gap longer than the TTL, the next request writes the prefix again. Bedrock's default explicit TTL is five minutes for models without a longer floor.
