🔍 Read the full analysis: How A KV Cache Leak Can Put An AI Answer In The Wrong Session on ThorstenMeyerAI.com
Get hardware and tech essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Cloudflare’s incident report says its hosted GLM-5.3 model returned 155 responses containing text from other customers’ requests between Aug. 27 and Sept. 9, 2026. The issue required prompts longer than 681,280 tokens and stemmed from a Cloudflare configuration mismatch combined with a bug in the SGLang inference engine. Cloudflare says it found no evidence of deliberate exploitation; its report does not explain how affected customers were notified.
Cloudflare disclosed that its hosted GLM-5.3 model returned 155 responses containing text from other customers’ requests during a 14-day period, after a configuration mismatch and a bug in the SGLang inference engine caused parts of GPU memory to be read across sessions. The company says the issue affected only prompts longer than 681,280 tokens and that it found no evidence of deliberate exploitation.
In an incident report published Oct. 26, Cloudflare said the affected responses occurred between Aug. 27 and Sept. 9. Security researcher Gadi Evron of Knostic reported the issue through Cloudflare’s HackerOne program on Sept. 8, saying very long prompts to the hosted endpoint sometimes returned text belonging to other users’ sessions. Cloudflare counted 155 affected responses after excluding its own tests and the researcher’s requests.
The incident required two faults to coincide. Cloudflare configured the model with a 1,048,576-token context window, but the memory pool used to hold the prompt’s KV cache could store only 681,280 tokens. The company expected oversized requests to produce an error. A bug in SGLang, however, let rejected requests continue through parts of Cloudflare’s split prefill-and-decode system rather than stopping them.
Cloudflare said the decode stage had reserved memory pages for the full prompt, while the prefill stage wrote only a one-token placeholder. The remaining pages could still contain other customers’ prompts and partial generations. As a result, some responses were returned with an HTTP 200 status instead of an error. Cloudflare said it took the model offline 38 minutes after declaring an incident and restored a fixed version that evening.
Infographic 01 · Cross-request KV cache exposure
How a long prompt read someone else's data.
A memory pool smaller than the advertised context window, plus an engine bug that let rejected requests through, made one hosted model attend over other customers' leftover pages.
The mismatch
tokens; Cloudflare knew before launch and expected an error
Only prompts longer than 681,280 tokens could reach the faulty path.
The eight pages
Cloudflare’s own example of one over-long request
written by this requeststill holding earlier users’ prompts and replies
- Decode reserves 8 pages for the long prompt.
- Prefill rejects it and swaps in a one-token placeholder…
- …but the rejected request still passes the handoff checks.
- Prefill writes one page; decode reads all eight.
- The client gets HTTP 200 and a coherent reply, partly built on someone else’s data.
Why pages hold old data
speed trade-off
- GPU memory is a shared pool of fixed-size pages
- Pages are not zeroed between requests; the next one overwrites
- Bookkeeping is the only wall between customers
The fix
9 Sep, 19:40 UTC
- Pool resized so every valid prompt fits
- Engine retires rejected requests before any memory transfer
- Upstream: SGLang v0.5.20 to v0.5.22; operators should run v0.5.22+
Still open
not in the report
- Whether and how affected customers were notified
- How many customers’ requests contributed the leaked data
- What kind of data it was
Sources: Cloudflare incident report (26 Oct 2026); SGLang/LMSYS (26 Oct 2026). Figures and the no-exploitation finding are Cloudflare’s own.
Powered by Thorsten Meyer AI
The incident shows how a failure in inference-engine bookkeeping can expose information even when the underlying AI model is not itself at fault. KV cache data is temporary working memory created as a model processes a prompt. In this setup, memory pages were reused between requests without being cleared; the system relied on software tracking to prevent one request from reading pages associated with another.
For customers, the concern is that an answer can appear coherent and arrive successfully while including material from a different session. Cloudflare said the affected responses were associated with exceptionally long prompts, which narrows the reported condition, but the report does not establish what specific customer information appeared in each response or whether recipients recognized the material as foreign. The episode also highlights the need to test output correctness and isolation, not just response codes and system performance.
The disclosure comes alongside a wider challenge for security programs: AI tools can generate large volumes of bug reports, making it harder to identify valid findings. The source account cites Jump Crypto’s Firedancer audit contest, where 809 reports from 162 researchers were reduced to 127 unique findings; Jump believes AI agents found or wrote nearly every submission. That separate example does not establish a connection to Cloudflare’s incident, but it illustrates the pressure on teams to review security findings while also investigating vulnerabilities.
Products Worth Considering
Apple MacBook Pro 14.2-inch with M5 Max, 18-Core CPU, 20-Core GPU, 48GB Memory, 2TB SSD, Space Black
Apple MacBook Pro 16.2-inch with M5 Pro, 18-Core CPU, 20-Core GPU, 48GB Memory, 1TB SSD, Nano-Texture Display, Space Black
Apple MacBook Air 15-inch Laptop with M5 Chip, 16GB Memory, 512GB SSD, and Liquid Retina Display
As an affiliate, we earn on qualifying purchases.
The Cache Mismatch Behind the Leak
A KV cache stores intermediate calculations—keys and values—that a language model uses to generate text without repeatedly processing the entire prompt. Long prompts require substantial GPU memory. In Cloudflare’s configuration, the cache was divided into fixed-size pages that requests could borrow and return. Cloudflare said those pages were not zeroed between requests because the next request was expected to overwrite them, a practice the company described as common and useful for performance.
Cloudflare ran GLM-5.3 in a disaggregated arrangement: one GPU pool processed the prompt and built the cache, while another generated the answer. According to the report, the SGLang bug allowed a request marked as rejected to pass between those stages because the queues checked whether it had finished, but not whether it had been rejected. The decode stage then read more reserved pages than the prefill stage had populated.
The SGLang issue had been reported publicly as issue #30233 on July 6, 2026, according to Cloudflare, which said it noticed the earlier report during its investigation. Cloudflare also acknowledged that its pre-launch long-context testing checked HTTP response codes and engine performance without validating the model’s output and coherence.
““pages are not currently zeroed between requests; the next request that gets a page simply overwrites it.””
— Cloudflare incident report
Customer Impact Still Undetailed
Cloudflare’s report gives the number of affected responses, the incident dates and the prompt-length condition, but does not identify the customers involved or describe the contents of the exposed text. It also does not say how affected customers were notified. The company said the requests were consistent in timing and volume with ordinary long-context use rather than repeated attempts to exploit the bug, but that assessment does not establish who received each affected response.
The report does not provide a full account of whether any information was retained or acted on by recipients. Cloudflare said it found no evidence of deliberate exploitation; that is the company’s finding, not proof that misuse was impossible. The source material also does not specify whether the SGLang issue has been fixed upstream or describe any additional safeguards beyond the patched Cloudflare deployment.
Follow-Up on Notices and Safeguards
Cloudflare says it restored a fixed version of the model on the evening it took the service offline. Further information that would clarify the incident includes whether the company has contacted affected customers, what data appeared in the responses, and what additional isolation or output-validation checks it has introduced. The report does not give dates for any further disclosures or audits.
For customers and other hosted-model operators, the incident puts attention on testing oversized prompts, rejected-request handling and memory isolation across the full inference path. Cloudflare’s report says long-context tests had focused on status codes and performance; checking generated output and session boundaries is relevant to whether an apparently successful response contains only the requesting customer’s data.
Key Questions
What happened in the Cloudflare incident?
Cloudflare says its hosted GLM-5.3 service returned 155 responses containing text from other customers’ requests between Aug. 27 and Sept. 9, 2026. The company attributed the incident to a memory-pool configuration mismatch combined with a bug in SGLang.
Which requests could trigger the leak?
According to Cloudflare, the affected requests were prompts longer than 681,280 tokens. The model’s advertised context window was 1,048,576 tokens, exceeding the capacity of the relevant KV cache pool.
Did Cloudflare find evidence of an attacker?
Cloudflare said it found no evidence of deliberate exploitation. Its report said the timing and volume of the requests were consistent with ordinary long-context use, while noting that the report does not identify who received each affected response.
What is a KV cache?
A KV cache stores intermediate calculations a language model uses while processing a prompt and generating an answer. It can occupy substantial GPU memory for very long prompts, so systems reuse memory pages; software must keep each request confined to its own data.
How did Cloudflare respond?
Cloudflare said it took the model offline 38 minutes after declaring an incident and restored a fixed version that evening. Its report does not state how affected customers were notified or provide a timeline for further updates.
Source: ThorstenMeyerAI.com
