Your AI feature shares a cache, a key store and a trace store. Who else is in them?
Under an AI feature sit three things your customers share: the inference engine's cache, the gateway that holds your provider keys, and the store of agent traces; on a hosted model, other companies' customers share the first one too. In October 2026 each one failed at a different company. A customer's security review will test tenant isolation one layer at a time, asking who operates it, who else shares it and what version runs, and the answer needs a row per layer.
Which parts of your AI feature are shared across customers?
A security review used to accept one row for AI: we use provider X. October 2026 split that row into three layers, each with a dated public failure. Cloudflare's inference engine built replies partly from other users' cached data; LiteLLM, an open source model gateway, shipped a compromised release; and BleepingComputer reported a breach at one of Warp's vendors that exposed agent logs.
For each layer, a customer's security team will ask the same four questions. Who operates it? Who else shares it: only your customers, or other companies' customers too? What version or configuration runs? And what is the customer told?
Three layers under your AI feature, and who else shares each
Can one customer's prompt end up in another customer's response?
- 155
Responses Cloudflare found built partly from other users' cached data, all to prompts over 681,280 tokens.
Source: Cloudflare, Oct 2026 - v0.5.22
The SGLang release Cloudflare recommends, or later. LMSYS has not backported the fix.
Source: LMSYS and Cloudflare, Oct 2026 - ~1%
Warp users whose agent logs or telemetry may have been exposed through a vendor.
Source: BleepingComputer, Oct 2026
Yes. Cloudflare reported on 26 October 2026 that GLM 5.3 on Workers AI built 155 responses partly from other users' cached data between 27 August and 9 September 2026, a cross-tenant leak in LLM inference through the shared KV cache. Every one of them was a reply to a prompt longer than 681,280 tokens. Cloudflare says a small number of customers were affected, publishes no count, and found no evidence of deliberate exploitation.
The replies looked fine. They came back with HTTP 200, a prompt_tokens count that matched the long prompt, and fluent text. The only tell was speed: each arrived faster than a prompt that long should take to prefill. Cloudflare's long-context tests before launch checked status codes and engine performance, so they passed.
The cause, in Cloudflare's account, was a gap between two numbers. The model advertised a context of 1,048,576 tokens, and the KV cache pool on the prefill side held 681,280. A request too long for the pool should have been refused. It was admitted. In Cloudflare's diagram the decode step reserves eight cache pages, of which the request had written one; the other seven still held what earlier requests left there.
The model accepted prompts the cache could not hold
GLM 5.3 on Cloudflare Workers AI, tokens
A researcher, Gadi Evron of Knostic, reported it through HackerOne at 23:11 UTC on 8 September. Cloudflare declared an incident at 13:22 UTC the next day, made the model private at 14:00 and restored it at 19:40.
The flaw sat in the engine, and the engine is open source. In a security update on 26 October 2026, LMSYS, which maintains SGLang, traces it to disaggregated prefill and decode serving with a KV transfer backend (Mooncake, NIXL or MORI), on default settings from the v0.4.x line in early 2025. Version 0.5.21 protects Mooncake and NIXL by default, MORI needs v0.5.22, and fixes for failed NIXL and MORI sends ship in v0.5.23. There is no backport. A team running that topology on an affected version in its own cloud had the same exposure; on a single node, the same over-length request returned an error.
LMSYS's post also describes a second bug, now fixed: when a transfer was aborted, late RDMA writes could land in cache slots already handed to another request, corrupting its output with no error and no log line. An optional KV checksum catches some of these late writes; it is off by default. Cloudflare's advice to operators: "do not assume the engine will reject what does not fit."
Where are your provider keys, and who could have read them on 21 October?
A model gateway sits between your product and every model provider. Many teams issue a virtual key per customer through it, which keeps billing and rate limits apart. Underneath, each customer's virtual key and every provider key sit in one database, read by one process on one host.
On 21 October 2026 that host became the target. LiteLLM v1.107.0, published to PyPI and to the project's container registries, contained compiled malware that steals credentials and loads remote code execution tools. LiteLLM publishes no count of affected users, and the enterprise release on its private registry was not affected. Its follow-up on 23 October traced the way in to an administrative key committed to a public repository in May and never revoked.
The compromised release stayed on Docker Hub about 50 minutes longer than on PyPI
LiteLLM v1.107.0 availability on 21 October 2026, minutes
LiteLLM's guidance assumes the worst: if the release ever ran on a host with outbound internet, treat the theft as successful. Rotate the instance role, every provider key stored in the proxy database, the master and virtual keys, the DATABASE_URL password and every secret in the environment of any process on the host.
For the VP Engineering, the test is speed. Can your team say, from build and registry logs, which hosts, containers and CI jobs pulled that version or those tags between 11:39 and 14:20 UTC on 21 October? A customer's security team will want that answer within days, along with the date its keys were last rotated.
Where are agent traces stored, and by whom?
Start with what a trace holds. An agent run records each step: what it read, which tools it called, what came back and what it wrote. In a vertical SaaS product, what it read is your customers' records, such as a patient's chart or a tenant's ledger. Wherever the traces go, copies of those records go with them.
On 27 October 2026 BleepingComputer reported that Warp had disclosed a breach through an attack on a third-party vendor in its infrastructure, exposing agent decision logs and telemetry. Warp's CEO, Zach Lloyd, said roughly 1% of users may have been affected and that the company detected the operation within hours. Warp did not say whether the logs held source code, prompts or credentials.
Your customers will ask the question Warp left open: what is in your traces, and who can reach them? The trace store and every vendor that touches it, for observability, evaluation or support, belong on the subprocessor list as rows of their own, each with a retention period, who has access, and whether one customer's traces are stored apart from another's.
What should you tell customers?
Tell them where each layer runs and who shares it, in the same document as your subprocessor list and your DPA. The table below is a first draft, one row per layer.
With a hosted model, assume the KV pool serves other companies' requests unless the provider confirms otherwise in writing, and ask for the engine version and how the provider tests that replies match their prompts. With a self-hosted model, the same questions go to your own platform team.
Tenancy by layer, for the security questionnaire
| Layer | Who operates it | Shared with | What to confirm | What to tell the customer |
|---|---|---|---|---|
| Inference engine and KV cache | Your inference provider, or your team if self-hosted | Every request on the same nodes; with a hosted model, other companies' customers too | SGLang v0.5.22 or later (v0.5.23 for NIXL or MORI send failures); KV pool covers the advertised context on every node; KV checksum on | Provider name as a subprocessor, the engine patch level, and how you test that replies match their prompts |
| Model gateway and key store | Your platform team, or the gateway vendor | All your customers' virtual keys and every provider key, in one database | Which hosts installed LiteLLM 1.107.0 or pulled the affected Docker Hub or GHCR tags on 21 October; rotation done per LiteLLM's list | Where provider keys live, who can read them, when they were last rotated |
| Trace store | Your observability vendor, or your team | Every customer's runs, unless traces are partitioned per customer | Where traces are stored, retention, which vendors can reach them | Traces as a subprocessor row with retention and access, and whether a customer's traces stay in their region or boundary |
Our draft, from the Cloudflare, LMSYS, LiteLLM and Warp accounts cited in this post.
Above these three layers sits the application: what an agent may read, believe and do inside one customer's account. That is a separate review, covered in our post on agent state in multi-tenant products.
If you sign the security questionnaire
- Security: add the three rows above to the subprocessor list and the DPA draft this week.
- VP Engineering: confirm the SGLang version and KV pool setting on every node, or get both from your inference provider in writing.
- Platform: run the query for hosts that pulled LiteLLM 1.107.0 on 21 October, and rotate per LiteLLM's list if any did.
- VP Engineering: add a long-context canary per test tenant that checks the reply refers only to its own prompt, and alert when a long prompt answers faster than its prefill should allow.
Which layer under your AI feature would be hardest to answer for in a security review today?
Anonymous. One vote per browser.
Where SynOS fits
SynOS installs in your cloud, so customer data, traces and models stay inside your system boundary, with a Brain for each of your customers. Every run is traced, and agents act under a named user's identity that can be revoked. The trace store and each customer's context become rows you operate yourself. The inference engine underneath, its version and its KV pool settings stay your team's to patch and check, with or without SynOS.
Questions
What does tenant isolation mean for an AI feature?
It means one customer's prompts, cached data, keys and traces stay out of every other customer's requests at each layer under the feature: the inference engine and its KV cache, the model gateway that holds provider keys, and the store of agent traces. For each layer a security review asks who operates it, who else shares it, what version runs and what the customer is told.
Does self-hosting the model prevent KV-cache leakage between customers?
By itself, no. The Cloudflare exposure came from SGLang's disaggregated prefill and decode path, so a team running the same engine and topology on an affected version in its own cloud had the same exposure. Upgrading to v0.5.22 or later and checking that the KV pool covers the advertised context are what close it.
What should we rotate if a host installed LiteLLM 1.107.0?
LiteLLM's list: the instance IAM role, every provider key stored in the proxy database, master and virtual keys, the DATABASE_URL password and every secret in the environment of any process on the host. It advises assuming exfiltration succeeded if the host had outbound internet.
Are agent traces a subprocessor question?
Yes, whenever a vendor stores them or can reach them. A trace holds what the agent read, which in a vertical SaaS product means customer records, so list the vendor, the retention period and who has access.
Sources
- Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang, Cloudflare (26 October 2026; exposure window, 155 responses, token figures, eight reserved pages, timeline, prelaunch testing, advice to operators)
- SGLang security update: improving request isolation, LMSYS (26 October 2026; affected topology and versions, no backport, the aborted-transfer bug, single-node behaviour, checksum off by default)
- Contained: Oct 21, 2026 LiteLLM release v1.107.0 Compromised, LiteLLM (21 October 2026; registry windows, malware, rotation list, private registry unaffected)
- The attack chain behind the v1.107.0 LiteLLM release, LiteLLM (23 October 2026; the entry point, an administrative key committed to a public repository)
- Warp discloses breach exposing AI agent logs, user telemetry data, BleepingComputer (27 October 2026; vendor breach, agent logs and telemetry, roughly 1% of users, contents not clarified)