For two years the framing question about object storage and AI was "how do we get training data out of the bucket fast enough." That question is aging out. The question the second half of 2026 keeps asking is stranger: how much of a model's working memory — the intermediate state it computes and recomputes millions of times a day — belongs in the bucket too?
This index started mapping that inversion in May, when we built out the AI memory infrastructure wing — KV-cache persistence, Mooncake, LMCache, cache-fronted object storage — and the honest caveat at the time was that the economics were asserted, not shown. That caveat just expired.
The arithmetic arrived
The mechanics first, for anyone who hasn't lived in an inference bill: every token a transformer generates re-reads attention state for every token before it. That state — the KV cache — is expensive to compute (the prefill phase is compute-bound) and bulky to hold (roughly 2.6 GB per request against a 70B model, per the TopKV paper). Throw it away and you pay GPU time to rebuild it; keep it in VRAM and you pay for the most expensive memory on earth to hold data you might not touch for hours.
Tensormesh — the company commercializing LMCache — published the break-even math for the third option: park it on S3-compatible storage. Their model prices Tier-2 object storage for activation tensors at roughly $4,000–$5,000 per TB over three years, and assumes a cache hit is ~10× cheaper than recomputing the same tokens. Under those assumptions, about 280 cache hits over three years — one reuse every 94 hours — makes the storage pay for itself. The numbers are vendor-published and the assumptions are tunable, which is exactly why the framework matters more than the figure: for the first time "should we persist inference state to object storage" is arithmetic, not intuition.
The systems side was already ahead of the spreadsheet. Mooncake — the KVCache-centric architecture behind Moonshot's Kimi, running production at thousands of nodes and 100B+ tokens per day — pools underused CPU, DRAM, SSD, and RDMA capacity across a datacenter into one disaggregated cache, explicitly treating remote storage as an active tier rather than a backup. LMCache tiers KV blocks down a CPU-memory → NVMe → S3 hierarchy and recovers reuse even on non-prefix hits. And the TopKV work puts hard edges on the topology problem: NVLink inside a node moves ~900 GB/s, InfiniBand across nodes ~50 GB/s, and cross-datacenter TCP collapses to ~12.5 GB/s — a 72× spread that makes where a cached tensor lives as consequential as whether it exists.
The storage side is building for it
What makes this a structural story rather than an inference-nerd story is that storage vendors are now shipping products shaped exactly like that demand curve — under pricing pressure from the other direction, because the NAND flash supply crisis has enterprise SSD prices up ~472% year-over-year and every hot tier newly scarce.
Wasabi's answer is Fire: an NVMe-backed object storage class at $19.99/TB/month, egress-free and API-fee-free, launched adjacent to GPU compute in San Jose. That is object storage priced and positioned as a memory tier — fast enough for real-time inference pipelines, cheap enough that the 94-hour rule clears with room to spare.
Redpanda's answer is architectural. Cloud Topics, part of the R1 engine, streams heavy message payloads directly to S3/GCS while keeping only metadata and Raft consensus on local NVMe — cutting cross-availability-zone replication traffic by up to 90% (vendor-published). The broker keeps coordination local and pushes bulk state to the bucket: the same tiering instinct as Mooncake, applied to streaming.
Even the query engines are converging on the assumption. The DuckDB 2.0 preview makes the entire I/O layer asynchronous — the change that matters most when your tables live on network-attached object storage, not local disk — and on the ingestion side, pg_duckpipe now streams Postgres changes straight into DuckLake tables on S3 with no broker tier at all.
The part that keeps it honest
Two governance notes belong in this story, because "put your working state in the bucket" widens the blast radius of storage-layer mistakes.
First: ClickHouse 26.7 shipped a deliberately backward-incompatible change — S3 access from user SQL no longer silently borrows the server's cloud credentials, closing an SSRF-shaped exfiltration path that most teams didn't know they had. If your pipelines relied on that behavior, the upgrade will tell you loudly. That is the right kind of breaking change, and it previews the discipline the agentic data plane era demands: when autonomous agents rather than analysts are the ones touching storage, "whose credentials is this request actually using" stops being a philosophical question.
Second: LiteLLM — the router layer sitting in front of production LLM traffic for thousands of deployments — had two of its March releases compromised on PyPI in the TeamPCP supply-chain campaign (per Datadog Security Labs). The current line is clean. The lesson generalizes: the layers that concentrate AI traffic concentrate credentials, and the memory tier we're describing in this post is precisely such a layer.
Our read
Dated call, 2026-08-24: object storage's role in AI systems is reclassifying from cold persistence to active memory tier, and I/O topology — not capacity — becomes the axis storage products compete on for AI workloads. We revise if S3-tier cache-hit latency proves too slow for production TTFT budgets outside long-context niches, or if NAND pricing normalizes enough that local NVMe undercuts the remote-tier arithmetic. Logged on the calls ledger.
What changed on the index
Two new nodes: pg_duckpipe, the Postgres-to-DuckLake WAL-CDC extension (flagged with a source note — currently author-documented only); and Agentic Data Plane, the architecture pattern composing MCP servers, catalog-enforced authorization, and gateway governance into one policy surface for autonomous data access. Enrichments landed on DuckDB (the official 2.0 preview), ClickHouse (the 26.7 credential change), DataFusion (55.0.0), LiteLLM (1.97.0 plus the supply-chain history), SeaweedFS (4.42, sealed directories), Redpanda (Cloud Topics pass-through), LMCache (the break-even framework), and Apache Parquet (Hardwood 1.0, FLOAT16). The index stands at 433 nodes.
Works cited
- LMCache Storage ROI Calculator (Tensormesh) — the 94-hour break-even framework; vendor-published methodology.
- Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving (arXiv) — the production architecture behind Kimi.
- Topology-Aware Data Movement for Disaggregated GPU Inference (arXiv) — the bandwidth-topology ladder and per-request KV sizing.
- A Preview of DuckDB v2.0 (duckdb.org) — engine-wide asynchronous I/O.
- Wasabi unveils AI-ready storage & expands to Silicon Valley (IT Brief) — Wasabi Fire tier and positioning.
- Redpanda Cloud Topics — write pass-through architecture.
- ClickHouse 2026 changelog — the 26.7 S3 credential-resolution change.
- LiteLLM and Telnyx compromised on PyPI: the TeamPCP campaign (Datadog Security Labs).
- pg_duckpipe: Real-time CDC for streaming Postgres tables into columnar DuckLake (dev.to).