Definition

What it is

A new storage tier — also referred to as **Context Memory eXtension (CMX)** — sitting between traditional NVMe SSDs and cold S3 buckets, specifically optimized for AI inference state. Leverages high-performance DPUs (NVIDIA BlueField-4) and DPU-attached flash to offload data placement and context retrieval at the pod level. Solidigm and other flash vendors are productizing CMX as a distinct SKU class, separate from general-purpose enterprise SSDs, with media tuned for the bursty, mixed read/write access patterns of agentic state and KV-cache offloading.

Why it exists

AI inference workloads have a specific shape that traditional storage hardware doesn't serve well — long sessions interleaved with short bursts, KV-cache state that's written once and read many times, agent memory that persists for hours-to-days but doesn't need cold-tier economics. CMX-class storage targets that exact workload, with hardware-accelerated paths for the most common inference operations (KV-cache lookup, agent-state checkpoint, prefill token streaming).

Primary use cases

Persistent KV-cache pools for disaggregated prefill, "instant resume" agentic state across cluster failures, per-pod inference scratchpads, shared-tenant KV-cache fabric, inference-aware data placement targets.

Recent developments

Latest signals

KV-cache quantization goes production: Google TurboQuant merged in vLLM; IndexCache reports 1.82× prefill speedup (July 2026). TurboQuant — Google Research's 3-bit KV-cache quantization, peer-reviewed at ICLR 2026 — was merged upstream in vLLM on April 15 with four production presets (llama.cpp declined the port on June 2), AWS published a GPUDirect-on-FSx-for-Lustre reference architecture pairing it with direct-to-HBM model loading, and it now has an independent reproduction: a community measurement on consumer AMD RDNA4 hardware ran a 31B model at a full 256K context in 27.1 GB versus 44.1 GB unquantized — the first non-Google confirmation of the memory claims. Separately, IndexCache reports a 1.82× prefill speedup on 200k+-token sequences via cross-layer token index reuse (no peer-reviewed paper yet — research-summary provenance). Both shrink the KV-cache bytes-per-token that this storage tier exists to hold — quantization and tiering are converging on the same cost problem from opposite ends. Per AWS ML Blog — GPUDirect + TurboQuant on FSx for Lustre, independent AMD RDNA4 measurement, open-source implementation (GitHub), and IndexCache — attention mechanisms 2026.
Taxonomy settled: CMX is the pod tier; STX is the rack-scale blueprint; AIDP is the pipeline design. NVIDIA's naming stabilized at GTC 2026: CMX (Context Memory eXtension, the renamed ICMSP from CES) is only the G3.5 KV-cache tier; STX (Storage Technology eXtensions) is the rack-scale superset mandating three modules — a CMX system, a Storage Frontend on Vera CPUs, and an AI Data Platform system — interconnected exclusively over Spectrum-X Ethernet; AIDP is the parallel end-to-end RAG-pipeline reference (RTX PRO GPUs + BlueField + NIM/NeMo/cuVS). CMX's design tradeoff is explicit: it deliberately drops replication and integrity guarantees (KV cache is recomputable) to buy up to 5× tokens/sec and 5× power efficiency. Hardware reality check: NVIDIA's CMX enclosure spec matches the AIC F2032-G6 JBOF exactly (32 drives, 4× BlueField-4, ~600 TB) — AIC is the sole manufacturer today, and VAST the only storage software running the native DPU loop. Per Cloudian — NVIDIA STX blueprint, NAND Research — STX/CMX, and Lockwood — ICMS notes.
Solidigm D7-PS1010 demonstrated behind NVIDIA DPU SNAP — remote NVMe-oF with GPUDirect Storage, zero host-CPU involvement (June 2026). The DPU's SNAP framework emulates remote NVMe-oF devices, letting GPUDirect Storage DMA directly from GPU memory to remote SSDs over InfiniBand with no host CPU in the data path. Solidigm's PCIe Gen5 D7-PS1010 is the demonstrated media — the concrete hardware pairing behind the CMX-tier story. Per Solidigm — Accelerating AI with High-Performance Storage.
NVIDIA standardized ICMS / CMX as a new G3.5 tier (Ethernet-attached flash for KV cache). ICMS introduces tier 3.5 between local SSD (Tier 3) and cold S3 (Tier 4), optimized for KV-cache + multi-step MoE inference. NVIDIA + storage partners ratified the framing in Q1 2026. Per NVIDIA Technical Blog — BlueField-4-Powered CMX Context Memory Storage Platform and Blocks & Files — Nvidia's basic context memory extension infrastructure.
Storage-partner roster: VAST, DDN, Dell, HPE, Pure Storage, WEKA. Six major enterprise-storage vendors signed up to ship ICMS-class tier-3.5 hardware. Cross-vendor adoption is what makes ICMS a market category rather than a single-vendor pitch. Per Blocks & Files — Nvidia pushes AI inference context out to NVMe SSDs.
VAST CNode on BlueField-4 demonstrated 90% inference-efficiency gain + 20× TTFT improvement. Real numbers: zero-copy from remote SSD to GPU memory via virtio-fs control plane; KV-cache offload via VAST Undivided Attention (VUA, open-source). Per VAST Data — More Inference, Less Infrastructure: VAST and NVIDIA.
Solidigm productizing CMX as a distinct SSD SKU. Solidigm explicitly positions CMX as a "different SSD product class" — media tuned for the mixed read/write + bursty access pattern of inference context, not the steady-state read-heavy pattern of training data. SSD market segmentation is fragmenting around AI workload shapes. Per Solidigm — ICMS: AI Inference is a Flash Storage Problem.
NetApp introduced the "new memory tier for AI" framing in industry terms. Community-discussion-level adoption signal: NetApp + storage-industry press now talk about ICMS as the canonical 4th memory tier, not just an NVIDIA marketing label. Per NetApp Community — Introducing a New Memory Tier for AI: Inference Context Memory Storage.
Ship timeline: ICMS partner hardware expected H2 2026. Hardware partners (DDN, VAST, Dell, etc.) begin shipping ICMS-class systems in the second half of 2026 — substantially aligned with the NVIDIA Vera Rubin + BlueField-4 STX rollout window. Per Blocks & Files — Nvidia pushes inference context to NVMe.