MinIO MemKV
A flash-native context-memory store embedded in the AI storage tier, exposing petabytes of NVMe to GPU pods as shared KV cache over RDMA on NVIDIA BlueField-4 — a dedicated "memory tier" that eliminates the inference recompute tax.
Summary
A flash-native context-memory store embedded in the AI storage tier, exposing petabytes of NVMe to GPU pods as shared KV cache over RDMA on NVIDIA BlueField-4 — a dedicated "memory tier" that eliminates the inference recompute tax.
The convergence point of the S3 persistence layer with real-time inference memory: it lets a whole inference cluster draw KV cache from a petascale shared pool at microsecond latency, with no host CPU in the data path.
- It is not standard object or file access — it bypasses both protocols for throughput-oriented 2–16 MB blocks tuned for GPU ingestion.
- The win is utilization, not raw storage: GPUs stop wasting cycles rebuilding evicted KV cache (~50%→~90% useful utilization).
- MinIO MemKV
extendsMinIO — runs within the AIStor tier - MinIO MemKV
acts_asInference Context Memory Storage (ICMS) - MinIO MemKV
solvesHigh Cloud Inference Cost — eliminates the KV recompute tax
Definition
A purpose-built, flash-native context-memory store embedded in the AI storage tier. It exposes petabytes of NVMe as shared KV cache to GPU pods over 800 GbE RDMA, operating independently of file or object protocols to deliver microsecond access — a dedicated "memory tier" for inference context.
In 2026 production inference, GPUs waste over half their cycles rebuilding evicted KV cache for 128K+ token conversations because VRAM is exhausted — the "recompute tax." MemKV relocates the context boundary into a petascale shared pool with no host CPU in the data path, using large throughput-oriented blocks (2–16 MB) tuned for GPU ingestion.
Shared KV-cache pool for inference clusters, eliminating recompute on long-context conversations, raising useful GPU utilization, microsecond context retrieval at petascale.
Recent developments
- Runs natively on NVIDIA BlueField-4 (STX). Deploys as a sub-200MB ARM64 static binary on the DPU, drawing context from a shared pool at microsecond latency with no host CPU in the path. Per MinIO — Introducing MemKV and MinIO — AIStor on BlueField-4.
- Throughput-oriented "G3.5" memory tier. Replaces 4KB blocks with 2–16 MB blocks for GPU ingestion; targets up to ~60% lower cost-per-token. Per MinIO — Introducing MemKV.
Connections 5
Outbound 5
scoped_to1extends1optimizes_for1solves1Resources 2
The launch post explaining MemKV as a flash-native shared KV-cache tier over RDMA — the recompute-tax problem, the 2–16 MB block design, and the utilization/cost-per-token claims.
Details MemKV/AIStor running natively on NVIDIA BlueField-4 with no host CPU in the data path — the hardware substrate for the shared KV pool.