Definition

What it is

An open-source LLM serving engine originally developed at UC Berkeley (Sky Computing Lab) that introduced **PagedAttention** — a paging-style KV-cache memory manager modeled on OS virtual memory. The block-based allocator eliminates KV-cache fragmentation, enables zero-copy prefix sharing across requests, and is now the reference implementation that most KV-cache-aware infrastructure (LMCache, Mooncake, NIXL, ObjectCache) targets.

Why it exists

Naive contiguous KV-cache allocation wastes 60-80% of GPU memory to internal fragmentation and pre-reserved padding. PagedAttention treats the KV-cache as fixed-size pages that can be allocated on demand, shared across requests with identical prefixes, swapped to CPU memory under pressure, and (with LMCache or ObjectCache) persisted to remote tiers. The result is multi-x throughput improvements at equivalent latency budgets.

Primary use cases

Production LLM serving (text generation, chat completion, embedding), multi-tenant inference platforms (a single GPU pool serving many tenants with diverse prompt prefixes), prefix-cached agent workflows, RAG inference where the retrieved-context prefix is hot across requests.

Recent developments

Latest signals

Latest release: v0.23.0 (current as of June 2026). vLLM ships rapidly; earlier references in this node reflect older minors. Per vLLM releases.
Native LMCache integration via the KVConnector interface. vLLM exposes a KVConnector interface; LMCache plugs in as a remote-tier connector so prefix-cache lookups span GPU HBM → CPU DRAM → NVMe → S3-compatible object storage transparently. Per the vLLM repo and the LMCache repo.
Prefill-decode disaggregation generally available. vLLM 0.10 split prefill-heavy (compute-bound) from decode-heavy (memory-bound) request scheduling into separate workers, enabling specialized GPU pools and reducing decode-stage tail latency. Per the vLLM repo.
Adopted as the kernel under Mooncake's serving plane. Moonshot's Mooncake project (the KV-cache-centric inference system behind Kimi) uses vLLM kernels internally with Mooncake's transfer engine replacing the prefix-cache store. Per Mooncake repo.
NIXL transfer-engine integration for RDMA prefix shipment. vLLM 0.10.5 added a NIXL-compatible transport so prefix-caches can be shipped between vLLM workers over RDMA (BlueField-4) at near-line-rate without staging through the host. Per the NVIDIA NIXL repo.