Definition

What it is

An open-source LLM serving engine originally developed at UC Berkeley (Sky Computing Lab) that introduced **PagedAttention** — a paging-style KV-cache memory manager modeled on OS virtual memory. The block-based allocator eliminates KV-cache fragmentation, enables zero-copy prefix sharing across requests, and is now the reference implementation that most KV-cache-aware infrastructure (LMCache, Mooncake, NIXL, ObjectCache) targets.

Why it exists

Naive contiguous KV-cache allocation wastes 60-80% of GPU memory to internal fragmentation and pre-reserved padding. PagedAttention treats the KV-cache as fixed-size pages that can be allocated on demand, shared across requests with identical prefixes, swapped to CPU memory under pressure, and (with LMCache or ObjectCache) persisted to remote tiers. The result is multi-x throughput improvements at equivalent latency budgets.

Primary use cases

Production LLM serving (text generation, chat completion, embedding), multi-tenant inference platforms (a single GPU pool serving many tenants with diverse prompt prefixes), prefix-cached agent workflows, RAG inference where the retrieved-context prefix is hot across requests.

Recent developments

Latest signals

AFD plugin — attention-FFN disaggregation for MoE serving (July 23). Splits attention and feed-forward computation onto separate workers so Mixture-of-Experts models can scale each phase independently; runs on both NVIDIA GPUs and Ascend NPUs — the second hardware ecosystem is a deliberate signal. Per the vLLM blog.
Production-scale Kimi K3 support with KDA-aware prefix caching + MXFP4 MoE (July 22). vLLM's prefix cache is now aware of Kimi's KDA attention structure, and MXFP4 quantization lands for MoE weights — frontier open-weight models get first-class serving the week they matter. Per the vLLM blog.
Semantic Router expands to Mixture-of-Models systems (July 21). The router that picked per-request paths inside one model family now routes across DIFFERENT models — vLLM positioning itself as the serving layer for heterogeneous model fleets, not single deployments. Per the vLLM blog.
Latest release: v0.27.1 (current as of August 2026), succeeding v0.25.1 (July 2026). vLLM ships rapidly; earlier references in this node reflect older minors. Per vLLM releases.
Native LMCache integration via the KVConnector interface. vLLM exposes a KVConnector interface; LMCache plugs in as a remote-tier connector so prefix-cache lookups span GPU HBM → CPU DRAM → NVMe → S3-compatible object storage transparently. Per the vLLM repo and the LMCache repo.
Prefill-decode disaggregation generally available. vLLM 0.10 split prefill-heavy (compute-bound) from decode-heavy (memory-bound) request scheduling into separate workers, enabling specialized GPU pools and reducing decode-stage tail latency. Per the vLLM repo.
Adopted as the kernel under Mooncake's serving plane. Moonshot's Mooncake project (the KV-cache-centric inference system behind Kimi) uses vLLM kernels internally with Mooncake's transfer engine replacing the prefix-cache store. Per Mooncake repo.
NIXL transfer-engine integration for RDMA prefix shipment. vLLM 0.10.5 added a NIXL-compatible transport so prefix-caches can be shipped between vLLM workers over RDMA (BlueField-4) at near-line-rate without staging through the host. Per the NVIDIA NIXL repo.
v1 engine core deadlocks under concurrent load when FP8 quantization, prefix caching, and Qwen3.5 architectures combine — tracked as issue #37729. Maintainers advise pinning to the older stable v0.6.14 release while avoiding v0.6.15, which carries its own autotuner regressions; teams running Qwen3.5 in FP8 with prefix caching enabled should hold at 0.6.14 rather than following the 0.27.x release line until the deadlock is resolved. Per vllm-project/vllm issue #37729 (single-source GitHub issue, not yet corroborated by an official vLLM release note or blog post). Sources: vLLM project homepage · vLLM repo · LMCache repo · arXiv 2309.06180 — Efficient Memory Management for LLM Serving (the original PagedAttention paper, foundational reference)