vLLM

AFD plugin — attention-FFN disaggregation for MoE serving (July 23). Splits attention and feed-forward computation onto separate workers so Mixture-of-Experts models can scale each phase independently; runs on both NVIDIA GPUs and Ascend NPUs — the second hardware ecosystem is a deliberate signal. Per the vLLM blog.

Production-scale Kimi K3 support with KDA-aware prefix caching + MXFP4 MoE (July 22). vLLM's prefix cache is now aware of Kimi's KDA attention structure, and MXFP4 quantization lands for MoE weights — frontier open-weight models get first-class serving the week they matter. Per the vLLM blog.

Semantic Router expands to Mixture-of-Models systems (July 21). The router that picked per-request paths inside one model family now routes across DIFFERENT models — vLLM positioning itself as the serving layer for heterogeneous model fleets, not single deployments. Per the vLLM blog.

Latest release: v0.30.0. vLLM ships rapidly; earlier references in this node reflect older minors (v0.27.1 was the current release as of August 2026). Per vLLM releases.

Native LMCache integration via the KVConnector interface. vLLM exposes a KVConnector interface; LMCache plugs in as a remote-tier connector so prefix-cache lookups span GPU HBM → CPU DRAM → NVMe → S3-compatible object storage transparently. Per the vLLM repo and the LMCache repo.

Prefill-decode disaggregation generally available. vLLM 0.10 split prefill-heavy (compute-bound) from decode-heavy (memory-bound) request scheduling into separate workers, enabling specialized GPU pools and reducing decode-stage tail latency. Per the vLLM repo.

Adopted as the kernel under Mooncake's serving plane. Moonshot's Mooncake project (the KV-cache-centric inference system behind Kimi) uses vLLM kernels internally with Mooncake's transfer engine replacing the prefix-cache store. Per Mooncake repo.

NIXL transfer-engine integration for RDMA prefix shipment. vLLM 0.10.5 added a NIXL-compatible transport so prefix-caches can be shipped between vLLM workers over RDMA (BlueField-4) at near-line-rate without staging through the host. Per the NVIDIA NIXL repo.

v1 engine core deadlocks under concurrent load when FP8 quantization, prefix caching, and Qwen3.5 architectures combine — tracked as issue #37729. Maintainers advise pinning to the older stable v0.6.14 release while avoiding v0.6.15, which carries its own autotuner regressions; teams running Qwen3.5 in FP8 with prefix caching enabled should hold at 0.6.14 rather than following the current release line until the deadlock is resolved. Per vllm-project/vllm issue #37729 (single-source GitHub issue, not yet corroborated by an official vLLM release note or blog post). Sources: vLLM project homepage · vLLM repo · LMCache repo · arXiv 2309.06180 — Efficient Memory Management for LLM Serving (the original PagedAttention paper, foundational reference)

0.28–0.29 attack cold-start and prefix reuse — but still not to object storage. Per vLLM releases, "Fast Start" adds a persistent per-GPU weight-cache daemon holding post-quantized, TP-sharded weights in GPU memory; a restarting engine maps them over CUDA IPC via --load-format ipc_cache, skipping the reload from local disk or object storage entirely. Mamba prefix caching landed with a measured 9–25% TTFT improvement on internal prefill checkpoints. prefix_cache_retention_interval became a CLI argument defaulting to 0, with dense retention restored automatically for hybrid EAGLE/MTP models, and Model Runner V2 is now the default for all models, adding CUDA-graph memory profiling for KV-cache auto-sizing and cutting per-step logits memory by a factor of the TP size. The honest negative, and it matters for this index: no KV-cache offload to an object-storage backend ships in these releases. The memory-tier work is real and stops at GPU/host memory — the S3 tier is still LMCache/Mooncake territory.