Disaggregated Serving for Hybrid SSM Models in vLLM Apr 21, 2026 · 15 min read Hybrid architectures that interleave Mamba-style SSM layers with standard full-attention (FA) layers — such as NVIDIA Nemotron-H — are gaining traction as a way to combine the linear-time...

Disaggregated Serving for Hybrid SSM Models in vLLM Introduction Hybrid architectures that interleave Mamba-style SSM layers with standard full-attention (FA) layers — such as NVIDIA Nemotron-H — are gaining traction as a way to combine the linear-time efficiency of state-space models with the expressiveness of attention. vLLM already supports disaggregated prefill/decode (P/D) for standard transformer models through its NIXL-based KV connector: a prefill instance computes KV cache blocks and a decode instance pulls them over RDMA, eliminating redundant recomputation. But extending this to hybrid models is not straightforward. FA and SSM layers store fundamentally different state, in different layouts and different sizes, yet the block manager and NIXL connector were designed around a single, uniform KV cache format. In this post we describe how we extended the NIXL connector to support hybrid SSM-FA models in disaggregated mode. The key ideas are: - Dual descriptor views — two sets of NIXL block descriptors that index the same physical memory regions with different offsets and sizes, one for FA blocks and one for SSM blocks. - Physical/logical block bridging — handling the mismatch between the logical block abstraction seen by the block manager and the physical block sizes required by attention kernels. - 3-descriptor conv transfer — a decomposition of the Mamba conv state that enables heterogeneous tensor-parallel transfers without reshuffling data on the sender side. None of these changes modify the existing workflow for standard transformer models. They are purely additive extensions that activate only when the model contains SSM layers. This feature is available with vllm>=v0.20.0 . This work builds on the HMA interface for NIXL and spans several PRs: - #36687 — Dual descriptor views and homogeneous-TP support for hybrid SSM-FA models - #37416 — DS conv state layout for Mamba kernels - #37635 — Heterogeneous-TP 3-descriptor conv state transfer - #37310 — N-1 prefill for Mamba P/D disaggregation Background: The NIXL KV Transfer Workflow Before diving into the hybrid-model changes, let us briefly recap how NIXL disaggregated P/D works for a standard transformer. The workflow has four phases: - Register memory regions — Each worker registers its KV cache tensors with NIXL so they can be accessed via RDMA. - Create block descriptors — For each registered region, we create per-block descriptors that specify (address, length, device_id) . These descriptors are our unit of transfer: rather than moving entire regions, we transfer individual blocks. - Handshake — When a decode (D) worker first needs to pull from a prefill (P) worker, the two exchange metadata: agent handles, block counts, block lengths, and so on. This is done once per P-D pair. - Transfer — The scheduler tells D which blocks to pull from P. D maps block_id -> descriptor_id , issues an RDMA READ, and polls for completion. For a standard model with M registered regions and N blocks, the descriptor list looks like: +----------------------------------+ | Region 0: desc_0 ... desc_{N-1} | | Region 1: desc_0 ... desc_{N-1} | | ... | | Region M: desc_0 ... desc_{N-1}…