
llm-d
SandboxJoined the CNCF 12 Mar 2026llm-d is a Kubernetes-native, high-performance distributed LLM inference framework built on vLLM and the Kubernetes Gateway API Inference Extension, providing intelligent inference scheduling, prefix-cache-aware routing, prefill/decode disaggregation, hierarchical KV offloading, and traffic- and hardware-aware autoscaling across NVIDIA, AMD, Intel, and Google TPU accelerators.
What is llm-d?
llm-d is a Kubernetes-native, high-performance distributed LLM inference framework built on vLLM and the Kubernetes Gateway API Inference Extension, providing intelligent inference scheduling, prefix-cache-aware routing, prefill/decode disaggregation, hierarchical KV offloading, and traffic- and hardware-aware autoscaling across NVIDIA, AMD, Intel, and Google TPU accelerators. It is a CNCF sandbox project in framework.
- Tags
- ai · cncf · distributed-inference · gpu · inference · intelligent-routing
- Licence
- Apache License 2.0 · Python, Shell
Stars and activity
Star history: daily readings started on 21 Nov 2026; the chart appears after a few days.
Maintainers
- Abdullah GharaibehGoogle LLC · @ahg-g
- Carlos CostaIBM Research · @chcost
- Clayton ColemanGoogle LLC · @smarterclayton
All 6 maintainers
- David SimmonsRed Hat · @davidgs
- Pete CheslockRed Hat LLC · @petecheslock
- Robert ShawRed Hat LLC · @robertgshaw2-redhat
llm-d FAQ
Is llm-d a CNCF sandbox project?
Yes. llm-d is a CNCF sandbox project, since 12 Mar 2026.
Is there a llm-d certification?
Not yet. CNCF certifications cover Kubernetes and a growing list of projects.
How many GitHub stars does llm-d have?
5,297 on 21 Nov 2026.
What are the alternatives to llm-d?
Other CNCF framework projects: KServe, KAITO, AI Runway.


