Skip to content
CertDirectory.io

Limited Time: 40% OFF all Linux Foundation Certifications. Save now

llm-d

SandboxJoined the CNCF 12 Mar 2026

llm-d is a Kubernetes-native, high-performance distributed LLM inference framework built on vLLM and the Kubernetes Gateway API Inference Extension, providing intelligent inference scheduling, prefix-cache-aware routing, prefill/decode disaggregation, hierarchical KV offloading, and traffic- and hardware-aware autoscaling across NVIDIA, AMD, Intel, and Google TPU accelerators.

5.3kGitHub stars
251contributors
v1.0.0latest release, 20 Oct 2026
86CLOMonitor score
6maintainers
2025first commit

What is llm-d?

llm-d is a Kubernetes-native, high-performance distributed LLM inference framework built on vLLM and the Kubernetes Gateway API Inference Extension, providing intelligent inference scheduling, prefix-cache-aware routing, prefill/decode disaggregation, hierarchical KV offloading, and traffic- and hardware-aware autoscaling across NVIDIA, AMD, Intel, and Google TPU accelerators. It is a CNCF sandbox project in framework.

Tags
ai · cncf · distributed-inference · gpu · inference · intelligent-routing
Licence
Apache License 2.0 · Python, Shell

Stars and activity

Star history: daily readings started on 21 Nov 2026; the chart appears after a few days.

Commits each week, the last 52 weeks

Maintainers

6 maintainers, from CNCF's maintainers list · companies include Google LLC, IBM Research, Red Hat, Red Hat LLC
All 6 maintainers

llm-d FAQ

Is llm-d a CNCF sandbox project?

Yes. llm-d is a CNCF sandbox project, since 12 Mar 2026.

Is there a llm-d certification?

Not yet. CNCF certifications cover Kubernetes and a growing list of projects.

How many GitHub stars does llm-d have?

5,297 on 21 Nov 2026.

What are the alternatives to llm-d?

Other CNCF framework projects: KServe, KAITO, AI Runway.