Preview of vllm-project/vllm

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

★ 96.9K Stars
⑂ 24.8K Forks
○ 96.9K Watchers
✖ 10.3K Issues
Python
amd blackwell cuda deepseek deepseek-v3 gpt gpt-oss inference kimi llama llm llm-serving model-serving moe openai pytorch qwen qwen3 tpu transformer

Last push:

Created:

Loading additional information…