Preview of vllm-project/vllm

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

86.4K Stars
19.5K Forks
86.4K Watchers
5.8K Issues
Python
amd blackwell cuda deepseek deepseek-v3 gpt gpt-oss inference kimi llama llm llm-serving model-serving moe openai pytorch qwen qwen3 tpu transformer

Last push:

Created:

Loading additional information…