smg-project/smg

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

★ 717 Stars

⑂ 218 Forks

○ 717 Watchers

✖ 191 Issues

Rust

anthropic anthropic-api chat claude gemini inference-gateway lightseek llm mcp openai responses-api routing sglang tensorrtllm tokenspeed trtllm vllm

Last push: 1d ago

Created:

↗ View on GitHub

Loading additional information…