llm-serving

An open-source handbook for production LLM serving and inference at scale, covering GPU fundamentals, KV cache, batching, quantization, speculative decoding, and engines like vLLM, SGLang, and TensorRT-LLM.

A research repo that replays real Claude Code and Mooncake traces to test prefix-cache eviction policies and compare them against LRU.

A detailed vLLM blog post explaining speculative decoding on AMD GPUs, with mechanics, supported drafting methods, configuration guidance, tuning advice, and benchmark discussion.