CUDA

AutoRound is an open-source quantization toolkit for LLMs and VLMs, focused on high-accuracy low-bit inference across CPU, XPU, CUDA, and multiple deployment backends.

Open-source C++ and CUDA LLM inference engine inspired by vLLM, with a teaching-focused course that walks through model serving, batching, KV cache, and attention kernels.

Open-source AI infrastructure project focused on GPU memory management and serving, with a Python control plane, C++ data plane, and optional CUDA kernels.