performance

Release page for vLLM v0.28.0, a high-throughput and memory-efficient inference and serving engine for LLMs. The snapshot shows release highlights, model support updates, breaking changes, and installable artifacts for PyPI, Docker, ROCm, CPU, and XPU.

A web calculator for estimating whether local LLMs fit on specific hardware and how fast they may run. It covers VRAM, token throughput, latency, power, and cost across multiple GPUs and devices.

A drop-in proxy for Ollama that automatically tunes local LLM requests to reduce RAM use and speed up first-token latency. The page shows install steps, benchmark results, a live dashboard, and a local-only admin UI.

A pure C implementation for training and running a tiny GPT, with build instructions, sample output, and performance notes for macOS, Linux, and Windows.

Tempered is an AI-assisted Windows optimizer that explains performance issues in plain language, shows measured before-and-after results, and keeps changes revertible through an Undo Center.

LiteLLM is moving its AI gateway to Rust and claims major gains in throughput, memory use, and request overhead while keeping the same config, API, and provider coverage.

A Rust library for sparse byte-range caching and async read coalescing on immutable objects, with support for range semantics, eviction, and optional read-through caching.