CPU

A pure C runtime for Gemma 4 E2B CPU inference, with export, benchmarking, and numerical validation tools included.

Voice agent platform that runs on CPUs, with built-in turn-taking, noise suppression, speech-to-text, and speech synthesis for cloud, on-prem, or device deployment.

AutoRound is an open-source quantization toolkit for LLMs and VLMs, focused on high-accuracy low-bit inference across CPU, XPU, CUDA, and multiple deployment backends.

DocsGPT documents a small classifier used for bounded decisions like prompt-injection checks, relevance scoring, and conversation analytics. It explains where the model runs, how it is enabled, what endpoints it uses, and how it behaves on CPU, Mac GPU, or NVIDIA GPU.

Flux is a Rust-based LLM inference planner and runtime that measures local hardware, chooses a model placement plan, and serves models behind an OpenAI-compatible API.