openai-compatible-api

WebLLM is a high-performance in-browser LLM inference engine that runs locally in the browser with WebGPU acceleration. It exposes an OpenAI-compatible API, supports streaming and JSON mode, and includes examples for building chat apps and browser extensions.

AURA is a Rust-based SRE agent platform for investigating and fixing production incidents. The repository shows a quick start, production-safety controls, multi-model support, MCP and RAG integrations, and an example incident workflow using traces, logs, and Kubernetes history.

Open source inference engine for agents that self-tunes on the user’s hardware to run open models faster, with desktop app support and one-click connections to popular agents.

An open-source mixture-of-agents server that fans prompts out to multiple LLMs, judges the responses, and synthesizes a final answer. It exposes an OpenAI-compatible API and also includes an agentic coding harness plus integrations for tools like Claude Code, aider, Continue, LangChain, and Pi.

GIDE is a privacy-first AI coding command center that runs local or cloud models in a code editor and terminal. The page shows live session examples, a write-approval gate, CLI support, and an OpenAI-compatible local API.

Open-source self-hosted inference server for agent workloads, with an OpenAI-compatible API, multi-model serving, and production cluster tooling.

Wyolet Relay is a self-hosted LLM router that exposes OpenAI- and Anthropic-compatible endpoints in front of multiple providers. It pools API keys, handles failover and rate limits, tracks usage and cost, and includes a quickstart plus Docker-based deployment.

A pure-Python LLM inference engine and server with CUDA/HIP/Metal code generation, OpenAI-compatible API support, built-in RAG, and multi-GPU backend support.

A pure Go CLI and local API proxy for connecting apps and coding agents to DeepSeek R1/V3 through a web account, with OpenAI, Anthropic, and Ollama-compatible endpoints.

A self-hosted Go service that turns headless coding-agent CLIs like Claude Code, Codex, and Kimi Code into an OpenAI-compatible API with chat, streaming, tool calling, MCP, sessions, and image support.

A macOS command-line tool for on-device AI and voice tasks, including speak, listen, transcribe, chat, and an OpenAI-compatible local server.

Flux is a Rust-based LLM inference planner and runtime that measures local hardware, chooses a model placement plan, and serves models behind an OpenAI-compatible API.

A detailed walkthrough for running a local coding agent on macOS with llama.cpp, Gemma 4, MTP speculative decoding, image support, and Pi as the agent interface.

An Apple Silicon–optimized inference build of Bonsai 1.7B with custom Metal kernels, benchmark results, quick-start instructions, and a bundled OpenAI-compatible server.

A technical tutorial that walks through building a coding agent in Python, including tool calling, file editing, shell execution, web search, and permission prompts. It includes code snippets, a demo workflow, and a linked GitHub repository with the full implementation.

A vLLM blog post explaining Semantic Router as an open serving-layer runtime for bounded micro-agents, with routing patterns like Confidence, Ratings, ReMoM, Fusion, and Workflows behind a single model API.

MIKE is an integrated Kanban environment and LLM harness for software development work, with gVisor-based sandbox isolation, custom extensions, and support for llama.cpp and OpenAI-compatible inference providers.