Developer Tools / AI Inference / LLM Serving
Open-source C++ and CUDA LLM inference engine inspired by vLLM, with a teaching-focused course that walks through model serving, batching, KV cache, and attention kernels.
The page clearly describes an AI infrastructure project: a high-performance LLM inference engine in C++ and CUDA. The README gives concrete implementation details, mentions Safetensors and Llama 3.2 1B Instruct support, and lists engine features like KV cache, static and continuous batching, online softmax, and PagedAttention. It is useful for AI builders and readers who want both code and an educational walkthrough.
The snapshot does not show installation prerequisites, runtime examples, benchmark results, API usage, or maintenance signals beyond the file list and commit count. A visitor still cannot tell how to run the server or what interfaces it exposes.
Last evaluated 104 days ago. Current rank #328. Down 3 spots in the rankings.
CodeGraph is a local code knowledge graph for AI coding agents like Claude Code, Cursor, Codex, OpenCode, and Hermes Agent. It aims to cut token use, tool calls, and runtime by letting agents query pre-indexed code structure instead of scanning files repeatedly.
Mercemur is a store automation platform with three interfaces: a REST API, an MCP server for AI agents, and a CLI for editing storefronts as files. The docs show authentication, scopes, rate limits, error handling, pagination, idempotency, and a large API surface for catalogue, orders, customers, pricing, and money.
A zero-dependency C++23 CLI and MCP server that gives coding agents a ranked map of a repository, including call context, blast radius, and tests to run.
Traccia is an AI agent observability and governance platform with OpenTelemetry-native tracing, runtime policy enforcement, prompt registry, evals, cost attribution, and compliance evidence export.
