llm

A Redis-backed concurrency lease manager for FastAPI and Starlette that limits active SSE, LLM, and WebSocket streams instead of request counts.

Open-source memory system for LLM characters with library, API, MCP server, and WebUI options. The repo explains how it stores and recalls character, user, episode, emotion, world, and calendar memory, and includes install and usage examples.

Self-hosted AI business analyst for querying company spreadsheets, contracts, PDFs, emails, and sample data, with answers that cite source files and run locally with your own model key.

Deterministic harness for LLM code edits. The model proposes SEARCH/REPLACE blocks, a local runtime applies them in a shadow worktree, and TypeScript compilation gates the merge.

Self-hosted voice agent runtime that streams speech-to-text, LLM output, and text-to-speech together in one process. The page shows quickstart commands, a minimal agent example, browser/client embedding, authentication notes, and performance benchmarks.

Rust library for building modular, scalable LLM applications and agentic workflows. It supports multi-turn streaming, multiple model providers, vector store integrations, and agent runtime abstractions.

A native macOS Postgres GUI that can draft SQL from natural-language questions, using either a configured cloud model or Apple’s on-device Foundation Model on supported Macs.

An open-source, local-first canvas for editing LLM context as a graph. It lets users branch, prune, merge, and inspect the exact context sent to a model, with a desktop app, web demo, and support for Ollama and OpenAI-compatible endpoints.

An open-source reinforcement learning training loop for LLMs, spanning small CPU-friendly experiments like CartPole to async RLVR runs on GPU clusters. The repo highlights a compact codebase, quickstart commands, task customization, and support for algorithms like REINFORCE, PPO, GRPO, and RLOO.

An infinite canvas that turns LLM conversations, readings, and notes into an editable thought graph. It supports human-in-the-loop workflows, cited passages, replayable dependencies, and local-first backups.

Jargo is a Go framework for building realtime voice agents over WebRTC, with streaming STT→LLM→TTS pipelines, turn-taking, barge-in, and pluggable model providers.

Turo is a CLI proxy that reduces prompt text to content words to save tokens for coding agents. The repo shows install options, command examples, and support for multiple agents and modes.

A Python package and CLI for checking how closely an AI-generated answer reuses its source text, with support for verbatim matching, subsequence matching, quotation verification, and pipeline gating.

An open-source project that runs full games of Diplomacy between 7 LLM-controlled powers, with negotiation rounds, validated orders, and saved transcripts for replay and analysis.

An autonomous research system that runs on a single workstation and aggressively checks its own claims with adversarial self-verification, replication, and calibration audits.

A desktop browser that hallucinates web pages on the fly with an LLM, including interactive apps and customizable “lenses” for different styles of generated sites.

A from-scratch ~1B parameter language model on Hugging Face with open weights, open data, training details, and loading guidance. The page is unusually candid about capabilities, limits, and evaluation.

Open-source SDK and libraries for building local-first, peer-to-peer AI apps. It supports local LLM inference, embeddings, translation, transcription, TTS, OCR, image generation, and an OpenAI-compatible API across desktop and mobile platforms.

A Python library that transpiles stateful imperative LLM workflows into optimizable DSPy programs, with typed outputs, self-correction loops, validation reporting, and prompt tuning support.

An AI API gateway/proxy in Go that sits between coding clients and upstream LLM providers, with request routing, secret redaction, context handling, MCP tool integration, and transparent SSE streaming.

tokentoll is a GitHub Action and CLI that scans Python and JavaScript/TypeScript code for LLM API calls, estimates spend, and blocks pull requests when cost or model-policy thresholds are exceeded.

Dream Server is an open-source stack that turns a PC, Mac, or Linux machine into a private AI server with local model inference, chat UI, voice, agents, workflows, RAG, and image generation. The page shows setup instructions, platform support, release notes, and a clear positioning for self-hosted use.

SwarmWright is a self-hosted multi-agent AI orchestration platform for designing, running, and inspecting swarms of agents from markdown and config. The page highlights Docker-based setup, topology-enforced agent graphs, human approval steps, local storage, and detailed audit logs.

A Go-based text editor built for LLM agents, with branching undo, conflict handling, audit logs, MCP support, and agent skills installation.