llm

A fully local voice-controlled 3D sandbox where a sassy robot turns spoken commands into voxel builds. The repo explains the stack, runtime requirements, service contracts, controls, and tests.

Open-source harness for measuring how consistently hosted LLM APIs return the same output across repeated runs and over time, with raw transcripts, scoring, and a static leaderboard generator.

A technical primer that explains how LLM inference works, covering weights in GPU memory, prefill vs. decode, the KV cache, and batching.

LayoutLens is an AI-assisted UI testing tool that combines natural-language visual checks with deterministic WCAG audits using axe-core and Playwright. It supports keyless accessibility scans, hybrid LLM-grounded checks, a Python API, and a CLI for running tests in CI.

StateM is a Python CLI for managing long-running AI agents with explicit states, transition checks, runtime history, and resume/compaction support. The repo shows installation, quick-start commands, and a concrete coding-agent runbook example.

CLI and GitHub Action that scans repos, logs, and CI artifacts for leaked LLM reasoning-trace blocks from Anthropic, OpenAI, and Google.

A live demo and write-up showing a 3.16M-parameter INT4 transformer running entirely in on-chip memory on a Xilinx Kria KV260 FPGA, with measured token rates and hardware details.

Symbio is a local AI assistant that learns from user corrections, builds training data from notes, and can fine-tune itself over time. It supports file operations, sandboxed shell commands, RAG, a Telegram bot, and a demo with MoA and skill adapters.

Minimal LLM post-training experiments on a single 8GB GPU, covering SFT, DPO, and GRPO with a focus on forgetting, seed variance, and RL behavior.

A lightweight Flask-based LLM proxy that forwards Ollama, OpenAI, and llama.cpp-style requests to NVIDIA’s OpenAI-compatible API, with streaming support, retries, caching, auth, and usage stats.

CLI utility that counts LLM tokens in files or directories, with support for stdin, recursive scans, file-type grouping, profiles, JSON output, and multiple encodings.

Google’s announcement for new Gemini models focused on agentic workflows, coding, multimodal work, and cybersecurity use cases. The page includes performance claims, pricing, safety notes, and example applications.

A local-first AI reading assistant for Chrome that rewrites pages into customizable briefs, supports reusable reading recipes, and can run with local or hosted models.

A local Rust-plus-Python middleware layer for LLM apps that claims to reduce token usage and add in-process security checks before prompts are sent to a provider.

An open-source tool that watches Playwright failures, inspects the live page, and opens a GitHub PR with a proposed fix.

AI-powered security scanner for modern web apps that combines SAST, DAST, and LLM-assisted code review in a single command. The repository includes install steps, quick-start commands, supported stacks, and examples for scanning live URLs or source code.

Soup is a Python library that routes Agent Skills into LLM prompts, injecting only the skills relevant to each request. It supports local skills, remote repositories, and multiple providers including OpenAI, Anthropic, Gemini, Ollama, LiteLLM, and LangChain.

Local-first Python SDK for AI agents that compresses web pages before LLM use, flattens Shadow DOM, builds action maps, and extracts structured data from tables, forms, and lists.

An open-source, self-hosted AI DevOps pipeline that ingests GitHub webhooks, generates code patches with an LLM, runs them in network-isolated Docker sandboxes, and posts validated fixes back as PR comments.

Cachet is a local semantic cache for LLM APIs that sits in front of OpenAI- or Anthropic-compatible endpoints and serves repeated or rephrased requests from cache. It includes a dashboard for hit rate and estimated savings, supports streaming responses, and is implemented as a single Rust binary.

Open-source MCP server and CLI that lets an LLM control hardware workflows like compiling, flashing, and reading serial output for firmware development and device diagnostics.

Rudi is a Python tool for LLM memory management that replaces a growing chat transcript with a causal dependency graph, aiming to keep token usage flat across long sessions. The README shows benchmarking results, fold behavior, and example usage with get_slice, store_decisions, and run_turn.

Go packages for building agents in Go, with a streaming runner, tool dispatch, guardrails, compaction, provider adapters, and verified completion helpers.

Open-source MCP server that exposes SQL databases as tools and prompts for AI agents, with support for MSSQL and PostgreSQL, read-only defaults, and local or remote transport.