agent-evaluation

Mirrors is a staging and replay environment for AI agents. It rebuilds the systems an agent calls, replays real sessions against prompts, tools, and model changes, and surfaces regressions before deployment.

A local-first Python tool that audits Git history to quantify rework in AI-generated engineering work, including PR correction rates, churn clusters, superseded work, and line survival.

Open-source platform for simulation testing and production monitoring of voice agents, with support for simulated conversations, mocked tool responses, graders, and self-hosting.

An interactive site showcasing short AI-built arcade games and a model bench that compares coding models across the same tiny game constraints.

A prompt-injection capture-the-flag arena for testing how AI agents behave under attack. Users can play challenges, create guarded agents, add optional guards and sandboxed tools, and track scores on a leaderboard.

A research post from Prime Intellect comparing 153 autonomous runs across 18 frontier models on a nanoGPT optimizer speedrun. It presents the setup, harness, results, and discussion around autonomous AI research performance.

A DeepSeek Harness plugin that uses an LLM to verify and rank candidate outputs with select, compare, track, and rollout tools.

A blog post describing an evaluation harness for comparing agentic coding tools and prompts across realistic SWE tasks, with token-cost results and model-specific behavior notes.