llm-judge

VoiceGremlin is a CI/CD testing tool for AI voice agents that places real phone calls, evaluates results through an API, and returns pass/fail outcomes with transcripts.

A Python pipeline that ranks 1,000 Show HN posts with an LLM judge and TrueSkill, then compares estimated merit against Hacker News points.

A blog post from Archestra analyzing Jev on 100 real AI tool calls, with benchmark results, failure analysis, and comparisons against other models used for annotating agent traces.