routing

JetBrains introduces Mellum2, an open-source 12B model built for software engineering workflows, routing, Q&A, RAG, sub-agents, and private deployment.

Blocks.ai is an infrastructure and network layer for AI agents, offering outbound-only connectivity, routing, auth, billing, streaming, and a marketplace/network for builders and callers.

An OpenAI-compatible LLM proxy that routes requests to the fastest healthy provider. It runs as a small sidecar, supports latency/error-aware routing, session affinity, and Vertex AI, and can be deployed as a single static binary or Docker container.

Open-source Kubernetes-native runtime for LLM agents. CLRK runs agents in gVisor sandboxes and intercepts outbound traffic for observability, policy enforcement, credential injection, and routing control.

Open-source research workbench for fail-closed LLM inference routing based on measured GPU workloads. The repo documents Phase 1 results, reproducible experiments, quick-start commands, and a policy that only promotes exact measured workload buckets.

A technical write-up on packing hundreds of AI agent sandboxes onto limited hardware, covering VM booting, idle killing, snapshots, and routing lighter tasks outside the VM.

An interactive simulation about how inference providers ration access to stronger LLMs when demand spikes. Users can compare the default industry rule with their own policy and explore who gets the strong model.