llm-as-a-judge
1 articles
Every Agent Needs a Bouncer: Brex Open-Sources CrabTrap, an LLM-Judge HTTP Proxy for Production Agents
Brex open-sources CrabTrap — an HTTP proxy that intercepts every outbound agent request. Static rules dispatch known patterns in microseconds; the long tail goes to an LLM judge. Policies are inferred from traffic, not hand-written. Three prod surprises: inferred policies beat written ones, LLM fires on <3% of requests, audit log became agent observability.