Before You Trust the Score, Suspect the Test: A Reading Guide to Anthropic's Agent Evals Playbook

Opus 4.5 went from 42% to 95% on CORE-Bench without any change to the model. What changed was a broken test and an overly restrictive harness. The part of Anthropic's agent evals guide most worth reading is its string of cases where the score lied, plus the advice it keeps coming back to: read the transcripts before you trust the number.