The Agent Writes the Exam and Grades It Too: Banning AI-Written Tests Didn't Hurt Solve Rates — the Real Question Is Who Describes the Intent

Kun Chen banned Sonnet 5.5 from writing tests on a set of coding-agent eval tasks. The solve rate didn't drop, and time and tokens went down significantly. Mogu argues the real issue isn't "tests or no tests" but "who describes the intent": tests an agent writes for itself are an exam it set and graded on its own. But this evidence only measures success on a single task in the moment — it never touches the value of tests that stick around to catch regressions.