# testing

Garry Tan argues the real breakthrough in AI coding is not speed. It's turning tests, docs, and evals into a forward-only quality ratchet, so every change locks in what the team learned and makes the codebase harder to silently degrade.

Claude Code has 512K lines of TypeScript, 64K lines of production code, and zero tests. The sharper question is not why Anthropic skipped tests, but why it did not use its own AI coding tool to write them. Can the same brain write and grade the exam?

You use unit tests to check your code and CI to protect your pipeline. But who checks your AI? Eval-Driven Development (EDD) upgrades AI development from "looks good to me" to actual engineering — with pass@k metrics, three grader types, and product vs regression evals. This is TDD for the AI era.

Simon Willison's First Run the Tests pattern is four words with three effects: the agent learns the test command, gauges codebase size, and shifts into a test-maintenance mindset. Tiny instruction, large behavioral nudge.

Simon Willison introduces Agentic Manual Testing: let AI agents manually operate code and UI like humans do, catching bugs that automated tests miss. With Playwright, Rodney, and Showboat, the 'tests pass but it's broken' nightmare becomes a thing of the past.

The philosophy behind 1,086 tests. Why tests matter more than code review in the AI era. How to use tests as specs. The changing role of a Tech Lead.

Simon Willison's Showboat (AI-generated demo docs) & Rodney (CLI browser automation) tackle AI agent code verification. How to know 'all tests pass' means it works? Agents were caught cheating by directly editing demo files. #AI #OpenSource