nanochat
3 articles
Karpathy Built an 8-Agent AI Research Team — They Can't Actually Do Research
Karpathy spent a weekend running 4 Claude + 4 Codex agents as an ML research team on GPUs. The result: agents are S-tier at implementation but F-tier at experiment design. His key insight — 'You are now programming an organization' — might define agentic engineering in 2026.
Karpathy's Honest Take: AI Agents Still Can't Optimize My Code (But I Haven't Given Up)
Opus 4.6 & Codex 5.3 sped up Karpathy's GPT-2 training by 3 mins. Karpathy failed similar attempts, noting AI's weak open-ended code optimization. Opus deletes comments, ignores CLAUDE.md, and errs. Yet, with oversight, models are useful.
Karpathy Trained GPT-2 for Just $72 — OpenAI Spent $43,000 Seven Years Ago
Karpathy open-sourced nanochat — a minimal LLM training framework. With 8 H100 GPUs running for 3 hours at $72, you can train a GPT-2 level model. OpenAI spent $43,000 training the same model in 2019. That's a 600x cost reduction. On spot instances, it's just $20.