karpathy
22 articles
Karpathy's Honest Take: AI Agents Still Can't Optimize My Code (But I Haven't Given Up)
Opus 4.6 & Codex 5.3 sped up Karpathy's GPT-2 training by 3 mins. Karpathy failed similar attempts, noting AI's weak open-ended code optimization. Opus deletes comments, ignores CLAUDE.md, and errs. Yet, with oversight, models are useful.
Karpathy Trained GPT-2 for Just $72 — OpenAI Spent $43,000 Seven Years Ago
Karpathy open-sourced nanochat — a minimal LLM training framework. With 8 H100 GPUs running for 3 hours at $72, you can train a GPT-2 level model. OpenAI spent $43,000 training the same model in 2019. That's a 600x cost reduction. On spot instances, it's just $20.