gpt-2
1 articles
Karpathy Trained GPT-2 for Just $72 — OpenAI Spent $43,000 Seven Years Ago
Karpathy open-sourced nanochat — a minimal LLM training framework. With 8 H100 GPUs running for 3 hours at $72, you can train a GPT-2 level model. OpenAI spent $43,000 training the same model in 2019. That's a 600x cost reduction. On spot instances, it's just $20.