AnalysisAI ModelsNovember 13, 2026

Karpathy argues AI models are bloated by raw internet data

Read original source →youtube.com

In a Dwarkesh Patel interview, Karpathy says models bloat just compressing the internet, and wants smarter models to strip pre-training data down to what teaches thinking. Patel pressed on why distillation hasn't gone below a billion parameters in a decade.

People · Andrej Karpathy

1 source

More stories today

Open the live feed