AnalysisAI ModelsNovember 7, 2026

Karpathy: curated data could shrink AI's cognitive core to 1B parameters

Read original source →youtube.com

Karpathy argues current models waste capacity memorizing unneeded data like SHA hashes because training data isn't curated; strip that out and the "cognitive core" could run on a billion parameters. The mechanism he cites is distillation from a very large model or large compute.

People · Andrej Karpathy

1 source

More stories today

Open the live feed