3Blue1Brown explains cross-entropy in 'Compression is Intelligence Part 2'

Explains cross-entropy loss as a natural consequence of compression, tracing the idea from information theory to LLM training. Video is part of the 'Compression is Intelligence' series by Grant Sanderson.