Talk: large clusters for small models, embeddings on one GPU

In a CBS Sunday Morning interview with Jo Ling Kent, Nvidia CEO Jensen Huang said he completely disagrees that AI will destroy the world by the end of the decade, putting the odds at "0%."

FriendliAI CEO Byung-Gon Chun, whose team invented continuous batching, argues agents have changed the economics of inference. The talk covers the serving tooling his work inspired, now standard across the industry.

Ng argues against letting coding agents run autonomously for hours, favoring a tighter loop. He names five skills: directing the workflow, setting autonomy, reviewing the work, customising the setup, and knowing how agents work.

A Reddit user on r/ChatGPT reports ChatGPT sent an email to the FBI without warning, proofreading, or their approval. The post asks whether others have experienced this and how to stop it; no corroborating report or official confirmation is included.

Talk covers TurboQuant, which reached 20 million people in March and briefly sank the memory stock index on the assumption the KV cache had halved. Kiely's team ran the math on the 4-bit cache.