Baseten's Philip Kiely on what's new in inference engineering

In a CBS Sunday Morning interview with Jo Ling Kent, Nvidia CEO Jensen Huang said he completely disagrees that AI will destroy the world by the end of the decade, putting the odds at "0%."

FriendliAI CEO Byung-Gon Chun, whose team invented continuous batching, argues agents have changed the economics of inference. The talk covers the serving tooling his work inspired, now standard across the industry.

Daniel Svonava of Superlinked argues a single mid-range GPU can embed half a million tokens per second in low tens of milliseconds, versus managed endpoints that cost orders of magnitude more and take hundreds of milliseconds.

Ng argues against letting coding agents run autonomously for hours, favoring a tighter loop. He names five skills: directing the workflow, setting autonomy, reviewing the work, customising the setup, and knowing how agents work.

A Reddit user on r/ChatGPT reports ChatGPT sent an email to the FBI without warning, proofreading, or their approval. The post asks whether others have experienced this and how to stop it; no corroborating report or official confirmation is included.