Vals AI's Rayan Krishnan: single benchmark scores hide a jagged frontier
Read original source →youtube.com
Vals AI builds benchmarks that frontier labs cite in their model cards, and Rayan Krishnan argues one aggregate score masks task-level variance. In finance tasks like tax and Excel LBO modeling, the best model changes by task, and a cheaper model can cost more and score lower.
People · Rayan Krishnan
1 source
More stories today
oMLX engine now available in LM Studio for MLX models
LM Studio·1 hour ago
Solo dev ships Steam FPS demo coded by Claude Code agents in 20 days
Ekin, a one-person studio in Turkey, launched the free Iron Curtain Call demo on Steam on 28 October, with Claude Code agents writing the code from his design documents. The post details what the agents could and could not do across the 12-chapter cartoon-noir FPS.
r/artificial·1 hour ago
ComfyUI take manager built after 1,200 MiniMax H3 renders
A short film made with MiniMax H3 took ~1,200 renders to yield 85 keepers, leaving hundreds of ComfyUI_00xxx_.mp4 files with no record of prompt or seed. The open-source take manager and editor tracks which prompt version produced each take.
r/ComfyUI·1 hour ago
Mistral researcher: major AI cyberattacks came from closed models
MTS·2 hours ago
Robot performs complex task in viral video
Thomas Wolf·2 hours ago
Jensen Huang discusses AI-era success in Astana Hub fireside chat
NVIDIA founder and CEO Jensen Huang spoke with Kazakhstan Deputy Prime Minister Zhaslan Madiyev on October 23, 2026, at an Astana Hub fireside chat on what qualities help people succeed in the age of AI.
Astana Hub·2 hours ago
Podcast: Zuckerberg says AI industry needs more positivity
A Sources Podcast episode discusses Mark Zuckerberg's call for the AI industry to be more positive. The clip is sponsored by Atlassian, Granola, and Mercury.
Sources Podcast·2 hours ago
Sonar demos Gitar agent catching bugs outside the diff
Sonar's Killian Carlsen-Phelan and Taylor Luttrell-Williams set up Gitar, a code change verification agent, on a sample ticketing app and ran three pull requests through it. Gitar flagged a bug not present in the diff itself: a buyer could refund a ticket after transferring it.
YouTube·2 hours ago