Talk: a 100% eval score means the eval is wrong
Read original source →youtube.com
AI safety researcher Ramakrishnan Lokanathan argues high-stakes AI evals should test every step of a workflow, not just the final answer, and pair automated checks with human review. A perfect score signals a flawed eval rather than a solved task.
People · Ramakrishnan Lokanathan
1 source
More stories today
Reddit user fine-tunes Krea 2 for realism, loses other abilities
Seven training runs produced a checkpoint with convincing photographic output but that appeared to forget every other capability. The experiment started from an old hard drive the poster found.
r/StableDiffusion·2 hours ago
Baby tracking app built with Claude Code in 5 weeks
Vaibhav Sisinty·2 hours ago
Northstar Security exits stealth with $7M for AI security agents
Wing Venture Capital led the $7M seed, with MVP, Outliers and veterans Gerhard Eschelbeck, Renaud Deraison and Justin Somaini also investing. Its Security Context Graph connects to 70+ tools including Okta, GitHub, Workday, AWS and Google Cloud, and exposes the graph to external agents via MCP.
SecurityWeek·2 hours ago
Chatbots Are Leading Hikers Astray, With Dangerous Consequences
Three 20-year-olds had to be rescued on Mount Shasta after planning their trip with Google's Gemini; a US Forest Service official said they "kind of turned their brains off." A 16-year-old was helicopter-rescued from Crown Mountain in BC after using Claude to plan his hike.
Wired·3 hours ago
109M-parameter model beats 72B models on image-text alignment
A 109M-parameter model reportedly outperforms models up to 72B parameters at image-text alignment while using 42× less training data, per an alphaXiv paper share.
r/LocalLLaMA·3 hours agoCompound AI rebuilt Office-style tools for autonomous finance agents
Yijia Liang, a former finance analyst, founded Compound to build autonomous AI analysts for institutional investors. The talk explains why personal agents work in the background but finance needs a from-scratch rebuild of Excel- and PowerPoint-style tooling.
YouTube·3 hours ago
Reddit user shares Halloween image made with Pika H3 Flex ref LoRA
A r/StableDiffusion post shows a Halloween-themed image generated with a Pika H3 Flex reference LoRA combined with the poster's own upscaler pipeline.
r/StableDiffusion·3 hours ago
YouTube tutorial walks through Claude's new dashboards
A 12-minute Ben AI walkthrough covers Claude's new dashboards feature, with a free prompt pack offered for every prompt used in the video. No product details, benchmarks, or availability dates appear in the listing.
YouTube·4 hours ago