AI labs report benchmark scores 6.4% higher than independent testers
Read original source →reddit.com
An analysis of 1,629 published benchmark results matched 611 comparisons across 180 models and 103 benchmarks, finding labs' self-reported scores average 6.4% above independent testers. The data comes from whatmodel.sh, an open-source AI leaderboard.
1 source
More stories today
Claude users complain automated safety checks trigger refusals
A r/ClaudeAI poster says Claude refused to complete work after a single automated safety-check flag, calling the guardrails unusable over two nights of use. The post stresses it is not calling Anthropic or Claude a scam, only criticizing the refusal behavior.
r/ClaudeAI·1 hour agoPritzker: AI could help us live to 100 or threaten us like nukes
Illinois Gov. JB Pritzker tells Decoder he sees AI as either a longevity breakthrough or a nuclear-scale threat, and discusses data centers, privacy, and Trump ahead of the midterms.
The Verge·1 hour ago
Nace AI releases Drex Max decision model with image and video input
TestingCatalog News·1 hour ago
California lawyers weigh SB 574 generative AI disclosure rule
Business and Professions Code section 6068.1 will require California attorneys to disclose use of generative AI in documents, per a r/LawFirm thread. The poster notes most of SB 574 restates existing duties — confidentiality, verifying citations and accuracy.
r/LawFirm·1 hour agoReport: Anthropic prepping new model that spots broader bug classes
Kimmonismus·1 hour ago
Talk compares GPT-5.5, Kimi, DeepSeek and Mistral in app-level model swaps
AI Engineer posted a workshop session arguing benchmarks don't predict how a model behaves inside a real app, so swapping models in and testing does. Speaker Pamela Fox, a principal cloud-focused engineer, leads the comparison across GPT-5.5, Kimi, DeepSeek and Mistral.
YouTube·1 hour ago
Databricks expands New York City R&D hub at 5 Bryant Park
Databricks is doubling down on its New York R&D hub at 5 Bryant Park, opened earlier this year, positioning the office as a premier AI talent hub.
Databricks Blog·1 hour ago
Hacker used AI agents to breach 100+ sites, including John Deere
Norforge found the attacker orchestrated agents via exposed instances of Cairn, an open-source "general-purpose problem-solving engine," hitting companies in the US, Canada, Australia and Japan. Successful breaches took about 23 hours, and Norforge flagged more than 20 exposed Cairn instances as likely malicious.
