Automated evals compared to human annotations in new study
The study evaluated 100 human-annotated traces against several automated eval systems. Results highlight discrepancies between automated and human judgments.
1 source
Daily brief
Get tomorrow's AI brief in your inbox
More stories today
- Liquid AI open-sources Pipette benchmarking suite for on-device models
- wikiHow sues OpenAI over copyright infringement in AI training
- Claude Code 2.1.246 adds Auto mode tab, Bash wildcard warning
- Korean AI startup Wrtn raises funds at $870M valuation
- Podcast: Google DeepMind's Vivek Natarajan on AI in healthcare