AnalysisDevelopersJuly 11, 2026
Automated evals compared to human annotations in new study

The study evaluated 100 human-annotated traces against several automated eval systems. Results highlight discrepancies between automated and human judgments.

The study evaluated 100 human-annotated traces against several automated eval systems. Results highlight discrepancies between automated and human judgments.