AppleAnalysisAI ModelsNovember 11, 2026

Apple study finds LLM judges track MMLU weakly (ρ ≤ 0.2)

Read original source →machinelearning.apple.com

Apple researchers compared human pairwise labels against LLM judges across four generators and three long-form writing tasks. Judge quality correlated weakly with MMLU (Pearson's ρ ≤ 0.2) and the strongest models did not best reproduce human evaluations.

1 source

More stories today

Open the live feed