New papers probe LLM-as-a-judge reliability and benchmark validity

Multiple arXiv papers examine LLM-as-a-judge evaluation, including JudgeArena, a unified framework for reproducible LLM-judge evaluation, and studies on scoring bias, frozen interfaces, and temporal leakage in backtesting. One paper finds standard contamination checks are uninformative, as four flagship models fail them on questions they cannot have memorized.