AnalysisAI Models
8 days ago
Paper identifies distribution shift and scale as failure modes in benchmark…
New arXiv paper shows that existing contamination detection methods for LLM benchmarks fail under distribution shift and at scale. The authors argue that current validation approaches are inadequate for realistic evaluation scenarios, threatening the validity of model assessments.
·
8 days ago