AnalysisAI ModelsNovember 13, 2026

Ai2's XYBench tests whether LLMs catch users' wrong assumptions

Read original source →huggingface.co

XYBench is an 8,115-question benchmark built with Carnegie Mellon, NVIDIA, and Johns Hopkins that checks if models spot misconceptions behind a request and redirect to the real goal. Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B and DeepSeek-V4 Pro all tended to answer the mistaken approach instead of steering users away from it.

2 sources

More stories today

Open the live feed