Arena launches Agent Arena for real-world agent evaluation

LMSYS Chatbot Arena Blog

Agent Arena uses millions of live user sessions with real tasks to evaluate agent performance, moving beyond static benchmarks. It focuses on causal evaluation of agents performing tool use and long-horizon tasks.

agents

See 8 more sources

Thrilled to see @arena launch Agent Arena, a major step forward in evaluating frontier AI agents....5 days agoHasan Toor

MacArena: Benchmarking Computer Use Agents on an Online macOS Environment3 days agoVictor Muryn, Maksym Shamrai, Sofiia Mazepa, Yehor Khodysko

Do More Agents Help? Controlled and Protocol-Aligned Evaluation of LLM Agent Workflows5 days agoYuhang Fu, Ruishan Fang, Jiaqi Shao, Huiyu Zheng, Zhengtao Zhu, Bing Luo, Tao Lin

Beyond Goodhart's Law: A Dynamic Benchmark for Evaluating Compliance in Multi-Agent Systems2 days agoYiyang Zhao, Zhuo Zhang, Qingxuan Le, Lizhen Qu, Zenglin Xu

The Sim-to-Real Gap of Foundation Model Agents: A Unified MDP Perspective2 days agoXiaoou Liu, Tiejin Chen, Weibo Li, Xiyang Hu, Hua Wei

Online Agent-as-a-Judge: Situation-Generating Evaluation for Interactive Agents2 days agoHyogon Ryu, Jeonghwan Kim, Yewon Lim, Chaeun Lee, Jeongwook Kim, Donghoon Ham

Agent Arena: Causal Evaluation of Agents in the Real World6 days agoArena Team

Is Grep All You Need? How Agent Harnesses Reshape Agentic Search1 day agoAnon84Discuss

LMSYS Chatbot Arena Blog·Arena Team

9 days ago