AnalysisDevelopersNovember 17, 2026

FrontierHarness eval: 64% of coding agent inference spend went to failed runs

Read original source →runta.com

Runta and Baseten ran 27,280 agent runs across 83B tokens, 22 harnesses, 5 models and 62 tasks. Best-value harness varied by model: Codex CLI ranked 1st with GLM-5.3 but 16th with Kimi-K3; OpenCode v2 had the lowest cost per task at $0.561.

1 source

More stories today

Open the live feed