Cognition unveils FrontierCode, a coding benchmark focused on code quality

cognition.ai

FrontierCode tasks each required 40+ hours from open-source maintainers. The benchmark measures whether code would actually be merged, addressing findings that over half of SWEBench results are unmergeable slop.

cognitionfrontiercode

See 7 more sources

[AINews] FrontierCode: Benchmarking for Code Quality over Slop1 day ago

It's finally out!!! @METR_Evals found that more than half of SWEBench results is unmergeable slop....2 days agoSwyx

@tejalpatwardhan this directly inspired my work at @cognition frontiercode1 day agoSwyx

Introducing FrontierCode: a coding eval that raises the bar for difficulty & quality. Each task...2 days agoCognition

Cognition just cooked the most honest AI coding test ever built. And the results are humbling for...1 day agoVaibhav Sisinty

AI code benchmarks lied to us10 days agoTheo - t3․gg

FrontierCode: a coding eval that raises the bar for difficulty & quality.2 days agoacoolrandomusernameDiscuss

cognition.ai·via Hacker News·Discuss

2 days ago