Agent evals are stuck in the chatbot era, says Raindrop's Ben Hylak

Hylak argues agent eval advice is still written for the chatbot era, when answer sets were known in advance. His complaint: build the recommended thousand-example eval suite, switch harnesses, and 80% of it stops meaning anything.

Featured · Ben Hylak