Ollama and Together AI talk evaluating open models on production traffic

A Reddit poster describes forcing themselves to use Microsoft Copilot at work and calls it a "wake up call," arguing Microsoft has the most agent-worthy workplace surfaces but shipped a weak product.

SentinelOne engineer Carter Church used GPT-6 Astra to crack a coded 1809 letter to French General Auguste de Marmont that had gone unread for 217 years. The letter held 1,300 encrypted elements and 155 distinct characters, of which researchers previously knew only 33.

A Reddit harness gave Claude models one web_search tool and 40 questions whose answers postdate training cutoffs, without executing the tool. Opus chose to search correctly 79 of 80 times; Sonnet 5 with no system prompt answered that Norway's King Harald V is still alive.

NVIDIA Developer session features Artificial Analysis explaining how it evaluates open models such as Nemotron and how they compare against the open-weights field. No benchmark figures or results are given in the available text.

The framework targets evaluation costs for self-improving coding agents by replacing full test-suite scoring with an LLM judge. MIT and Sakana AI collaborated on the method.