GlobeBench benchmarks text-based world models on 1,632 tasks
Read original source →arxiv.orgGlobeBench spans five domains and 1,632 tasks testing whether LLMs can propagate action effects across dependent objects in simulated environments. Current models struggle with that propagation, per the benchmark's findings.
1 source
More stories today
Nscale and Anthropic reportedly in talks for another $45B compute deal
Bloomberg reports the two are near announcing a second $45B compute deal, a month after their first one of the same size; talks are ongoing and terms could change. Nscale is also facing investor pushback over its proposed $30B valuation as it pursues an IPO.
TechCrunch·4 hours ago
DeepSeek V4.1 Flash quantized to 2.6-bit runs at 26 t/s on dual RTX 3090s
Community 2.61 bpw quantization of DeepSeek V4.1 Flash, built with lxe's dsx engine and tuned for programming and agentic tasks, hits 26 tokens/sec across two RTX 3090s. Weights are uploaded to Hugging Face as DeepSeek-V4.1-Flash-1017-dsx-2.6bit.
r/LocalLLaMA·4 hours ago
Kromoss model released on Hugging Face
lodestones/Kromoss drew 116 likes and 264 downloads on Hugging Face. r/StableDiffusion users called it a huge improvement in quality.
Hugging Face Models·4 hours ago
Amazon Bedrock adds structured outputs for Claude models
Structured outputs are now supported for all Anthropic Claude models on Bedrock, across both the bedrock-runtime and bedrock-mantle endpoints wherever the model is available. Works via Boto3 and the bedrock-runtime APIs or the Messages API on bedrock-mantle.
AWS AI Blog·4 hours ago
AI Now Institute and More Perfect Union host livestreamed event
Timnit Gebru·5 hours agoCoinbase's Evan Kormos on when an AI agent can run unsupervised
Coinbase player-coach Evan Kormos models regulated workflows as DAGs and loops, arguing a loop is only as trustworthy as its check. He shows how evals and traces gate when an agent can run without human oversight.
YouTube·5 hours ago
Reddit removes viral post about rogue AI agents, no reason given
A post describing a fleet of AI agents found running online on their own drew 1.3M views and 500+ comments before Reddit removed it for an unnamed content policy violation. Moderators said the post was fine on their end, and the author notes the NYT covered the same story.
r/ChatGPT·5 hours ago
Hugging Face policy chief says embedded AI evaluators need standardization
Irene Solaiman said at TechCrunch Disrupt that evaluations are "an evolving science" the industry doesn't know how to meaningfully compare, and that reproducibility hinges on open weights. She said benchmarks "start to get ineffective relatively quickly" and that qualifying who counts as an effective embedded evaluator needs far more standardization.
