Sliding-window attention beats linear attention on long-context tasks
Read original source →arxiv.org
A new arXiv paper finds sliding window attention with sinks outperforms post-trained linear attention on long-context tasks without retraining, offering cheaper and more reliable inference. Tested on Needle-in-a-Haystack and BABILong.
How this story unfolded
1 day · 1 report · 2 community posts · 3 of 4 shown
More stories today
Mark Wahlberg open to AI clone carrying on his legacy
Speaking at TechCrunch Disrupt, Wahlberg said he may use AI to enhance stadium settings in his upcoming Pete Rose biopic, but not to generate scripts or replace actors. He said he'd be open to the CAA Vault's scanning tech so his children could control his likeness.
TechCrunch·1 hour ago
MobiHealthNews urges investment focus on AI governance
MobiHealthNews argues healthcare organizations should direct investment toward AI governance. No specific figures, frameworks, or named programs are detailed in the available excerpt.
MobiHealthNews·1 hour ago
Gemini 4 Argon praised for 3D modeling in Blender
AshutoshShrivastava·1 hour ago
Open-source robotics demo surfaces on r/Singularity
A Reddit r/Singularity post links to a YouTube video with the caption "We got open-source robotics now." No project name, hardware specs, or repository details are given in the post.
r/Singularity·1 hour ago
Hinton's book "Smarter Than Us" gets earlier publication date
Geoffrey Hinton·1 hour ago
Meta's new terms may allow it to train AI on news sites for free
Meta's updated commercial terms, effective October 30th, authorize automated scraping of domains "associated" with business accounts and let Meta use that content to "develop, improve, research," and "train" its AI products. Legal experts tell Nieman Lab the terms could let Meta argue its commercial terms supersede robots.
The Verge·1 hour ago
DNI Jay Clayton warns companies to be 'wary of' AI espionage
Clayton said on CNBC's "Squawk Box" that AI "amplifies" existing cybersecurity risks to U.S. digital infrastructure. He framed AI-enabled espionage as a reality companies need to guard against.
CNBC Technology·1 hour ago
Hugging Face to livestream a new open weight model
Hugging Face announced a live stream with the team behind a brand new open weight model, covering what it's like to train frontier models. The stream also features a contest with $20k in cash and Hub credits as prizes.
Hugging Face·1 hour ago