LaunchAI Models
Jun 16, 3:04 PM
Subquadratic AI releases SubQ-1.1-Small with sparse attention up to 12M tokens
At 1M tokens, SubQ-1.1-Small requires 64.5x less compute than dense attention and runs 56x faster than FlashAttention-2. The model achieves near-perfect needle-in-a-haystack retrieval across up to 12 million tokens. It is being deployed with select design partners, with broader availability expected later this year.