YC Paper Club: inference spend outpaces training 6x per year
Read original source →youtube.com
For every FLOP spent training a model, roughly six go to serving it each year, and getting inference right has produced over $220 billion in market cap in the last decade. The session walks through the full lifecycle of an inference request.
1 source
More stories today
AWS makes Continuum security agent generally available
AWS says Continuum finds and validates code vulnerabilities, checks them against the customer's AWS environment, and opens pull requests with proposed fixes. AWS published a headline benchmark claiming an 89% fix rate, though the report notes a catch.
The New Stack·1 hour ago
Reddit user ships GANG WIZARD game built with Claude
A Reddit user released version 1.0 beta of GANG WIZARD, a game they say consumed over 10 billion tokens to build. The poster claims the GPU usage was so heavy their local data center sent a letter.
r/ClaudeAI·1 hour ago
Mini-AGI continual-learning model trained from scratch on 8GB GPU
A Reddit user trained a dynamic continual-learning model called mini-AGI entirely from scratch for a month on an 8GB VRAM GPU. Benchmarks compare a "frozen" evaluation against a "best effort" mode where the model specializes on tasks as it moves through the eval dataset.
r/LocalLLaMA·1 hour ago
Emily Bender questions LLM peer review: scholarship is a human enterprise
Emily M. Bender·1 hour agoAtlantic CEO Nicholas Thompson: AI's economic impact lags capabilities
Speaking on Bloomberg This Weekend, The Atlantic CEO Nicholas Thompson said AI's broader economic impact continues to lag behind rapidly advancing capabilities. He called the tools powerful for businesses but a significant risk to media, amid a growing public backlash to the AI boom.
Bloomberg Technology·2 hours ago
Sponsored post: build an agentic software factory starting with the sensor
A sponsored contributed post argues the gap between top engineering teams' shipping velocity and the rest of the industry is now an exponent rather than a multiple, as agents accelerate deploys. It says the dividing line is no longer tooling but observability, framing the sensor as the starting point.
AbliteratedOutTheBox ports inference-time abliteration hook to Strata
AbliteratedOutTheBox is a llama.cpp fork that abliterates any loaded model live at inference, with no fine-tuning and no extra files. The author has now ported the hook to Strata, aiming to give abliterated behavior without large VRAM overhead.
r/LocalLLaMA·2 hours agoMeta paper: byte-level models start worse than token models but may end up better
Meta trained ~1B-parameter models on raw bytes instead of tokens; byte models start out worse but the paper predicts they end up better. The approach has the model read one letter at a time rather than using a tokenizer.
r/LocalLLaMA·2 hours ago
