CoreWeave's Sitanshu Gupta on scaling inference from MVP to trillion-parameter workloads
On agentic requests, 80-90% of input tokens are identical to the prior request, making prefill the most expensive operation in an inference stack — which is why cached input tokens are priced far below fresh ones.
People · Sitanshu Gupta
1 source
More stories today
Nvidia: AI security is an engineering problem across the agent stack
Nvidia argues AI security needs defined requirements, enforceable controls, named owners and evidence protections work, applied across models, harnesses and runtime environments. Example: an agent reading malicious instructions in an attached document should be blocked by network policy, with protected logs capturing the tool call and destination.
Nvidia AI Blog·1 hour ago
Unitree Dex5-S dexterous hand offers 22 degrees of freedom
Unitree's Dex5-S robotic hand packs 22 degrees of freedom and costs $6,500 per hand, per a Reddit post in r/Singularity. The Wuji Hand 2, a 20-DoF alternative, is priced between $16,000 and $20,000.
r/Singularity·1 hour ago
jevals replaces LLM judges with typed Jev decisions
Openlayer's jevals swaps free-form LLM-as-judge scoring for typed Jev decisions in evals. Posted as a Show HN project on GitHub.
Hacker News·1 hour agoHugging Face releases tokenizers v1
Hugging Face shipped tokenizers v1, a major version of its text tokenization library, with the release post focused on encode, decode, and scaling measurements.
Hugging Face Blog·1 hour ago
V7 uses GPT-5.6 to give AI agents institutional memory
V7 turns scattered company files into context agents can use for complex, source-linked work, built on GPT-5.6.
OpenAI Blog·1 hour ago
Engineer Travis adds AI voice to Big Mouth Billy Bass
Vaibhav Sisinty·1 hour ago
Databricks DevHub launches with agentic app templates
Databricks·1 hour ago
Healthcare AI needs "provenance of meaning" before learning from archives
Guest article by Heather Fricke argues archived clinical data can encode old access patterns, reimbursement incentives, and unequal treatment that AI will faithfully learn without understanding why. She cites NIST's healthcare AI work on data quality and calls for adding provenance of meaning to governance.
Healthcare IT Today·2 hours ago
