Apple introduces ARBITRAGE for efficient LLM reasoning

ARBITRAGE is a step-level speculative generation framework that uses a lightweight router to dynamically choose between draft and target model outputs. It reduces inference latency by up to 2x on mathematical reasoning benchmarks while maintaining accuracy.