Claude Fable 5.5 traces spotted in Azure and LiteLLM configs

tandem.cpp, a llama.cpp patch set, runs Qwen3.8-Flash-Next at 97 tok/s on an RTX 4090 with 64 GB DDR4, claimed 3.3× faster than llama.cpp. It keeps hot experts on the GPU and offloads the rest to CPU, targeting single-user decode for any MoE model llama.cpp supports.

Berth hands GPU access to applications through leases instead of letting every app compete for VRAM, per its GitHub repo. It is pitched for live-swapping workloads like llama.cpp, ComfyUI and Strata.

Chatbots like Claude, ChatGPT and Gemini blend traditional googling with training-data recall, which works for simple look-ups like the Empire State Building's height but can pick the wrong source for obscure queries. Wired notes Google is pushing AI Mode in its own search engine.

A r/LocalLLaMA post proposes that community members build their own private quality benchmarks, especially to compare models across quantization levels, rather than fixating on tokens-per-second throughput.