Wired: when to use AI chatbots for web search, and when not to

tandem.cpp, a llama.cpp patch set, runs Qwen3.8-Flash-Next at 97 tok/s on an RTX 4090 with 64 GB DDR4, claimed 3.3× faster than llama.cpp. It keeps hot experts on the GPU and offloads the rest to CPU, targeting single-user decode for any MoE model llama.cpp supports.

Berth hands GPU access to applications through leases instead of letting every app compete for VRAM, per its GitHub repo. It is pitched for live-swapping workloads like llama.cpp, ComfyUI and Strata.

A r/LocalLLaMA post proposes that community members build their own private quality benchmarks, especially to compare models across quantization levels, rather than fixating on tokens-per-second throughput.

After an October ICO report said foundation-model training processes personal data, Meta, OpenAI and Apple all offer UK users opt-out routes. Meta asks third-party objectors to supply the prompt and a screenshot of the output; the ICO says technical means of checking whether a person's data sits in a trained model are currently limited.