AnalysisDevelopersOctober 21, 2026

Open RK3588 NPU runtime runs Qwen3.5-4B at ~10 tok/s

Read original source →reddit.com

open_rknpu2_lm_infer is a local LLM inference engine for the Rockchip RK3588 NPU that skips Rockchip's proprietary userspace inference libraries, with the upper layer in Rust and NPU operators hand-coded. The author reports ~10 tok/s on Qwen3.5-4B, roughly 2x faster than the vendor SDK.

1 source

More stories today

Open the live feed