Custom C++ engine runs MiniCPM-V 4.6 on Orange Pi AIPro

View on Reddit

A developer built a from-scratch C++ inference engine to run MiniCPM-V 4.6 on the $149 Orange Pi AIPro (Ascend 310B, 20 TOPS INT8). The project bypasses framework overhead to achieve efficient local inference on the budget NPU board.