AnalysisAI ModelsOctober 26, 2026

Ternary Nemotron 3.5 Lightning 30B-A3B GGUF runs on a 12GB GPU

Read original source →huggingface.co

The 1.58-bit ternary MoE quantization runs with 368K context on a 12 GB GPU, hitting 172 tok/s decode and 7000+ tok/s prefill. One GGUF file works in both stock llama.cpp and the PrismML fork.

1 source

More stories today

Open the live feed