Qwen 30b MoE runs at 30 tokens/s on 6GB VRAM
A user posts that Qwen 30b MoE hits 30 tokens/s on an RTX 3050 6GB, using expert offloading to 22GB DDR4 to target Hermes with context above 60k.
1 source
Daily brief
Get tomorrow's AI brief in your inbox
More stories today
- GOP panics over Big Tech ties as Trump shifts on AI regulation
- Ethan Mollick: Claude's skill creator beats ChatGPT for reusable skills
- Aident Loadout gives agents 27,000+ tools and logs every action
- Etched gains sizable fan base for AI inferencing computers
- Corbell generates technical specs from repository knowledge graphs