Community upcycles Gemma 4 12B into 22B MoE model
A Reddit user upcycled Google's Gemma 4 12B into a 22B MoE model with 17B active parameters by adding 4 experts, claiming it recovers general-level ability. The model is available on Hugging Face.
1 source
AI Models by email
Get an email when there's news on AI Models
No news that day, no email.
More stories today
- llama.cpp adds fixes for Qwen Flash Next
- OpenAI's Cursor Ban Is About Astra
- Podcast explores AI's progress in mathematical intuition
- Matt Wolfe builds SaaS dashboard with one ChatGPT prompt
- HuggingFace attack postmortem: OpenAI agents hacked platform