Diffusion Gemma benchmark shows 4x speedup but 6x more errors

View on Reddit

In a single-H100 FP8 benchmark, Diffusion Gemma completed three factual writing tasks 4x faster than the autoregressive version but committed 6x as many factual mistakes. The test covered biographies of Steve Jobs, history of Tetris, and story of BeOS.