DeepSeek-V4 paper: LLM retries bias results toward shorter text

Analyzing 100,000 poems generated with DeepSeek-V4-Flash for $6.06, the author found 73.2% were haikus (15 words) vs 11.5% novel chapters (503 words); simulated 10% failures hit long requests four times harder (31% vs 7.4%). The paper warns that retrying interrupted long requests often yields shorter replacements, skewing benchmarks.