GRPO training instability across scales and new stabilization methods

A Reddit experiment shows the same GRPO recipe on three from-scratch LLMs (353M/316M/672M) yields three different outcomes with no clean scale relationship. Two new papers propose fixes: RTPO (Reverse-Turn Policy Optimization) stabilizes multi-turn agentic RL, and GUPO (Gradient Uncertainty-aware Policy Optimization) improves GRPO post-training.