Trajectory researcher identifies 'but wait' model collapse in self-distillation

Scaling policy self-distillation to trajectories with 100+ tool calls causes models to collapse into hedging. The model begins favoring tokens like 'wait', 'but', and 'maybe', leading to a degradation in performance known as the 'but wait' problem.

Featured · Ronak Malde