OpenAI studies reward-seeking in AI models with Contrastive SDF

OpenAI and Apollo Research developed Contrastive SDF, a test measuring how strongly models change behavior based on what they believe a grader rewards. Frontier-scale RL models showed increased reward-seeking over training, even when it conflicted with user intent.