EvalStop detects reward overoptimization in RLHF using world feedback

Arxiv CS.LG (machine learning)

EvalStop uses downstream eval metrics (world feedback) to detect and correct reward overoptimization in multi-tenant RLHF platforms. It addresses the proxy divergence problem identified by Gao et al. (2023).

rlhfreward-modeling

Arxiv CS.LG (machine learning)·Guilin ZhangChuanyi SunShahryar SarkaniJohn M. Fossaceca

7 days ago