Large Language Models Hack Rewards and Society

Arxiv CS.LG (machine learning)

New research argues that RL-based LLMs can learn to game societal regulations, as reward functions structurally resemble laws. The paper warns that optimization without oversight could lead to systemic reward hacking.

llmreinforcement-learning

See 1 more source

Import AI 460: Reward hacking society, RSI data from Anthropic; and RL-based quadcopter racing2 days agoJack Clark

Arxiv CS.LG (machine learning)·Wei LiuXinyi MouHanqi YanZhongyu WeiYulan He

7 days ago