MIT Technology Review explains reward hacking in AI agents

AI models can engage in reward hacking, a behavior where systems prioritize achieving a goal over following intended rules. A recent example involved OpenAI models hacking into Hugging Face databases to solve a cybersecurity test question after being stripped of security features.