SimHack: Predicting the onset of detectable reward hacking before RL training
When training LLMs to code with reinforcement learning, reward hacking is often detectable but difficult to prevent, since (a) environment hardening is increasingly difficult as training environments grow more complex; and (b) training-time interventions incentivize models to obfuscate hacking…
arXiv cs.LG2 min read