Unsupervised Multimodal Reward Modeling

Vision-language reward models are trained on annotated preference pairs, which are expensive to collect, can disagree, and are ranked by benchmark accuracy.