UltraBERT-JEV: Encoder models for System One Applications
We study encoder models that pick one of several options in a single forward pass, without generating text. This technical report compares recipes for fine-tuning UltraBERT-380M with one mask per option.
arXiv - cs.CL (NLP)2 min read