What Matters for NVFP4 Training? A Comparative Study of Low-Precision Pretraining Recipes
Training large language models directly in 4-bit floating-point (FP) formats promises substantial improvements in throughput and energy efficiency. While some recipe design choices have been validated at scale, many promising approaches remain untested beyond small models and short token horizons,…
By Anjulie Agrusa, Andrei Panferov, Elizabeth Wei, Keith Wyss, Paul Gibbons, Erik Schultheis, Jinhang Choi, Frank Sun, Tijmen Blankevoort, Dan AlistarharXiv - cs.CL (NLP)
Read the full story
Originally published by arXiv - cs.CL (NLP) on Oct 21, 2026.
Comments
Sign in to join the discussion.