FARO2 min read
FARO: Function-Space Adaptive Return-to-Risk Optimization via Cross-Block Equal-Risk-Contribution Budget Allocation
Training a large language model is compute-bound, and current recipes divide the work between optimizers by hand: orthogonalized momentum for attention and MLP matrices, Adam-style updates for the rest.
By Xiaojun Wu, Cehao Yang, Xuhui Jiang, Chengjin Xu, Jia Li, Jian GuoarXiv cs.LG
Read the full story
Originally published by arXiv cs.LG on Nov 2, 2026.
Comments
Sign in to join the discussion.