MUON-RT2 min read
MUON-RT Reusing the Teacher's Trajectory for Momentum-Free Muon for LLM Self-Distillation
Self-distillation methods for large language models with an exponential moving average (EMA) teacher already retain a history of student updates. Muon nevertheless stores a separate momentum buffer, increasing persistent training memory.
By Aryan Das, Vinay Kumar Verma, Aakansha MishraarXiv - stat.ML
Read the full story
Originally published by arXiv - stat.ML on Nov 2, 2026.
Comments
Sign in to join the discussion.