Know a great AI tool or AI news source? Submit it to AiByts
2 min read

A Quadratic Lens on Muon's Polar Update: Benefits, Pitfalls, and the Role of Newton-Schulz

Muon updates matrix-valued parameters along approximately orthogonalized directions and often trains large language models faster than AdamW, yet it remains unclear when and why orthogonalization helps.

By Egor Shulgin, Sam Laing, Antonio Orvieto, Peter Richt\'arikarXiv cs.LG

Read the full story

Originally published by arXiv cs.LG on Oct 23, 2026.

Read on arXiv cs.LG(opens in a new tab)

Comments

Sign in to join the discussion.

No comments yet — be the first to share your thoughts.