Can Representation Learning Decouple from Loss Minimization? Polar Updates Have an Answer

Does representation learning stop when the training loss stops improving? We study this question for matrix Muon, whose polar-normalised updates have a step length set by the gradient's rank rather than its norm.