Softmax Attention on Gaussian Mixtures: Linear When It Can, Selective When It Must
Softmax attention, at the heart of Transformers, has demonstrated remarkable capabilities. Yet its underlying mechanisms remain only partially understood.
By Simon Gabet (LMO), Etienne Boursier (LMO, CELESTE), Claire Boyer (LMO, IUF)arXiv - stat.ML
Read the full story
Originally published by arXiv - stat.ML on Oct 9, 2026.
Comments
Sign in to join the discussion.