Panel (HoloViz)12 min read
Opening up Attention: A QK Transcoder Dashboard for Feature Pairs
Multi Head Attention in Transformers continues to be an elusive topic in mechanistic interpretability. One way of trying to decompose MHA into interpretable parts is applying SDL (Sparse Dictionary Learning) methods to the attention mechanism.
By nuppalLessWrong

Read the full story
Originally published by LessWrong on Oct 22, 2026.
Comments
Sign in to join the discussion.