machine learning2 min read
Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout (Google Research, 10M tokens in 25MB state) [R]
Paper: https://arxiv.org/abs/2611.08376 | HF: https://huggingface.co/papers/2611.08376 At 10M tokens, a 1.3B Transformer needs 2.1 TB of KV cache (0.32 TB even at 2.5-bit quantization).
By /u/CategoryNormal149Reddit - r/ML
![Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout (Google Research, 10M tokens in 25MB state) [R]](https://d3qy4oidjlxje0.cloudfront.net/news/original/reddit-r-ml-b322863946c7bd33.webp)
Read the full story
Originally published by Reddit - r/ML on Nov 6, 2026.
Comments
Sign in to join the discussion.