Know a great AI tool or AI news source? Submit it to AiByts

Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout (Google Research, 10M tokens in 25MB state) [R]

Paper: https://arxiv.org/abs/2611.08376 | HF: https://huggingface.co/papers/2611.08376 At 10M tokens, a 1.3B Transformer needs 2.1 TB of KV cache (0.32 TB even at 2.5-bit quantization).

By /u/CategoryNormal149Reddit - r/ML
Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout (Google Research, 10M tokens in 25MB state) [R]

Read the full story

Originally published by Reddit - r/ML on Nov 6, 2026.

Read on Reddit - r/ML(opens in a new tab)

Comments

Sign in to join the discussion.

No comments yet — be the first to share your thoughts.