Know a great AI tool or AI news source? Submit it to AiByts
MoEZip2 min read

MoEZip: Routing-Aware KV Cache Compression for Sparse Mixture-of-Experts LLMs

Mixture-of-Experts (MoE) language models scale model capacity efficiently by activating a small subset of parameters per token. However, in long-context inference, memory remains a major bottleneck because the KV cache grows linearly with sequence length.

By Minsang Kim, Seungjun BaekarXiv cs.LG

Read the full story

Originally published by arXiv cs.LG on Oct 23, 2026.

Read on arXiv cs.LG(opens in a new tab)

Comments

Sign in to join the discussion.

No comments yet — be the first to share your thoughts.