DuplexTune2 min read
DuplexTune: Coupled-State Replay for Memory-Efficient Mixture-of-Experts Adaptation
Sparse expert execution does not remove the stored states needed to adapt a mixture-of-experts language model. We present DuplexTune, a framework for replaying coupled hidden states through pretrained attention and expert modules.
By Paul Raj, Simon Caldwell, Daniel Whitmore, Mutlu MetearXiv cs.LG
Read the full story
Originally published by arXiv cs.LG on Oct 28, 2026.
Comments
Sign in to join the discussion.