I Built a O(NlogN) attention system that retains 97% accuracy over long context (MQAR)[P]
ALHR- Adaptive learnable Hierarchical Routing is a static binary tree based system that uses learnable functions to REDUCE the amount of keys used. It takes less memory and SCALES much better VRAM with tokens submitted by /u/Alarming-Emotion-894 [link] [comments]
Reddit - r/ML1 min read