machine learning1 min read
Teachers other than EMA for distillation ? [R]
Self-distillation methods, see Self-Distillation Fine Tuning SDFT, usually use an EMA (Exponential Moving Average) teacher to get synced and stable targets.
By /u/ifapostoReddit - r/ML
Read the full story
Originally published by Reddit - r/ML on Oct 27, 2026.
Comments
Sign in to join the discussion.