Know a great AI tool or AI news source? Submit it to AiByts

Teachers other than EMA for distillation ? [R]

Self-distillation methods, see Self-Distillation Fine Tuning SDFT, usually use an EMA (Exponential Moving Average) teacher to get synced and stable targets.

By /u/ifapostoReddit - r/ML

Read the full story

Originally published by Reddit - r/ML on Oct 27, 2026.

Read on Reddit - r/ML(opens in a new tab)

Comments

Sign in to join the discussion.

No comments yet — be the first to share your thoughts.