P$^2$-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors
Multi-teacher on-policy distillation consolidates expert capabilities by supervising student-generated trajectories, whereas task arithmetic offers a training-free alternative through linear parameter combinations.
arXiv cs.LG2 min read