Quadratic Weak-to-Strong Generalization in Random Feature Networks via Random Matrix Theory

Weak-to-strong generalization is the phenomenon where a strong student model trained with labels produced by a weak teacher model is able to generalize better than the teacher.