On Information Theoretic Bounds for SGD

Few days ago we had a talk by Gergely Neu, who presented his recent work:Gergely Neu Information-Theoretic Generalization Bounds for Stochastic Gradient DescentI'm writing this post mostly to annoy him, by presenting this work using super hand-wavy intuitions and cartoon figures. If this isn&