IDEAS/STAT Optimization Seminar: “The Size of Teachers as a Measure of Data Complexity: PAC-Bayes Excess Risk Bounds and Scaling Laws”
Amy Gutmann Hall, Room 414 3333 Chestnut Street, Philadelphia, United StatesZoom link: https://upenn.zoom.us/j/98220304722 Abstract: We study the generalization properties of neural networks through the lens of data complexity. Recent work by Buzaglo et al. (2024) shows that random (nearly) interpolating networks generalize, provided there is a small ``teacher'' network that achieves small excess risk. We give a short single-sample PAC-Bayes proof of this result and […]