FOLDS SEMINAR: The Hidden Width of Deep ResNets
Zoom link: https://upenn.zoom.us/j/6130182858 We present a mathematical framework to analyze the training dynamics of deep ResNets that rigorously captures practical architectures (including Transformers) trained from standard random initializations. Our […]
FOLDS seminar: Function Space Perspectives on Neural Networks
Zoom link: https://upenn.zoom.us/j/98220304722 This talk reviews a theory of the functions learned by neural networks with Rectified Linear Unit (ReLU) activations. At its core is the observation that deep ReLU […]
FOLDS seminar: Learning in Strategic Queuing
Zoom link: https://upenn.zoom.us/j/98220304722 Over the last two decades we have developed good understanding how to quantify the impact of strategic user behavior on outcomes in many games (including traffic routing […]
FOLDS Seminar: ACS: An interactive framework for machine-assisted selection with model-free guarantees
Zoom link: https://upenn.zoom.us/j/98220304722 In this talk, I will introduce adaptive conformal selection (ACS), an interactive framework for model-free selection with guaranteed error control. Building on conformal selection (Jin and Candès, […]
FOLDS seminar: Weak to Strong Generalization in Random Feature Models
Zoom link: https://upenn.zoom.us/j/98220304722 Weak-to-Strong Generalization (Burns et al., 2023) is the phenomenon whereby a strong student, say GPT-4, learns a task from a weak teacher, say GPT-2, and ends up […]
FOLDS seminar: An Information Geometric Understanding of Deep Learning
Zoom link: https://upenn.zoom.us/j/98220304722 I will argue that properties of natural data are what predominantly make deep networks so effective. To that end, I will show that deep networks work well […]
FOLDS seminar: A New Paradigm for Learning with Distribution Shift
Zoom link: https://upenn.zoom.us/j/98220304722 We revisit the fundamental problem of learning with distribution shift, where a learner is given labeled samples from training distribution D, unlabeled samples from test distribution D′ and […]
FOLDS seminar: Theory and practice of LLM quantization
Zoom link: https://upenn.zoom.us/j/98220304722 Modern LLMs process information by repeatedly applying a basic primitive of matrix multiplication. Estimates show that about 60-84% of the energy consumed by LLMs goes into […]
FOLDS seminar: Propagation-of-Chaos in Shallow Neural Networks beyond Logarithmic Time.
Zoom link: https://upenn.zoom.us/j/98220304722 The analysis of gradient-based learning of Neural Networks remains an outstanding challenge, even for the simplest shallow architectures. A powerful mathematical framework that has emerged over recent […]
FOLDS seminar: Heaviside Composite Optimization: A new paradigm of optimization
Zoom link: https://upenn.zoom.us/j/98220304722 A Heaviside function is an indicator function of a semi-infinite interval. A Heaviside composite function is a Heaviside function composed with a multivariate function that may be […]