FOLDS seminar: A New Paradigm for Learning with Distribution Shift
Zoom link: https://upenn.zoom.us/j/98220304722 We revisit the fundamental problem of learning with distribution shift, where a learner is given labeled samples from training distribution D, unlabeled samples from test distribution D′ and […]
FOLDS seminar: Theory and practice of LLM quantization
Zoom link: https://upenn.zoom.us/j/98220304722 Modern LLMs process information by repeatedly applying a basic primitive of matrix multiplication. Estimates show that about 60-84% of the energy consumed by LLMs goes into […]
FOLDS seminar: Propagation-of-Chaos in Shallow Neural Networks beyond Logarithmic Time.
Zoom link: https://upenn.zoom.us/j/98220304722 The analysis of gradient-based learning of Neural Networks remains an outstanding challenge, even for the simplest shallow architectures. A powerful mathematical framework that has emerged over recent […]
FOLDS seminar: Heaviside Composite Optimization: A new paradigm of optimization
Zoom link: https://upenn.zoom.us/j/98220304722 A Heaviside function is an indicator function of a semi-infinite interval. A Heaviside composite function is a Heaviside function composed with a multivariate function that may be […]
FOLDS seminar: Algorithmic stability for regression and classification
In a supervised learning setting, a model fitting algorithm is unstable if small perturbations to the input (the training data) can often lead to large perturbations in the output (say, […]
FOLDS Seminar: Positive random walks and positive-semidefinite random matrices
On the real line, a random walk that can only move in the positive direction is very unlikely to remain close to its starting point. After a fixed number of […]
IDEAS/STAT Optimization Seminar: “Negative Stepsizes Make Gradient-Descent-Ascent Converge”
Zoom link: https://upenn.zoom.us/j/98220304722 Abstract: Solving min-max problems is a central question in optimization, games, learning, and controls. Arguably the most natural algorithm is Gradient-Descent-Ascent (GDA), however since the 1970s, conventional wisdom […]
IDEAS/STAT Optimization Seminar: Resilient Distributed Optimization for Cyberphysical Systems
Zoom link: https://upenn.zoom.us/j/98220304722 Abstract: This talk considers the problem of resilient distributed multi-agent optimization for cyberphysical systems in the presence of malicious or non-cooperative agents. It is assumed that […]
IDEAS/STAT Optimization Seminar: “Gradient Equilibrium in Online Learning”
We present a new perspective on online learning that we refer to as gradient equilibrium: a sequence of iterates achieves gradient equilibrium if the average of gradients of losses along […]
IDEAS/STAT Optimization Seminar: “Stochastic-Gradient-based Algorithms for Solving Nonconvex Constrained Optimization Problems”
Zoom link: https://upenn.zoom.us/j/98220304722 Abstract I will present recent work by my research group on the design and analysis of stochastic-gradient-based algorithms for solving nonconvex constrained optimization problems, which may […]