- This event has passed.
FOLDS seminar & PENN AI seminar: Optimization Challenges in Physics-Informed Neural Networks
March 5 at 12:00 PM - 1:00 PM
Zoom link: https://upenn.zoom.us/j/98220304722
Physics-informed neural networks (PINNs) minimize composite losses that penalize PDE residuals alongside boundary and initial conditions. While this resembles multi-task learning, the optimization landscape is fundamentally different. Differential operators amplify high-frequency error modes by polynomial factors, while the neural tangent kernel’s eigenspectrum suppresses precisely those modes — creating a spectral mismatch absent in standard supervised learning. Through NTK analysis, I will show that this leads to orders-of-magnitude disparities in per-component convergence rates, and that the resulting composite gradient is not merely imbalanced in magnitude but conflicted in direction. I will present a gradient alignment score that quantifies these directional conflicts and provide theoretical evidence that first-order methods are intrinsically limited in resolving them. On the practical side, I will show how layer-wise preconditioning (via the SOAP optimizer) achieves implicit gradient alignment and 2-10x accuracy gains on challenging benchmarks including the simulation of turbulent fluid flows, and how adaptive residual architectures restore trainability at depth. Throughout, I will highlight the structural properties that distinguish these problems from generic multi-task optimization — known operator spectra, deterministic residuals, a priori inter-task coupling — and argue that these present rich opportunities for rigorous theory and scalable algorithm design.
Paris Perdikaris
Associate Professor of Mechanical Engineering and Applied Mechanics at the University of Pennsylvania
Paris Perdikaris is an Associate Professor of Mechanical Engineering and Applied Mechanics at the University of Pennsylvania. He received his Ph.D. in Applied Mathematics from Brown University (2015), and worked as a postdoctoral researcher at the Massachusetts Institute of Technology (2015-2017). His research interests span a range of topics at the interface of computational science and machine learning, including the development of foundation models for Earth system modeling. physics-informed neural networks and neural operators, generative models, and uncertainty quantification for sequential decision making in scientific and engineering applications.