ESE Ph.D. Thesis Defense: “Machine Learning for Large-Scale Cyber-Physical Systems”

Directly training deep learning models for applications in large-scale cyber-physical systems can be intractable due to the large number of components and decision variables. Instead, we focus on exploiting spatial symmetries in systems by designing size-generalizable architectures. Once trained on small-scale examples, such architectures exhibit equivalent or comparable performance on large-scale systems. The first example we consider is a fully convolutional neural network, for which we prove a bound that guarantees generalization performance. We demonstrate generalizability empirically with applications to multi-target tracking and mobile infrastructure on demand. Next, we introduce a novel spatial transformer architecture design with two key properties in mind: locality and shift-equivariance. The proposed architecture uses shift-equivariant positional encodings and spatially windowed attention. Our experiments in two distributed collaborative multi-robot tasks show that these design features are necessary for size generalizability. Moreover, we demonstrate that the spatial transformer architecture is capable of decentralized execution, robust to communication delays, can generalize to unseen tasks, and performs state-of-the-art graph neural networks. Finally, we refocus on a particularly challenging optimization problem in power systems: optimal power flow (OPF). By appropriately formulating the Lagrangian dual problem, we train graph attention networks with improved optimality and feasibility. The training performance can also be reproduced on new power systems without further hyperparameter tuning.