CIS Seminar: “Bridging Learning and Decision Making”

Machine learning is becoming widely used in decision making, in domains ranging from personalized medicine and mobile health to online education and recommendation systems. While (supervised) machine learning traditionally excels at prediction problems, decision making requires answering questions that are counterfactual in nature, and ignoring this mismatch leads to unreliable decisions. As a consequence, our understanding of the algorithmic foundations for data-driven decision making is limited, and efficient algorithms are typically developed on an ad hoc basis. Can we bridge this gap and make decision making as easy as machine learning?

Focusing on the contextual bandit, a core problem in data-driven decision making, we bridge the gap by providing the first optimal and efficient reduction to supervised machine learning. The algorithm allows users to seamlessly apply off-the-shelf supervised learning models and methods to make decisions on the fly, and has been implemented in widely-used, industry-standard tools for decision making.

Our results advance a broader program to develop a universal algorithm design paradigm for data-driven decision making. I will close the talk by discussing challenges and opportunities in building such a framework, including efforts to extend our developments to difficult reinforcement learning problems in large state spaces.