- This event has passed.
ASSET Seminar: “Testing AI’s Implicit World Models”
November 19 at 12:00 PM - 1:15 PM
Many of the robustness properties that are required for real-world applications of AI would be realized by a model that has understood the world. But it is unclear how to measure understanding, let alone how to define it. This talk will propose theoretically-grounded definitions and metrics that test for a model’s implicit understanding, or its world model. We will focus on two kinds of settings: one where implicit world models are tested behaviorally, and another that tests a model’s representation. These exercises demonstrate that models can make highly accurate predictions with incoherent world models, revealing their fragility.
Zoom: https://upenn.zoom.us/j/95189835192
Passcode: 797599
Keyon Vafa
Postdoctoral Fellow
Keyon Vafa is a postdoctoral fellow at Harvard University. His research focuses on understanding and improving the implicit world models learned by generative models. Keyon completed his PhD in computer science from Columbia University, where he was an NSF GRFP Fellow and the recipient of the Morton B. Friedman Memorial Prize for excellence in engineering. He organized the NeurIPS 2024 Workshop on Behavioral Machine Learning and the ICML 2025 Workshop on Assessing World Models, and serves on the early career board of the Harvard Data Science Review.