GRASP on Robotics: “Approaches to Grounded Language Acquisition from Human Interaction”

*This will be a HYBRID Event with in-person attendance for Dr. Matuszek’s in-person talk in Wu & Chen Auditorium and Virtual attendance via Zoom Webinar here. 

As robots move from labs and factories into human-centric spaces, it becomes progressively harder to predetermine the environments and interactions they will need to be able to handle. Letting robots learn from end users via natural language is an intuitive, versatile approach to handling novel situations robustly. Grounded language acquisition is concerned with learning to understand language in the context of the physical world. In this presentation, I will give an overview of our work on using joint statistical models to learn the grounded semantics of natural language describing an agent’s environment, and will describe work on applying those models in a sim-to-real language learning environment. I will also discuss the role of speech understanding in grounded language learning, including introducing a new dataset and results on learning directly from that speech.