Loading Events

« All Events

  • This event has passed.

Spring 2024 GRASP SFI: Karl Pertsch, University of California, Berkeley and Stanford University, “Building Open-Source Generalist Robot Policies”

April 17 at 2:00 PM - 3:00 PM

This will be a hybrid event with in-person attendance in Towne 337 and virtual attendance on Zoom.

ABSTRACT

Generalist robot policies, trained on large and diverse robot datasets, have the potential to transform how robot learning research is done: in the same way that current models in NLP are almost universally derived from pretrained large language models, future robot policies might be initialized from generalist robot models and finetuned with only modest amounts of target domain data.

In this talk I will discuss our efforts on building such generalist robot policies. I will focus on two key ingredients: data and models. On the data side, I will discuss our recent works on building the largest open-source real robot manipulation datasets to date, the Open X-Embodiment dataset and DROID, with a total of 2M+ robot trajectories. On the model side, I will summarize our learnings from building RT-X and Octo, the first generalist robot policies trained on the Open X-Embodiment dataset. I will discuss their current limitations and outline important steps for future research towards ubiquitous robot foundation models.

Karl Pertsch

University of California, Berkeley and Stanford University

Karl Pertsch is a postdoc at UC Berkeley and Stanford, jointly advised by Sergey Levine and Chelsea Finn, where he focuses on building robot foundation models. Karl obtained his PhD from USC, advised by Joseph Lim. During his PhD he interned at MetaAI and Google Brain. He is a recipient of a Fulbright Scholarship and the CoRL’20 best paper presentation award.

Details

Date:
April 17
Time:
2:00 PM - 3:00 PM
Event Category:
Website:
https://www.grasp.upenn.edu/events/spring-2024-grasp-sfi-karl-pertsch/

Organizer

General Robotics, Automation, Sensing and Perception (GRASP) Lab
Email
grasplab@seas.upenn.edu
View Organizer Website

Venue

Towne 337