A Data Diet for Robot Learning: Data that Scales Passively

Abhishek Gupta

University of Washington, USA

Abstract

Robotic automation, powered by machine learning driven methods, has the potential to build systems that change the future of work, daily life and society at large by acting intelligently in human centric environments. As with most modern machine learning methods, a key component in building such a robotic system the availability of data, abundant, diverse and high quality. In domains of nature language or computer vision, data of this form has scaled passively with internet scale, since people naturally interact through the medium of language or images. In contrast, robots are hardly deployed in human-centric settings and certainly are not collecting internet scale data passively. The key question I will ask in this talk is - how can we develop a data diet for robotic learning that scales passively? I will discuss how alternative sources of information, such as simulation, video data or generative models, despite being fundamentally inaccurate, can provide a scalable source of data for robotic learning. I will discuss a set of techniques for both generating such data, and using this data for informing scalable robotic learning in the wild. In doing so, I hope to shed some light on the unique challenge that data acquisition plays in robot learning and discuss how developing truly open-world robotic learning systems will necessitate rich new machine learning paradigms.