Training Deep Learning to Drive in the Real World with Computer Vision

Alex Kendall

Wayve and University of Cambridge, UK

Abstract

Autonomous driving is a very hard problem. Commercial efforts spend $5B per year and have driven 10s of millions of autonomous miles but are yet to create a system which is safe and scalable enough for society. End to end deep learning offers a viable solution, we've seen it learn more powerful models than anything humanity can hand-engineer in many other fields rich in data (image recognition, game playing agents, etc). But in contrast to many other popular control problems, such as Go, Atari Games or DOTA, in self-driving the main challenge is learning to represent the world with computer vision.

In particular, I'll focus on the role of computer vision in robotic systems which learn end-to-end from perception to action. How can we learn to understand the world around us with computer vision? How can we leverage computer vision to learn to make decisions? Can we leverage knowledge from simulation? How do we understand and interpret these deep learning models?

This talk will touch on a number of areas of computer vision. First I'll cover research on semantic segmentation, depth estimation, scene understanding and multi-task learning. Secondly I'll talk about Bayesian deep learning, probabilistic modelling and understanding deep models with saliency. Finally I'll show applications of this research to autonomous driving with domain adaption, sim2real, imitation learning and reinforcement learning.