Probabilistic and Deep Models for 3D Reconstruction

Andreas Geiger

University of Tübingen and MPI-IS Tübingen, DE

Abstract

3D reconstruction from images is an inherently ill-posed problem. Prior knowledge is required to resolve ambiguities in the observations. Furthermore, probabilistic outputs are desirable. In this lecture, I will first give a brief historical overview and introduce the problem of 3D reconstruction, covering the most fundamental principles of multi-view geometry. I will then talk about different representations for 3D reconstructions including voxels, points and meshes, and demonstrate how those can be inferred using classical optimization-based approaches. Moreover, I will introduce a probabilistic framework for volumetric 3D reconstruction where the reconstruction problem is cast as inference in a Markov random field. More recently, deep learning based 3D reconstruction methods have demonstrated compelling results from as little as a single image or sparse point cloud by combining recognition with geometric reasoning. In the second part of my lecture, I will cover the most popular voxel- and mesh-based approaches. Moreover, I will show very recent results using continuous and primitive-based 3D representations.