Visual Localization or Where Am I and Who Knows

Torsten Sattler

Czech Technical University in Prague, CTU

Abstract

Visual localization is the problem of estimating the exact position and orientation from which an image was taken with respect to a known scene. Solving the visual localization problem is a key component in many exciting applications of computer vision, including Augmented / Mixed / Virtual Reality and embodied systems such as self-driving cars and other autonomous robots. This lecture covers the topic of visual localization in two parts: The first part discusses the predominant visual localization approaches in the literature, ranging from classical, feature-based approaches, purely learning-based methods, and pipelines that combine hand-crafted and learned component. The first part is devoted to methods that optimize performance (be it run-time, camera pose accuracy, or memory requirements). One of the exciting applications of visual localization are Augmented / Mixed / Virtual Reality systems, e.g., systems allowing users to join Metaverses. In this context, multiple potential privacy issues arise. For example: How can users store data for visual localization in the cloud without revealing private information about the user? How can users of localization services in the cloud ensure that the data they send to the service does not reveal private details? The second part of the lecture thus discusses visual localization techniques focused on user privacy rather than performance. This part also takes a critical look at this (young) sub-field in terms of how valid claims made in the literature about privacy really are.