Foundational Issues in AI: Views from the real and the ideal worlds

Stefano Soatto

Amazon and University of California Los Angeles, USA

Abstract

Recent Developments in large-scale machine learning have triggered a mixture of excitement and consternation. The surprising emergence of seemingly “intelligent” behavior in models trained with simple predictive criteria on massive text corpora has sparked a Cambrian explosion of innovation, in both Academia and Industry. I will discuss some of the synergies, as well as occasional misalignments, where what appears important may be inconsequential, while seemingly mundane problems reveal fertile grounds for intellectual exploration. I will focus in particular on issues that arise when a “closed system,” like a large language model that is born and raised in the digital world, interfaces and interacts with the physical world. The ease with which large-scale generative models can synthesize realistic data has caused the resurgence of problems long confined to the realm of philosophy and the foundations of mathematics: Can a neural network “capture physical reality”? Is there something fundamentally different about interacting with physical space that makes the outcome of learning different than if done by predicting the next text token? Can a machine learning model generate new “laws of physics”, or “understand” if something is “real”? Even if (human) language arose late in the evolutionary tree, we are not bound by evolution. Still, what are the advantages of going backwards from language to grounding? And how would “language” emerge from interaction with physical space in the first place? And if we “close the loop” by allowing a generative model to interact with the physical world, will it be controllable? Will the overall system be stable? When one frames these questions, some limitations in the popular notions of “information”, “knowledge”, and even “language” become patent. I will discuss a few results, some positive, some negative, on what neural networks can do when it comes to interfacing with the physical world. Along the way, I will give examples of how some of these questions arise when addressing real problems, where Industry and Academia can fruitfully cooperate. Bio: Stefano Soatto is Professor of Computer Science at the University of California, Los Angeles, and Vice President at Amazon Web Services, where he leads AI Labs, that develop AI Application Services at AWS.