2024 IEEE MetroXRAINE
Good news!
The paper with title “ViLaBot: Connecting Vision and Language for Robots That Assist Humans at Home” has been accepted with minor revisions for publication and presentation at the 2024 IEEE MetroXRAINE!
In this paper we present ViLaBot (Vision and Language roBot), a system designed to aid humans in daily activities while at home. ViLaBot combines a language model with a library of basic visuomotor skills to understand human needs, create action plans and execute them. ViLaBot relies solely on onboard visual and proprioceptive sensing, eliminating the need for pre-built maps or precise object locations and facilitating real-world deployment in a variety of environments.
This is a paper in collaboration with Philips Research Eindhoven within the European Marie Curie project “Philhumans” (https://www.philhumans.eu)