Egocentric Visitor Localization and Artwork Detection in Cultural Sites Using Synthetic Data
Santi Andrea Orlando1, 2, **, Antonino Furnari1, ** and Giovanni Maria Farinella1, 3, **
3National Research Council, ICAR-CNR, Palermo, IT
**These author are co-first authors and contributed equally to this work.
You can download the tool V.1 at this link
Bellomo dataset - Tool V.2
The new version of the tool allows to generate a synthetic dataset for IBL localization and artworks detection in cultural sites with Simulated Egocentric navigations.
This tool allows to:
- import a photo-realistic 3D model of a real cultural site;
- set a navigation system that allows to simulate random navigations inside the cultural by using a virtual agent;
- capture RGB and Semantic mask of the environment and label automatically each frame with the 6 Degrees of Freedom (6DoF) camera pose.
Labeling process of semantic is shown in the figure above.
During each navigation the agent starts from a random room . The first exploration of the environment is performed following the layout of the museum in clockwise/counterclockwise whereas the sencond one will be performed in opposite way of previous choice. In each room the agent see each artwork from 5 different observation points perfoming a look at behaviour. The figure below shows a possible schema of the navigation:
You can download the NEW tool V.2 at this link
Stanford Tool V.1
The tool for Unity 3D proposed in our paper allows to generate a synthetic dataset for IBL localization with Simulated Egocentric navigations.
This tool allows to:
- import a photo-realistic 3D model of a real indoor environment;
- set a navigation system that allows to simulate random paths inside the model by using a virtual agent;
- capture frames of the different channels of the camera (RGB, Depth, Normal, etc.) and labell automatically each frame with the 6 Degrees of Freedom (6DoF) pose of the camera.
You can set the number of paths to simulate and the number of target points to reach during each navigation.
In this video we show an example of how a simulation works.
The window on the left shows the top view of the 3D model where you can see the frustum of the camera during the navigation, whereas the window on the right shows the Egocentric point of view of the camera during the navigation.
During the navigations the virtual agent chooses randomly the next target point to be reached. The figure below shows the possible target point to reach.
You can download the tool V.1 at this link
