Unsupervised Domain Adaptation for Object Detection in Cultural Sites
FPV@IPLAB - Department of Mathematics and Computer Science, University of Catania, Italy
Giovanni Pasqualino, Antonino Furnari, Giovanni Maria Farinella
We extended our work and obtained better results also on Cityscapes to Foggy Cityscapes dataset, please check these links: web site, paper, DA-RetinaNet code.
The ability to detect objects in cultural sites from the egocentric point of view of the user can enable interesting applications for both the visitors and the manager of the site. Unfortunately, current object detection algorithms have to be trained on large amounts of labeled data, the collection of which is costly and time-consuming. While synthetic data generated from the 3D model of the cultural site can be used to train object detection algorithms, a significant drop in performance is generally observed when such algorithms are deployed to work with real images. In this paper, we consider the problem of unsupervised domain adaptation for object detection in cultural sites. Specifically, we assume the availability of synthetic labeled images and real unlabeled images for training. To study the problem, we propose a dataset containing 75244 synthetic and 2190 real images with annotations for 16 different artworks. We hence investigate different domain adaptation techniques based on image to image translation and feature matching. Our analysis points out that such techniques can be useful to address the domain adaptation issue, while there is still plenty of space for improvement on the proposed dataset.Dataset
We propose a dataset of synthetic and real images related to 16 artworks present in "Galleria regionale Palazzo Bellomo" located in Siracusa, Italy. The dataset contains two set of images, synthetic and real which are divided has follows:
Synthetic Dataset
Real Dataset
You can download the whole dataset and annotations at this link .
Methods
We explore the following methods:
1) baseline approaches without adaption;
2) domain adaptation through image to image translation;
3) domain adaptation through feature matching;
4) domain adaptation combining feature matching and image to image translation.
Results
Paper
G. Pasqualino, A. Furnari, G. M. Farinella. Unsupervised Domain Adaptation for Object Detection in Cultural Sites. International Conference on Pattern Recognition (ICPR) 2020 Paper
@inproceedings{pasqualino2020unsupervised,
year = { 2020 },
booktitle = { International Conference on Pattern Recognition (ICPR)},
title = { Unsupervised Domain Adaptation for Object Detection in Cultural Sites },
author = { Giovanni Pasqualino, Antonino Furnari, Giovanni Maria Farinella},
}
Acknowledgement
This research is supported by the project VALUE - Visual Analysis for Localization and Understanding of Environments (N. 08CT6209090207) - PO FESR 2014/2020 - Azione 1.1.5. - "Sostegno all’avanzamento tecnologico delle imprese attraverso il finanziamento di linee pilota e azioni di validazione precoce dei prodotti e di dimostrazioni su larga scala", and by Piano della Ricerca 2016-2018 linea di Intervento 2 of DMI, University of Catania. The authors would like to thank Regione Siciliana Assessorato dei Beni Culturali dell’Identità Siciliana - Dipartimento dei Beni Culturali e dell’Identità Siciliana and Polo regionale di Siracusa per i siti culturali - Galleria Regionale di Palazzo Bellomo.
