Market Basket Analysis From Egocentric Videos
Vito Santarcangelo, Giovanni Maria Farinella,
Antonino Furnari, Sebastiano Battiato

Submitted to IEEE Transactions on Multimedia


This paper discusses a novel application domain for egocentric vision systems: Visual Market Basket Analysis (VMBA). The final goal is to infer of customers' behaviours during their shopping from image sequences acquired with cameras mounted on shopping carts. To set up the challenge and to perform a benchmark, a new dataset has been created by collecting egocentric videos during real shopping sessions in a retail. The video frames have been labelled by considering 14 different behaviours of the shopping carts which reflect the behaviour of the customers from the beginning (cart picking) to the end (cart releasing) of their shopping. The inferred information related to the time of stops of the carts within the store, or to the cash desks could be coupled with classic Market Basket Analysis information (i.e., receipts) to help retailers for a better management of spaces and marketing strategies. Behavior classification has been performed by exploiting both visual and audio descriptors in joint with the Directed Acyclic Graph SVM learning architecture. An accuracy of more than 82% has been obtained on the 14 classes of the considered dataset.


Example of Activity Map in some instants. See quantitative results in the section below.


VMBA15 dataset is composed by 15 different egocentric videos of narrative carts in a retail of the Southern of Italy.
Each narrative cart video has a duration between 3 to 20 minutes and resolution of 640x480 pixels.
Number of sample per class for each video is reported in Table 1.
The dataset is available upon request to the authors.