Market Basket Analysis From Egocentric Videos
Vito Santarcangelo, Giovanni Maria Farinella,
Antonino Furnari, Sebastiano Battiato
Submitted to IEEE Transactions on Multimedia
This paper discusses a novel application domain for egocentric vision systems: Visual Market Basket Analysis (VMBA). The final goal is to infer of customers' behaviours during their shopping from image sequences acquired with cameras mounted on shopping carts. To set up the challenge and to perform a benchmark, a new dataset has been created by collecting egocentric videos during real shopping sessions in a retail. The video frames have been labelled by considering 14 different behaviours of the shopping carts which reflect the behaviour of the customers from the beginning (cart picking) to the end (cart releasing) of their shopping. The inferred information related to the time of stops of the carts within the store, or to the cash desks could be coupled with classic Market Basket Analysis information (i.e., receipts) to help retailers for a better management of spaces and marketing strategies. Behavior classification has been performed by exploiting both visual and audio descriptors in joint with the Directed Acyclic Graph SVM learning architecture. An accuracy of more than 82% has been obtained on the 14 classes of the considered dataset.