Product counting using images with application to robot-based retail stock assessment

Abstract

In this paper, we propose a novel method for obtaining product count directly from images recorded using a monocular camera mounted on a mobile robot. This has application in robot-based retail stock assessment problem where a mobile robot is used for monitoring the stock levels on the shelves of a retail store. The products are recognized by carrying out a nearest-neighbor search in the template feature space using a k-d tree. Unlike current approaches which only provide approximate stock level, we propose a method which can compute the exact number of discrete products visible in a given image. The product count is obtained by fitting bounding box around each product and removing them sequentially from the image. A second stage of grid-based search is carried out in the neighborhood of each detected product to detect new products which were missed out in the previous step. This detection is based on a confidence measure that includes various information such as histogram matching and spatial location. The efficacy of the proposed approach is demonstrated through experiments on different datasets obtained using robot camera as well as mobile phone camera. These results show that the robot-based retail stock assessment may become a viable alternative to the currently prevailing manual mode of carrying out these surveys.