Detection and localization of pallets on shelves using a wide-angle camera

This paper proposes a method to detect and localize pallets on shelves using a wide-angle camera mounted on a vehicle several meters away along the shelves. Although wide-angle observation is a significant advantage in navigation, objects appear small and largely distorted in the observed image, thus creating a severe disadvantage for the detection and localization of objects. To solve this problem, we propose reprojecting wide-angle views onto a three-dimensional (3D) plane to obtain a good projection image for object detection, in which pallets appear in the same shape and size regardless of the location in the image. Since the image projected on the plane coplanar to the pallet's front surface satisfy the above condition, we approximate the projection plane using the front surface of the shelves with pallets on it. Owing to the constant appearance of the pallet on the projected image, it can be robustly detected using the matching with a fixed template. When a pallet is detected, it is three-dimensionally localized from the equation of the 3D plane of the projection plane and its 2D location in the projected image. Experiments using empirically captured scenes demonstrated the detection and localization ability of the proposed method. Despite small area in the original wide-angle view, pallets of interest were discriminatively detected from others and localized with acceptable accuracy.