Tile detection using mask R-CNN in non-structural environment for robotic tiling

Abstract

Robotic tiling is an efficient way to replace manual work, with tile detection and positioning serving as a pivotal technology. However, the tiling environment is characterized by its complexity. This paper introduces the instance segmentation method Mask R-CNN, which can detect tiles in non-structural environments after proper training. To address the difficulty of acquiring datasets and high annotation costs, a densely arranged tile dataset that allows for automatic labeling has been synthesized and various designed data augmentation techniques are employed. The trained model achieves a detection performance with AP75 = 98.94 % and AP95 = 88.14 % on 100 test images. Subsequently, shape reconstruction is performed to estimate 3D poses of tiles using PNP principle. Finally, a tiling system is developed and combining visual detection with laser detection method enables a successful tiling experiment. Results show that the positional error is less than 0.66 mm and the directional error is less than 0.27°, which meets industrial requirements.