Mo.Se.: Mosaic image segmentation based on deep cascading learning

Abstract

Mosaic is an ancient type of art used to create decorative images or patterns of small components. A digital version of a mosaic can be useful for archaeologists, scholars and restorers who are interested in studying, comparing and preserving mosaics. Nowadays, archaeologists base their studies mainly on manual operation and visual observation that, although still fundamental, should be supported by the aid of automatized procedure of information extraction. In this context, this research intends to overcome manual and time-consuming procedure of mosaic tesserae drawing by proposing Mo.Se. (Mosaic Segmentation), an algorithm that exploits deep learning and image segmentation techniques; specifically, the methodology combines U-Net 3 Network with the Watershed algorithm. The final purpose is to define a workflow which establishes the steps to perform a robust segmentation and obtain a digital (vector) representation of a mosaic. The detailed approach is presented, and theoretical justifications are provided, building various connections with other models, making the workflow both theoretically valuable and practically scalable for medium or large datasets. The automatic segmentation process was tested with the high-resolution orthoimage of an ancient mosaic, realized following a close-range photogrammetry procedure. Our approach has been tested in the pavement of St. Stephen's Church in Umm ar-Rasas, a Jordan archaeological site, located 30 km southeast of the city of Madaba (Jordan). Experimental results show that this generalized framework yields good performances, obtaining higher accuracy compared with other state-of-the-art approaches. Mo.se. has been validated using publicly available datasets as a benchmark, demonstrating that the combination of learning-based methods with procedural ones enhances segmentation performance almost 10% in terms of overall accuracy. The ambition is to provide archaeologists with a tool for automatic extraction of geometric of ancient mosaics, which expedites their work.