Automatic defect detection in infrared thermal images of ancient polyptychs based on numerical simulation and a new efficient channel attention mechanism aided Faster R-CNN model
Xin Wang, Guimin Jiang, Jue Hu, Stefano Sfarra, Miranda Mostacci, Dimitrios Kouis, Dazhi Yang, Henrique Fernandes, Nicolas P. Avdelidis, Xavier Maldague, Yonggang Gai, Hai Zhang
{"title":"Automatic defect detection in infrared thermal images of ancient polyptychs based on numerical simulation and a new efficient channel attention mechanism aided Faster R-CNN model","authors":"Xin Wang, Guimin Jiang, Jue Hu, Stefano Sfarra, Miranda Mostacci, Dimitrios Kouis, Dazhi Yang, Henrique Fernandes, Nicolas P. Avdelidis, Xavier Maldague, Yonggang Gai, Hai Zhang","doi":"10.1186/s40494-024-01441-9","DOIUrl":null,"url":null,"abstract":"<p>In recent years, the preservation and conservation of ancient cultural heritage necessitate the advancement of sophisticated non-destructive testing methodologies to minimize potential damage to artworks. Therefore, this study aims to develop an advanced method for detecting defects in ancient polyptychs using infrared thermography. The test subjects are two polyptych samples replicating a 14th-century artwork by Pietro Lorenzetti (1280/85–1348) with varied pigments and artificially induced defects. To address these challenges, an automatic defect detection model is proposed, integrating numerical simulation and image processing within the Faster R-CNN architecture, utilizing VGG16 as the backbone network for feature extraction. Meanwhile, the model innovatively incorporates the efficient channel attention mechanism after the feature extraction stage, which significantly improves the feature characterization performance of the model in identifying small defects in ancient polyptychs. During training, numerical simulation is utilized to augment the infrared thermal image dataset, ensuring the accuracy of subsequent experimental sample testing. Empirical results demonstrate a substantial improvement in detection performance, compared with the original Faster R-CNN model, with the average precision at the intersection over union = 0.5 increasing to 87.3% and the average precision for small objects improving to 54.8%. These results highlight the practicality and effectiveness of the model, marking a significant progress in defect detection capability, providing a strong technical guarantee for the continuous conservation of cultural heritage, and offering directions for future studies.</p>","PeriodicalId":13109,"journal":{"name":"Heritage Science","volume":null,"pages":null},"PeriodicalIF":2.6000,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Heritage Science","FirstCategoryId":"92","ListUrlMain":"https://doi.org/10.1186/s40494-024-01441-9","RegionNum":1,"RegionCategory":"艺术学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"CHEMISTRY, ANALYTICAL","Score":null,"Total":0}
引用次数: 0
Abstract
In recent years, the preservation and conservation of ancient cultural heritage necessitate the advancement of sophisticated non-destructive testing methodologies to minimize potential damage to artworks. Therefore, this study aims to develop an advanced method for detecting defects in ancient polyptychs using infrared thermography. The test subjects are two polyptych samples replicating a 14th-century artwork by Pietro Lorenzetti (1280/85–1348) with varied pigments and artificially induced defects. To address these challenges, an automatic defect detection model is proposed, integrating numerical simulation and image processing within the Faster R-CNN architecture, utilizing VGG16 as the backbone network for feature extraction. Meanwhile, the model innovatively incorporates the efficient channel attention mechanism after the feature extraction stage, which significantly improves the feature characterization performance of the model in identifying small defects in ancient polyptychs. During training, numerical simulation is utilized to augment the infrared thermal image dataset, ensuring the accuracy of subsequent experimental sample testing. Empirical results demonstrate a substantial improvement in detection performance, compared with the original Faster R-CNN model, with the average precision at the intersection over union = 0.5 increasing to 87.3% and the average precision for small objects improving to 54.8%. These results highlight the practicality and effectiveness of the model, marking a significant progress in defect detection capability, providing a strong technical guarantee for the continuous conservation of cultural heritage, and offering directions for future studies.
期刊介绍:
Heritage Science is an open access journal publishing original peer-reviewed research covering:
Understanding of the manufacturing processes, provenances, and environmental contexts of material types, objects, and buildings, of cultural significance including their historical significance.
Understanding and prediction of physico-chemical and biological degradation processes of cultural artefacts, including climate change, and predictive heritage studies.
Development and application of analytical and imaging methods or equipments for non-invasive, non-destructive or portable analysis of artwork and objects of cultural significance to identify component materials, degradation products and deterioration markers.
Development and application of invasive and destructive methods for understanding the provenance of objects of cultural significance.
Development and critical assessment of treatment materials and methods for artwork and objects of cultural significance.
Development and application of statistical methods and algorithms for data analysis to further understanding of culturally significant objects.
Publication of reference and corpus datasets as supplementary information to the statistical and analytical studies above.
Description of novel technologies that can assist in the understanding of cultural heritage.