Transformer-based deep learning model and video dataset for installation action recognition in offsite projects

Abstract

This paper developed and evaluated the Precast Concrete Installation Dataset (PCI-Dataset), a large-scale video dataset for automatically recognizing precast concrete (PC) installation activities. The dataset comprises 12,791 video clips (5 s each, 1080 × 1080 resolution, 30fps) from actual PC construction sites, including 12 balanced activity classes combining three component types and four work stages. Evaluation of six Transformer-based video classification models showed VideoMAE V2 achieved the highest overall accuracy of 98.10 %, followed by UniFormer V2, Video Swin, MVIT, ViViT, and TimeSformer. VideoMAE V2 achieved F1 scores above 80 % for most activities, with a peak of 92.20 % for slab assembly. In a case study on a real PC construction site, the model demonstrated high recognition accuracies: 100 % for lifting, 85.83–100 % for rigging, and 93.75–100 % for assembly operations. The paper contributes to PC construction management theory by applying computer vision for real-time and automated work recognition and analysis.