TGSYOLO: Template-Guidance Siamese Network for SMT Welding Defect Detection

Abstract

Surface-mounted technology (SMT) welding defect detection plays a key role in the printed circuit board assembly (PCBA) production process, which affects the use of electronic products and cost. Previous works tend to realize defect detection with only defect samples and they assume that there are sufficient defect samples. However, defect samples are usually difficult to collect in real-life scenarios while enough template samples can be easily obtained. In addition, most existing works carry out defect detection based on benchmarks with simple backgrounds of PCBA, which is not suitable for PCBA with complex structures in modern electronic product manufacturing. To address the above issues, we propose a template-guidance Siamese network based on YOLO for SMT welding defect detection (TGSYOLO), which is deployed on a real SMT automatic optical inspection (AOI) system. First, the two-stream structure is introduced to extract deep features in defect images and template images, in which template features serve as guidance knowledge. Then, a template fusion Transformer (TFT) is proposed to model global features between detect and template features in the low-level stage, which could acquire long-range correlations to force the network to focus on potential defect regions. Next, to avoid the disappearance of tiny defect features during deep feature fusion, a multiscale attention feature pyramid network (MAFPN) is proposed to directly fuse defect semantic information from low-level features, which retains detailed expressions of defects through skip connection and obtains compact fusion features. Furthermore, we collect limited welding defect samples based on more complex PCBA backgrounds than previous works through a real SMT AOI system. Experiments on the limited dataset show that TGSYOLO could reach 0.985 of mAP@0.5, 0.885 of mAP@0.75, and 0.984 of F1, which is 0.008, 0.054, and 0.025 higher than other SOTA methods. Also, generalization experiments on the public DeepPCB show that TGSYOLO could still reach the best with 0.991 of mAP@0.5 and 0.89 of mAP@0.75, which proves that TGSYOLO has good generalization performance.