Rapid inference for penetration prediction of plasma arc welding using enhanced ShuffleNetV2 and FOS-ELM

Vision sensing is commonly employed in monitoring the forming process of medium and thick plate in plasma arc welding (PAW). However, due to physical constraints, direct observation of the backside forming process is impractical. Therefore, the weld image on the workpiece’s topside is commonly used to assess weld penetration status. Previous research typically relied on regression and machine learning algorithms to establish this relationship, while recent studies have employed deep learning methods for higher prediction accuracy, but they are computationally demanding, limiting real-time applications in welding. This study aims to improve deep learning model prediction times during welding. We avoid recursive neural network (RNN), vision transformer (ViT), and other high-accuracy approaches with significant computational overhead, opting instead for convolutional neural networks (CNN) for better real-time performance. After comparing six classical CNNs, ShuffleNetV2 backbone was chosen to extract features for its fast computational speed and high prediction accuracy. Innovatively, online sequential extreme learning machine with a forgetting mechanism (FOS-ELM) was introduced to classify penetration status instead of traditional full-layer classification for its high accuracy and speed. Welding experiments on a genuine embedded system validate our approach, reaching a prediction accuracy exceeding 94% on a small dataset, with a prediction time of just 5 ms per welded frame, meeting industrial-grade applications. On the basis of the ShuffleNetV2 backbone and OS-ELM model, transfer learning is used to expedite prediction convergence, while the squeeze excitation (SE) module is employed to enhance accuracy without compromising speed. Moreover, the model’s alignment with skilled welders’ key observation points is visually verified by using gradient-weighted class activation mapping (Grad-CAM). Finally, the deployment of the model in ONNX format on an industrial PC demonstrates its suitability for real-world PAW operations. Vision sensing is crucial for monitoring plasma arc welding (PAW) of medium and thick plates. However, direct observation of the backside formation process is impractical due to certain physical constraints. Therefore, weld image analysis from the workpiece’s topside is commonly used to assess weld penetration. Previous studies relied on fitting and machine learning algorithms, but recent research has shifted towards deep learning for improved accuracy. However, deep learning methods are computationally intensive, limiting their real-time application in welding. This study aims to enhance deep learning model prediction speed during welding by avoiding computationally demanding approaches like recurrent neural networks (RNNs) and vision transformers (ViTs). Instead, we utilize convolutional neural network (CNN) backbones for improved real-time performance. After evaluating six classical CNNs, we selected the ShuffleNetV2 backbone for its fast computational speed and high accuracy and introduced the online sequential extreme learning machine with a forgetting mechanism (FOS-ELM) for classification, achieving high accuracy and speed. Welding experiments validated the proposed approach, achieving over 94% prediction accuracy on a small dataset, with a prediction time of just 5 ms per welded frame. Transfer learning with the ShuffleNetV2 backbone and OS-ELM model expedited prediction convergence. The squeeze-and-excitation (SE) module enhanced accuracy without sacrificing speed. Visualization using gradient-weighted class activation mapping (Grad-CAM) verified the model’s alignment with skilled welders’ observations. Finally, deploying the model in ONNX format on an industrial PC demonstrated its suitability for real-world PAW operations.