Vision-Based Nut Quality Classification Using Conditional GAN and CNN

Abstract

In this study, the quality of a nut is discussed by considering images of the internal thread, and an analysis is conducted using traditional machine-learning and deep-learning algorithms. Compared to the traditional contact methods, the vision-based method has the advantage of fast computing speed and is not affected by the conditions of tapping speed. The pitch and pitch diameter of the internal thread are the indicators that characterize nut quality classification. For one nut, 36 internal thread images are collected, one image per 10 degrees, by the self-designed laser triangulation measurement platform. Using the laser triangulation method, the information on both indicators can be obtained and analyzed. In the traditional machine-learning methods, the internal thread images undergo several preprocessing procedures to obtain the region of interest and calculate the depth between the crest and root. Subsequently, 33 handcrafted features are used to extract the features from the 36 processed images. Finally, the features are classified by three families of machine-learning algorithms, including support vector machines, k-nearest neighbors, and decision trees. In the deep-learning method, conditional generative adversarial network and convolutional neural network (CNN) are used for data augmentation and nut quality classification, respectively. The experimental results show that the proposed CNN model can achieve a higher classification accuracy rate. Furthermore, the proposed CNN model trained with the generated images is better equipped to detect the nut quality under different decision thresholds. Note to Practitioners—This study explores the assessment of nut quality, particularly focusing on the internal thread. Traditional methods for grading thread quality often involve contact-based detection, which may risk damaging the thread surface. Additionally, most non-contact methods tend to have prolonged classification processes. This study introduces a vision-based nut-quality classification method based on the Japanese Industrial Standard (JIS) specification, offering advantages such as rapid computation and independence from tapping speed conditions. Internal thread images are collected using a self-designed laser triangulation measurement platform, and various learning algorithms are employed for analysis. In traditional machine-learning approaches, we propose an image preprocessing method to extract 33 statistical features from internal thread images. Feature-importance analysis, calculated from random forest (RF) and gradient boosting (GB), reveals the significance of features such as crest-to-root depth variation between teeth and the brightness of the crest. By performing feature reduction, some machine-learning algorithms can improve the classification accuracy rate and AUC of the model. In the realm of deep learning, we utilize conditional generative adversarial network (CGAN) to generate internal thread images and employ CNN for nut quality classification. Experimental results demonstrate that the proposed CNN model achieves higher classification accuracy with fewer training parameters compared to VGG16, VGG19, ResNet50, and Xception. The deep-learning models, particularly the CNN, outperform traditional machine-learning methods without the need for manual feature extraction. CGAN successfully augments the training dataset, and the models can detect nut quality under different decision thresholds. Although this proposed nut-quality analysis method achieves non-contact detection, it has not yet attained full automation. Future research will endeavor to integrate automated mechanisms for placing nuts onto the inspection platform and to conduct real-time image analysis for continuous assessment of nut quality.