HTBN: A Heterogeneous Network for Breast Ultrasound Image Classification

Ultrasound (US) is one of a primary imageological examination and preoperative assessment for brleast nodules. However, in the field of ultrasound diagnosis, it relies heavily on the experience of physicians due to the overlapping image expression of benign and malignant breast nodules. The diagnostic accuracy of physicians with different qualifications differs by up to 30%. Therefore, it is easy to lead to misdiagnosis and increase the needless rate of puncture biopsy. On the other hand, the current computer-assisted breast ultrasound diagnosis requires lots of human interactions while the accuracy is not reliable enough. In this paper, an end-to-end model is proposed for automatically nodule classification. We presents a heterogeneous three-branch network (HTBN) for benign and malignant classification of the breast ultrasound images. In HTBN, the image information including ultrasound images, contrastenhanced ultrasound (CEUS) images and non-image information including patient’s age and other six pathological features are used simultaneously. In order to validate our method, a breast ultrasound data set with 1303 cases is collected. On this data set, the average diagnosis accuracy of physicians with five-year qualifications is 85.3%. However, the classification accuracy of our method is 92.41%. Through experiments, we confirmed our point of view that by incorporating medical knowledge into the optimization process, adding contrast-enhanced ultrasound images and non-image information to the network, the accuracy and robustness of breast diagnosis are greatly improved.