2d-3d Hierarchical Feature Fusion Network For Segmentation Of Bone Structure In Knee Mr Image

Abstract

Automatic segmentation of knee bone structures is an important task in orthopedics diagnosis of knee disease based on MRI images. Inspired by doctors’ diagnosis of knee in sagittal plane of MR image, we propose to first calculate the sagittal maximum intensity projection (MIP) of MR image, then construct a high precision 2D-3D hierarchical feature fusion network for automatic segmentation of knee based on convolutional encoding and decoding architecture. It includes: 1) A 2D bypass network extracting global features based on MIP; 2) A 3D backbone network calculating local details based on MR volume; 3) A feature fusion module integrating 2D global context and 3D local details hierarchically. Particularly, the global features as anchor points will be fused with the local details at each level of the encoding path to enrich the context of local details and improve the segmentation accuracy. Our method is verified on SKI10 dataset. The average dice coefficients of femur, femoral cartilage, tibia and tibia cartilage are 0.978, 0.848, 0.979 and 0.848, respectively, and the segmentation performance is far better than the state-of-the-art methods.