The surrounding rock condition of tunneling palm face can be obtained by recognizing the degree of rock fragmentation on the tunnel boring machine (TBM) conveyor belt. However, prolonged exposure to water mist and dust significantly degrades the image quality of rock fragmentation, which poses a significant challenge to achieving accurate image recognition. This paper proposes a multi-scale hybrid connected neural network (MHCNN) for anti-noise rock fragmentation classification in TBM construction. The proposed method designs two neural network branches to extract multi-dimensional features from rock fragmentation images and share features locally, and constructs multiple residual and dense connection blocks to capture the edge features of rock fragmentation. Moreover, a feature transfer bridge based on the feature transfer block is designed, which can adjust the feature dimensions as well as the weights, for sharing the features extracted from the special nodes of the two branches. Finally, the dataset taken at the site of Baolin Tunnel is used to verify the robustness and superiority of the method, and fully compare with the state-of-the-art algorithms. The experimental results show that the recall of MHCNN is 93.11% in the noiseless dataset, which is 5.19%-51.15% higher compared to other methods. The recall is 5.68%-58.93%, 14.86%-55.56%, and 26.61%-44.7% higher in light, medium, and heavy water-mist covered datasets, respectively. The recall of the proposed method increases by 2.88%-45.85%, 12.2%-41.99%, and 19.49%-44% for light, moderate, and heavy dust obscuration, respectively, which confirms the strong robustness and value of the method for practical engineering applications.