The pattern of connectivity between microfractures and matrix pores is important for in-depth understanding of ultra-deep porous-fractured clastic reservoirs. The evaluation of reservoir microstructure with digital core technology is an effective method. However, the information of fractures and pores cannot be acquired simultaneously from samples by merely using the traditional single-scale digital core analysis method, such that an integrated analysis of pores and fractures is impossible, which seriously restricts the evaluation of porous-fractured clastic reservoir. In this study, the 3D grayscale images of fractured samples are obtained through low-resolution micro-CT scanning, and the Otsu algorithm is adopted for binary segmentation of the 3D grayscale images. Then, matrix samples are drilled at the positions with small, moderate and large distances to fractures respectively for high-resolution scanning, and the maximum class spacing algorithm is used for binary segmentation to construct 3D fractured-porous digital core and the maximal ball algorithm is applied to extract the 3D fracture and pore network model. Finally, a fracture-matrix pore multiscale network model is built by the integration method. The results show that the matrix porosity decreases significantly with the increase of distance to the microfracture, that is, the further the distance to the microfracture, the smaller the matrix porosity. Therefore, a single small-scale pore-fracture model cannot reflect the structural characteristics of large-scale samples, which further verifies the necessity of establishing a representative pore-fracture fusion model. Since the fracture-matrix pore multiscale network model contains both fracture and matrix pore characteristics, the corresponding physical properties can be calculated. The fusion model has a total porosity of 13.6% and the pore-throat diameter/fracture aperture ranging in 3–2400 μm. The proposed modeling method provides a basic study platform for investigating the pore-fracture connectivity pattern in ultra-deep clastic reservoirs.