Resin-mineral composite materials (RMC) have attracted much attention due to their excellent dynamic properties. However, the mechanical models related to RMC have not fully considered the complex interactions between components and interface transition zones (ITZ), and have also given less consideration to the influence of initial defects in the material, resulting in lower prediction accuracy of RMC mechanical models. To address the problem, based on composite sphere model, generalized autonomous method, and improved Mori–Tanaka method, the theoretical prediction model of RMC elastic modulus considering the influence of ITZ and pores is established in this study. Then, based on the micromechanical analysis method and combined with the theoretical data, the numerical prediction model of RMC elastic modulus considering the impact of pores and ITZ is founded. Furthermore, the influence of ITZ, pore, aggregate, and matrix parameters on the elastic modulus of RMC is investigated. The research results indicate that: (1) The error between the predicted RMC effective elastic modulus and the corresponding experimental values is within a reasonable range, indicating that the theoretical and numerical models proposed in this study are theoretically feasible. (2) ITZ and pore parameters have remarkable impact on the effective elastic modulus of RMC, indicating that it is indispensable to take into account ITZ and pores. (3) It is the elastic modulus of RMC that can be sensitive concerning the volume fraction and effective modulus of aggregate and matrix. The research results provide a theoretical basis for the design and application of RMC.