In the present work, the sheet molding compound (SMC) composites with variable filler, resin, and fiber composition were prepared to investigate the effects of composition on the acid resistance of SMC composites. The water absorption was measured to estimate the influence of water on the mechanical properties and deterioration of SMC composites. Hardness and flexural properties tests were performed to investigate the evolution of degradation. The degradation mechanism was revealed by analyzing the change in molecule configuration, evaluating the thermal stability, and examining the fractured section. The chemical resistance and addition amount of the fillers had significant impacts on the acid resistance of SMC composites. The resin characteristics, molecule weight, free volume size, and polar group number had an important impact on the water absorption and acid resistance of SMC composites. Exposed to 25 °C and 20 wt% sulfuric acid solution for 28 days, the SMC composites exhibited a minor degradation of hollow glass microspheres (HGMs) and glass fibers (GFs). In this condition, the deterioration of mechanical properties was dominated by the resin matrix plasticization and decomposition, along with the interface degradation. It can be concluded that the original defects and weak interacted regions in the composites system initiated the degradation of SMC composites, whereas the microstructure and composition of SMC composites dominated the degradation progress.