Cover crops are one of the climate-smart agricultural practices used to increase soil organic carbon (SOC) sequestration. However, the SOC sequestration potential and underlying mechanisms under different cover crops, especially in orchard agroecosystems, have not been fully elucidated. Here, we investigated three orchards in China using SOC fractionation methods, high-throughput sequencing, and biomarker analysis. Our objectives were to determine the effect of cover crops on the physical fractions and chemical compositions of SOC, as well as on microbial properties, and to clarify why legume and non-legume cover crops sequester SOC differently. The results showed different increases in SOC between legume and non-legume cover crops (+38% vs. +16%) compared with those in the control plots without cover crops. Legume cover crops increased mineral-associated and particulate organic carbon, whereas non-legume cover crops increased mineral-associated organic carbon only. These differences were attributed to their distinct effects on microbial SOC transformation pathways. Legume cover crops positively impacted microbial pathways by increasing the availability of soil substrates and nitrogen, such as dissolved organic carbon (+84%), O-alkyl carbon (+18%), and ammonium nitrogen (+42%). These results were supported by the increases in carbon and nitrogen enzyme activities, microbial community diversity indices, the abundance of dominant fungal taxa (Sordariomycetes), microbial biomass carbon (+105%), and microbial necromass carbon (+47%). Non-legume cover crops might have induced microbial nitrogen starvation, decreasing the efficiency of microbial pathways, as evidenced by the low β-glucosidase to β-N-acetylglucosaminidase ratios (−7%) and the lack of significant changes in the bacterial Shannon index or microbial necromass carbon. In addition, redundancy analysis revealed that enzyme activity, the microbial community, and microbial necromass carbon collectively dominated the changes in the SOC physical fraction. Site-specific soil properties such as soil texture and nitrogen availability were important factors influencing SOC sequestration under cover crops. Our study provides essential insights for optimizing cover crop management to increase SOC sequestration in orchard agroecosystems.