Land use changes profoundly affect the equilibrium of soil organic carbon (SOC) sequestration and greenhouse gas emissions. With the current global climatic changes, it is vital to understand the influence of ecological restoration and conservation management on the dynamics of SOC under different land uses, especially in erosion-endangered Loess soils. Therefore, we investigated changes in SOC through a suit of labile fractions, namely: light fraction organic C (LFOC), heavy fraction organic C (HFOC), coarse particulate organic C (CPOC), fine particulate organic C (FPOC), and dissolved organic C (DOC), from two forests i.e., Robinia pseudoacacia (RP) and Platycladus orientalis (PO), with different ages, in comparison with farmland (FL). The SOC and STN contents significantly increased over 42 years in the RP forest where the contents of CPOC and FPOC were significantly higher than in the FL. Moreover, total SOC and its labile fractions, in the studied land use types, significantly correlated with soil CaCO3, pH, and STN contents, indicating their key roles in SOC sequestration. The results reported here from different vegetation with different ages provide a better understanding of SOC and STN alterations at different stages of vegetation restoration. Our findings suggest that long-term natural vegetation restoration could be an effective approach for SOC sequestration and soil conservation on the Loess soil.