Food security is a critical global issue, requiring that the challenges of farmland abandonment be addressed. As a vital component of agricultural production in China, farmland use is affected by mechanization conditions and the crop cultivation decisions of rural households. With the decreasing economic viability of farmland in China’s mountainous areas, the effects of mechanization conditions and non-grain crop production on farmland abandonment remain controversial. This study developed a theoretical framework and used unique survey data from 2400 rural households in China’s mountainous areas to analyze the relationship between mechanization conditions, non-grain crop production, and farmland abandonment using multiple models, heterogeneity analysis, and factual characteristics analysis. These findings demonstrate that improving mechanization conditions and encouraging non-grain crop production could mitigate farmland abandonment. Non-grain crop production could mitigate abandonment through increasing agricultural labor input, and improving mechanization conditions could inhibit non-grain crop production. Additionally, improving mechanization conditions had a more significant effect on households with higher education level and lower incomes, while encouraging non-grain crop production showed an opposite trend. Both measures had greater effect in the western and low-altitude regions. Therefore, to protect farmland and ensure food security in mountainous areas, it is necessary to continue land consolidation appropriately, actively encourage non-grain crop production, and comprehensively balance the demands of different management objectives.