Plant factory is one of the controlled environment agriculture forms with huge potential to alleviate food crisis, but the high energy cost restricts its widespread adoption. Numerous researches have explored various factors for energy-saving in plant factories in their settings, but there is a lack of analysis of the importance of these factors in energy saving. In this work, the energy-saving effect assessment of various factors in the container plant factory is investigated. Four cities (Harbin, Taiyuan, Shanghai, and Guangzhou), three plant densities (cultivation area: floor area=100 %, 150 %, and 200 %), and two temperature/humidity setpoints (20/22 ℃, 60/70 %, and 16/22 ℃, 50/95 %) are selected as operating conditions to cover different weather conditions and plant heat loads. The energy-saving effect of each factor is calculated using a random forest algorithm based on large amounts of energy simulation data. We identify that envelope overall heat transfer coefficient (U), air conditioner coefficient of performance (COP), and light efficacy (Efficacy) are three factors that have the largest impact on energy-saving in plant factories, in which light efficacy is the most important factor. Simultaneous optimization of these three factors could possibly reduce electricity consumption by ∼50 % compared to the base case. Finally, employing weight-light intensity correlation, the minimum specific energy consumption is approximately 4.76 kWh/kg lettuce fresh weight. This study utilizes advanced machine learning methods to sort out important factors and shows that significant energy reduction may be achieved by optimizing dominant factors, which gives a general guidance for future designers to build energy-efficient plant factories.