The structural optimization of electric vehicles involves numerous design variables and constraints, making it a complex engineering optimization task over the past decades. Many population-based evolutionary algorithms encounter issues such as converging to local optima and lacking population diversity when tackling such optimization problems. Consequently, the solutions obtained for the optimization may be flawed or suboptimal. To address these problems, an improved genetic algorithm (GA) based on reinforcement learning is proposed in this paper. The proposed method introduces a population delimitation method based on individual fitness ranking. The population is divided into two parts: the excellent population and the ordinary population, and different selection and cross-mutation methods are applied to each part separately. More efficient crossover and mutation methods are then applied to the ordinary population to enhance the generation of excellent individuals. Furthermore, the proposed approach replaces the traditional fixed crossover and mutation rates with a dynamic selection method based on reinforcement learning to enhance optimization efficiency. A markov decision process model is constructed based on GA environment in this context. The population state determination method and reward method are designed for reinforcement learning in the GA environment, dynamically selecting the most appropriate genetic parameters based on the current state of the population. Finally, the uncertainty in the manufacturing process is introduced into the optimization problem and the case study results demonstrate that the proposed reinforcement learning-based GA significantly outperforms other evolutionary algorithms when applied to solving the structural optimization of electric vehicles.