Autonomous optimization of cutting conditions in end milling operation based on deep reinforcement learning (Offline training in simulation environment for feed rate optimization)

Abstract

Full automation of manufacturing is strongly desired to improve the productivity. Autonomous optimization of the cutting conditions in the end milling operation is one of the challenges in achieving this goal. This paper proposes a system for optimization of the cutting conditions based on Deep Q-Network (DQN), which is a kind of deep reinforcement learning. An end mill is used as an agent and the end milling simulation is employed to provide the environment in the proposed system. Geometric information of interference state between tool and workpiece in the simulation is considered as the state of the environment and acceleration of feed rate is the action for the agent to take. The action is optimized by DQN to maximize the accumulated reward given from the environment, which evaluates how good the scenario of action is. Therefore, the cutting conditions can be optimized according to the defined reward function. We performed three case studies to verify our proposed method, in which the cutting torque is controlled to be a specified value. The objective was successfully achieved regardless of differences in the end milling scenario. The obtained results strongly suggested a fact that the reinforcement learning is a promising solution to autonomous optimization of the cutting conditions.