Guided deep reinforcement learning framework using automated curriculum scheme for accurate motion planning

Collaborative robotic arms in smart factories should ensure the safety and interactivity during their operation such as reaching and grasping objects. Especially, the advanced motion planner including the path planning and the motion control functions is essential for human-machine co-working. Since the traditional physics-based motion planning approaches require extreme computational resources to obtain near-optimal solutions, deep reinforcement learning algorithms have been actively adopted and have effectively solved the limitation. They, however, have the easy task preference problem, primarily taking the simpler ways for the more rewards, due to randomly training the agents how to reach the target points in the large-scale search spaces. Therefore, we propose a novel curriculum-based deep reinforcement learning framework that makes the agents learn the motion planning tasks in unbiased ways from the ones with the low complexities to the others with the high complexities. It uses the unsupervised learning algorithms to cluster the target points with the similar task complexities for generating the effective curriculum. In addition, the review and buffer flushing mechanisms are integrated into the framework to mitigate the catastrophic forgetting problem where the agent abruptly lose the previous learned knowledge upon learning new one in the curriculum. The evaluation results of the proposed framework show that the curriculum significantly enhances the success rate on the task with the highest complexity from 12% to 56% and the mechanisms improve the success rate on the tasks with the easier complexities from an average of 66% to 76.5%, despite requiring less training time.