Guidance-As-Progressive in Human Skill Training Based on Deep Reinforcement Learning

To achieve psychological inclusion and skill development orientation in human skill training, this paper proposes a haptic-guided training strategy generation method with Deep Reinforcement Learning (DRL)-based agent as the core and Zone of Proximal Development (ZPD) tuning as the auxiliary. The information of the expert and trainee is stored first with a designed database that can be accessed in real-time, which establishes the data foundation. Then, under the DRL framework, a strategy generation agent is designed, which consists of an actor-network and two Q-networks. The former network generates the agent’s decision policy, while the other two Q-networks work to approximate the state-action value function, and the parameters of all of them are administrated by the Soft Actor-Critic (SAC) algorithm. In addition, for the first time, the psychological ZPD evaluation method is integrated into the strategy generation of the DRL-based agent, which is utilized to describe the relationship between a trainees intrinsic skills and guidance. With it, the problem of transitional guidance or insufficient guidance can be handled well. Finally, simulation experiments validate the proposed method, demonstrating its efficiency in regulating the trainee under favorable training conditions.