Pilot Performance Modeling via Observer-Based Inverse Reinforcement Learning

The focus of this brief is behavior modeling for pilots of unmanned aerial systems. The pilot is assumed to make decisions that optimize an unknown cost functional. The cost functional is estimated from observed trajectories using a novel inverse reinforcement learning (IRL) framework. The resulting IRL problem often admits multiple solutions. In this brief, a recently developed IRL observer is adapted to the pilot behavior modeling problem. The observer is shown to converge to one of the equivalent solutions of the corresponding IRL problem. The developed technique is implemented on a quadcopter where the pilot is a surrogate linear-quadratic controller that generates velocity commands for set-point regulation of the quadcopter. Experimental results demonstrate the ability of the developed method to learn equivalent cost functionals.