Multi-Objective Inverse Reinforcement Learning via Non-Negative Matrix Factorization

Abstract

In recent years, inverse reinforcement learning, which estimates the reward from the sequence of states followed by an expert (trajectory), has been attracting attention in terms of imitating complex behaviors and estimating the intentions of people or animals. Existing inverse reinforcement learning methods assume that the expert has a single objective. However, it is more natural to assume that experts have multiple objectives in the real world. A previous paper proposed a method for estimating an expert's preferences for each objective (i.e., weights) when the true multi-objective reward vector is known. In this study, we formulated the simultaneous estimation of the multi-objective reward vector and weights as a multi-objective inverse reinforcement learning (MOIRL) problem where both are unknown. In this paper, we propose a MOIRL method based on non-negative matrix factorization. Through the results of computational experiments, we show that the proposed method can estimate the rewards and weights from trajectories obtained in a multi-objective environment.