Context-aware inverse reinforcement learning for modeling individuals’ daily activity schedules

Abstract

Understanding individual and crowd dynamics in urban environments is critical for numerous applications, such as urban planning, traffic forecasting, and location-based services. Therefore, accurately modeling individuals' daily activity schedules is essential. Traditional methods, like utility-based and rule-based approaches, rely on expert knowledge and presumed model structures. While machine learning methods offer flexibility, they often ignore explicit behavioral mechanisms, particularly comprehensive discussion and integration of context related to individuals' daily travel. To address these, we propose a framework that integrates travel context with deep Inverse Reinforcement Learning (IRL), learning temporal patterns from sociodemographics, start time and duration of the current activity, travel modes, and land use. Specifically, individuals' activity schedules are initially formulated as a Markov Decision Process to simulate travelers’ sequential decision-making processes, laying the groundwork for the IRL framework; Then, a context-aware IRL method is proposed to model individuals' travel decision-making from observed temporal trajectories. Finally, we validate the proposed model by demonstrating its superior performance over discrete choice model and several well-known imitation learning benchmarks in tasks such as policy performance comparison, reward recovery, model generalizability, and computational efficiency using travel behavior datasets. This approach provides meaningful behavioral insights and paves the way for Artificial Intelligence-driven activity schedulers modeling.