Long–Short Observation-driven Prediction Network for pedestrian crossing intention prediction with momentary observation

Abstract

Pedestrian crossing intention prediction aims to predict whether the pedestrian will cross the road, which is crucial for the decision-making of intelligent vehicles and ensuring traffic safety. Existing methods just rely on long-term observation and rarely consider it challenging to obtain sufficiently long and precise observation in real-world scenarios. Focus on momentary observation, which only contains two frames of the preceding and current time, we propose a novel Long–Short Observation-driven Prediction Network (LSOP-Net). LSOP-Net comprises two critical components, the Momentary Observation feature Extraction Module (MOE-Module) and the Multimodal Long–Short-term feature Fusion Module (MLSFusion). Utilizing a hybrid training strategy and an external long-term feature pool, the MOE-Module is proposed to extract features with long-term patterns from momentary observations, which effectively mitigates feature deficiency arising from momentary observations. Based on a feature selection fusion mechanism, the MLSFusion is proposed to explicitly model the importance relationship between various modalities’ long–short-term features and the output, which adaptively fuses the long–short-term features from various modalities. Experimental results on the JAAD and PIE datasets demonstrate that our approach achieves superior performance in pedestrian crossing intention prediction with momentary observation.