Decoding the silent dialogue: Unveiling driver-pedestrian communication dynamics with a hidden Markov model

Abstract

Motivation

To enable vehicle automation to negotiate in a mixed traffic environment there is a need for formal methods that describe the communication process between drivers and pedestrians.

Aim

This work aims to model the underlying states of communication that a dyad of a pedestrian-driver system experiences when a pedestrian crosses the road in front of an approaching vehicle.

Method

We analyzed a naturalistic driving dataset comprised of 328 encounters which were identified using a forward-facing camera from 61 drivers. A Hidden Markov Model (HMM) was employed to model the underlying communication states in these encounters. The model accounted for observable behaviors, including vehicle and pedestrian movement, pedestrian and driver gaze, and instances of driver waving, capturing the dynamic interplay between both parties.

Results

The HMM revealed six distinct communication states: four representing unidirectional communication, where the driver either braked or accelerated/cruised while looking toward or away from the pedestrian and two bidirectional communication states including signs of negotiation, where drivers and pedestrians were likely to look in the direction of one another and pedestrians were likely to pause, with or without vehicle movement. Notably, bidirectional communication occurred in approximately 60% of the encounters, typically early on, but diminished as the encounter progressed. In contrast, unidirectional communication involving an accelerating/cruising vehicle became more frequent toward the end of the encounters.

Conclusion

This study introduces a novel model for analyzing pedestrian-driver encounters, highlighting the importance of bidirectional communication in the early stages of interactions. The findings suggest that not all encounters involve reciprocal communication and that communication patterns shift over time. Future work can build on this model to identify instances of communication breakdowns and explore potential strategies for mitigating them.