Dingyu Wang, Shaocheng Jia, Xin Pei, Chunyang Han, Danya Yao, Dezhi Liu
{"title":"DERNet:使用车载摄像头识别驾驶员情绪","authors":"Dingyu Wang, Shaocheng Jia, Xin Pei, Chunyang Han, Danya Yao, Dezhi Liu","doi":"10.1109/mits.2023.3333882","DOIUrl":null,"url":null,"abstract":"Driver emotion is considered an essential factor associated with driving behaviors and thus influences traffic safety. Dynamically and accurately recognizing the emotions of drivers plays an important role in road safety, especially for professional drivers, e.g., the drivers of passenger service vehicles. However, there is a lack of a benchmark to quantitatively evaluate the performance of driver emotion recognition performance, especially for various application situations. In this article, we propose an emotion recognition benchmark based on the driver emotion facial expression (DEFE) dataset, which consists of two splits: training and testing on the same set (split 1) and different sets (split 2) of drivers. These two splits correspond to various application scenarios and have diverse challenges. For the former, a driver emotion recognition network is proposed to provide a competitive baseline for the benchmark. For the latter, a novel driver representation difference minimization loss is proposed to enhance the learning of common representations for emotion recognition over different drivers. Moreover, the minimum required information for achieving a satisfactory performance is also explored on split 2. Comprehensive experiments on the DEFE dataset clearly demonstrate the superiority of the proposed methods compared to other state-of-the-art methods. An example application of applying the proposed methods and a voting mechanism to real-world data collected in a naturalistic environment reveals the strong practicality and readiness of the proposed methods. The codes and dataset splits are publicly available at <uri xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">https://github.com/wdy806/CDERNet/</uri>\n.","PeriodicalId":48826,"journal":{"name":"IEEE Intelligent Transportation Systems Magazine","volume":null,"pages":null},"PeriodicalIF":4.3000,"publicationDate":"2023-12-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"DERNet: Driver Emotion Recognition Using Onboard Camera\",\"authors\":\"Dingyu Wang, Shaocheng Jia, Xin Pei, Chunyang Han, Danya Yao, Dezhi Liu\",\"doi\":\"10.1109/mits.2023.3333882\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Driver emotion is considered an essential factor associated with driving behaviors and thus influences traffic safety. Dynamically and accurately recognizing the emotions of drivers plays an important role in road safety, especially for professional drivers, e.g., the drivers of passenger service vehicles. However, there is a lack of a benchmark to quantitatively evaluate the performance of driver emotion recognition performance, especially for various application situations. In this article, we propose an emotion recognition benchmark based on the driver emotion facial expression (DEFE) dataset, which consists of two splits: training and testing on the same set (split 1) and different sets (split 2) of drivers. These two splits correspond to various application scenarios and have diverse challenges. For the former, a driver emotion recognition network is proposed to provide a competitive baseline for the benchmark. For the latter, a novel driver representation difference minimization loss is proposed to enhance the learning of common representations for emotion recognition over different drivers. Moreover, the minimum required information for achieving a satisfactory performance is also explored on split 2. Comprehensive experiments on the DEFE dataset clearly demonstrate the superiority of the proposed methods compared to other state-of-the-art methods. An example application of applying the proposed methods and a voting mechanism to real-world data collected in a naturalistic environment reveals the strong practicality and readiness of the proposed methods. The codes and dataset splits are publicly available at <uri xmlns:mml=\\\"http://www.w3.org/1998/Math/MathML\\\" xmlns:xlink=\\\"http://www.w3.org/1999/xlink\\\">https://github.com/wdy806/CDERNet/</uri>\\n.\",\"PeriodicalId\":48826,\"journal\":{\"name\":\"IEEE Intelligent Transportation Systems Magazine\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":4.3000,\"publicationDate\":\"2023-12-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Intelligent Transportation Systems Magazine\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://doi.org/10.1109/mits.2023.3333882\",\"RegionNum\":3,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Intelligent Transportation Systems Magazine","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1109/mits.2023.3333882","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
DERNet: Driver Emotion Recognition Using Onboard Camera
Driver emotion is considered an essential factor associated with driving behaviors and thus influences traffic safety. Dynamically and accurately recognizing the emotions of drivers plays an important role in road safety, especially for professional drivers, e.g., the drivers of passenger service vehicles. However, there is a lack of a benchmark to quantitatively evaluate the performance of driver emotion recognition performance, especially for various application situations. In this article, we propose an emotion recognition benchmark based on the driver emotion facial expression (DEFE) dataset, which consists of two splits: training and testing on the same set (split 1) and different sets (split 2) of drivers. These two splits correspond to various application scenarios and have diverse challenges. For the former, a driver emotion recognition network is proposed to provide a competitive baseline for the benchmark. For the latter, a novel driver representation difference minimization loss is proposed to enhance the learning of common representations for emotion recognition over different drivers. Moreover, the minimum required information for achieving a satisfactory performance is also explored on split 2. Comprehensive experiments on the DEFE dataset clearly demonstrate the superiority of the proposed methods compared to other state-of-the-art methods. An example application of applying the proposed methods and a voting mechanism to real-world data collected in a naturalistic environment reveals the strong practicality and readiness of the proposed methods. The codes and dataset splits are publicly available at https://github.com/wdy806/CDERNet/
.
期刊介绍:
The IEEE Intelligent Transportation Systems Magazine (ITSM) publishes peer-reviewed articles that provide innovative research ideas and application results, report significant application case studies, and raise awareness of pressing research and application challenges in all areas of intelligent transportation systems. In contrast to the highly academic publication of the IEEE Transactions on Intelligent Transportation Systems, the ITS Magazine focuses on providing needed information to all members of IEEE ITS society, serving as a dissemination vehicle for ITS Society members and the others to learn the state of the art development and progress on ITS research and applications. High quality tutorials, surveys, successful implementations, technology reviews, lessons learned, policy and societal impacts, and ITS educational issues are published as well. The ITS Magazine also serves as an ideal media communication vehicle between the governing body of ITS society and its membership and promotes ITS community development and growth.