{"title":"共享控制自动驾驶的神经网络接管时间预测分析","authors":"C. Pasareanu","doi":"10.1145/3459086.3459630","DOIUrl":null,"url":null,"abstract":"Autonomous driving systems may encounter situations where it is necessary to transfer control to the human driver, for instance when encountering unpredictable dangerous road conditions. To be able to do so safely, the autonomous system needs an estimate of how long it will take for the human driver to take control of the vehicle. A neural network can be used for making such predictions. However ensuring that such a neural network can be used in safety-critical situations is very challenging. We discuss our recent efforts for building, analysing and formally verifying a neural network built for predicting takeover time in a shared-control autonomous driving system. The network was trained on data collected from a (semi-) autonomous driving simulator. We evaluated several techniques for the analysis of the neural network as follows. We performed robustness and sensitivity analysis for the neural network, using the Marabou formal verification tool. We evaluated off-the-shelf attribution tools to determine the important features upon which the neural network makes its predictions. We investigated trust and confidence analysis to better understand the neural network outputs. And finally, we performed adversarial training to improve the quality of the neural network. We discuss our results and outline directions for future work.","PeriodicalId":127610,"journal":{"name":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2021-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Analysis of neural network takeover-time predictions for shared-control autonomous driving\",\"authors\":\"C. Pasareanu\",\"doi\":\"10.1145/3459086.3459630\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Autonomous driving systems may encounter situations where it is necessary to transfer control to the human driver, for instance when encountering unpredictable dangerous road conditions. To be able to do so safely, the autonomous system needs an estimate of how long it will take for the human driver to take control of the vehicle. A neural network can be used for making such predictions. However ensuring that such a neural network can be used in safety-critical situations is very challenging. We discuss our recent efforts for building, analysing and formally verifying a neural network built for predicting takeover time in a shared-control autonomous driving system. The network was trained on data collected from a (semi-) autonomous driving simulator. We evaluated several techniques for the analysis of the neural network as follows. We performed robustness and sensitivity analysis for the neural network, using the Marabou formal verification tool. We evaluated off-the-shelf attribution tools to determine the important features upon which the neural network makes its predictions. We investigated trust and confidence analysis to better understand the neural network outputs. And finally, we performed adversarial training to improve the quality of the neural network. We discuss our results and outline directions for future work.\",\"PeriodicalId\":127610,\"journal\":{\"name\":\"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2021-05-18\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/3459086.3459630\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3459086.3459630","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Analysis of neural network takeover-time predictions for shared-control autonomous driving
Autonomous driving systems may encounter situations where it is necessary to transfer control to the human driver, for instance when encountering unpredictable dangerous road conditions. To be able to do so safely, the autonomous system needs an estimate of how long it will take for the human driver to take control of the vehicle. A neural network can be used for making such predictions. However ensuring that such a neural network can be used in safety-critical situations is very challenging. We discuss our recent efforts for building, analysing and formally verifying a neural network built for predicting takeover time in a shared-control autonomous driving system. The network was trained on data collected from a (semi-) autonomous driving simulator. We evaluated several techniques for the analysis of the neural network as follows. We performed robustness and sensitivity analysis for the neural network, using the Marabou formal verification tool. We evaluated off-the-shelf attribution tools to determine the important features upon which the neural network makes its predictions. We investigated trust and confidence analysis to better understand the neural network outputs. And finally, we performed adversarial training to improve the quality of the neural network. We discuss our results and outline directions for future work.