A Scalable Data Analytics Framework for Connected Vehicles Using Apache Spark

Abstract

Connected and autonomous vehicles are equipped with a large number of sensors, absorbing information from the vehicles and as well as from the environment. Handling and processing this vast amount of generated data is a big challenge. The first objective of this paper is to present a flexible framework for vehicle data processing, including driving behavior analysis. The framework is based on Apache Spark, a widely used big data engine. Secondly, the paper shows that the frameworks scales almost linearly with distributed big data analytics, and also performs well on a stand-alone multi-core machine.