A Practical Real-Time Flight Delay Prediction System using Big Data Technology

Flight delay is an unexpected incident in the field of aviation in particular and transportation in general. Predicting the possibility or delay of flights plays a vital role in proactively arranging a time for the airline as well as increasing the reputation of the airline among users. This study presents an implementation of a real-time flight delay prediction system. To ensure the practicality, the entire system is built using big data technology. Apache Kafka is used to stream the flight data to trained machine learning models integrated inside Apache Spark to output real-time prediction results, which will be displayed through a dash-board and stored in Cassandra database simultaneously. Consequently, the system can process a huge amount of input data and produce prediction results in real-time. Index Terms— Flight Delay Prediction, Machine Learning, Spark, Kafka, Streaming, Cassandra.