Research on Eliminating Abnormal Big Data based on PSO-SVM

Abstract

In order to improve detection rate and reduce missing detection rate and false detection rate of big data, an abnormal large data elimination method based on PSO-SVM is proposed. Big data is chosen as a set, proximity of which is measured, according to fuzzy sets in fuzzy theory to measure data’ similarity degree. In order to determine redundant data and judge whether big data is abnormal, using support vector machine to train each particle and get fitness function through measuring the proximity between data by a constructed function, and then eliminating abnormal big data through the sliding window. Taking KDD99 big data as object, simulation experiment has higher detection rate and low false detection rate based on PSO-SVM method.