Equipment Sensor Data Cleansing Algorithm Design for ML-Based Anomaly Detection

Abstract

Anomaly detection (AD) by exploiting machine learning (ML) of equipment sensory data can make significant contributions to yield improvements. Data cleansing is critical to provide ML-based AD with fixed-length input without distortion of data characteristics. We present a novel data cleansing design. Design innovations are: process step and mode-based input data length determination, importance indicator of sample data based on relative difference, and data cleansing priority by exploiting importance indicator and entropy. Experiment results demonstrate our cleansing design is superior to two frequently used methods in preserving data characteristics for effective AD by using an unsupervised ML approach.