Anomalous/Relevant Event Detection (A/RED): Active Machine Learning for Finding Rare Events

In many industrial applications, data comes in the form of an unlabeled stream, likely containing classes that a user has not seen before. In these cases, a user generally cares about four things: classification, class discovery, notification of events in certain classes, and the amount of data they need to label. In this work we present Anomalous/ Relevant Event Detection (A/RED), an active learning system that operates upon imbalanced data streams to find new classes and classify incoming events. A/RED is unique in that it takes into account user preference for the relevance of classes. An A/RED query involves asking for a label and a binary relevance label. A relevant class is queried more often, and as a result, the classifier performs better for these instances.