Learning natural language filtering under noisy conditions

Abstract

Describes a novel AI technique, called a plausibility network, that allows for learning to filter natural language phrases according to predefined classes under noisy conditions. We describe the automatic knowledge acquisition for representing the words of natural language phrases using significance vectors and the learning of filtering of phrases according to ten different domain classes. We particularly focus on examining the filtering performance under noisy conditions, that is the degradation of these filtering techniques for incomplete phrases with unknown words. Furthermore, we show that this technique already scales up for a few thousand real-world phrases, that it compares favorably to some classification techniques from information retrieval, and that it can deal with unknown words as they might occur based on incomplete lexicons or speech recognizers.<>