Sparse adaptive memory and handwritten digit recognition

Abstract

Pattern recognition is a budding field with many possible approaches. This article describes sparse adaptive memory (SARI), an associative memory built upon the strengths of Parzen classifiers, nearest neighbor classifiers, feedforward neural networks, and is related to learning vector quantization. A key feature of this learning architecture is the ability to adaptively change its prototype patterns in addition to its output mapping. As SAM changes the prototype patterns in the list, it isolates modes in the density functions to produce a classifier that is in some senses optimal. Some very important interactions of gradient descent learning are exposed, providing conditions under which gradient descent will converge to an admissible solution in an associative memory structure. A layer of learning heuristics can be built upon the basic gradient descent learning algorithm to improve memory efficiency in terms of error rate, and therefore hardware requirements. A simulation study examines the effects of one such heuristic in the context of handwritten digit recognition.<>