Multi-Class Document Image Classification using Deep Visual and Textual Features

Abstract

The digitalization era has brought digital documents with it, and the classification of document images has become an important need as in classical text documents. Document images, in which text documents are stored as images, contain both text and visual features, unlike images. Therefore, it is possible to use both text and visual features while classifying such data. Considering this situation, in this study, it is aimed to classify document images by using both text and visual features and to determine which feature type is more successful in classification. In the text-based approach, each document/class is labeled with the keywords associated with that document/class and the classification is realized according to whether the document contains the related key-words or not. For visual-based classification, we use four deep learning models namely CNN, NASNet-Large, InceptionV3, and EfficientNetB3. Experimental study is carried out on document images obtained from applicants of the Kocaeli University. As a result, it is seen ii that EfficientNetB3 is the most superior among all with 0.8987 F-score.