OCR from Video Stream of Book Flipping

Abstract

Optical Character Recognition (OCR) in video stream of flipping pages is a challenging task because flipping at random speed cause difficulties to identify frames that contain the open page image (OPI) for better readability. Also, low resolution, blurring effect shadows add significant noise in selection of proper frames for OCR. In this work, we focus on the problem of identifying the set of optimal representative frames for the OPI from a video stream of flipping pages and then perform OCR without using any explicit hardware. To the best of our knowledge this is the first work in this area. We present an algorithm that exploits cues from edge information of flipping pages. These cues, extracted from the region of interest (ROI) of the frame, determine the flipping or open state of a page. Then a SVM classifier is trained with the edge cue information for this determination. For each OPI we obtain a set of frames. Next we choose the central frame from that set of frames as the representative frame of the corresponding OPI and perform OCR. Experiments are performed on video documents recorded using a standard resolution camera to validate the frame selection algorithm and we have obtained 88% accuracy. Also, we have obtained character recognition accuracy of 82% and word recognition accuracy of 77% from such book flipping OCR.