A hybrid intelligent system for assisting low-vision people with over-the-counter medication

Abstract

Because people with low vision have difficulty pur-chasing and taking the correct medicine and dosages on time, this paper presents a system with a Flask Server and Android application that assists low-vision people with using over-the-counter (OTC) medication correctly. The system is mainly divided into three parts: an Android application, a Flask server and a MongoDB database. The application provides a medication time reminder, medicine information retrieval and image capture for recognition functions. A server recognizes the medication package by combining optical character recognition (OCR) and an image classification convolutional neural network (CNN). A database is used to store and provide medicine information. The experimental results show that the recognition performance has up to 96.1% accuracy. Moreover, the approach is shown to be able to handle out-of-distribution images.