Comparative Analysis of Machine and Deep Learning Techniques for Text Classification with Emphasis on Data Preprocessing

Abstract

Physician-written discharge medical notes include vital details regarding their patients' health. Numerous deep learning algorithms have shown effective in gleaning crucial insights from unstructured medical notes data, leading to potentially useful outcomes in the medical field. The goal of this research is to determine how different deep learning algorithms perform as models for text classification issues in long short term memory (LSTM). Titanic Disaster Dataset has been used for pre-processing is essential since there is a lot of unnecessary information in textual data. Next, clean up the data by eliminating duplicate rows and filling in the blanks. Besides traditional machine learning algorithms such as naive bayes (NB), gradient boosting (GB), and support vector machine (SVM), we use deep learning algorithms to classify data, including bidirectional – LSTM using Conditional Random Fields (CRFs). BiLSTM is the most precise model compared to other models and baseline research, with a classification accuracy of 98.5%.