An efficient extraction method of journal-article table data for data-driven applications

Abstract

To improve the accuracy and automation of table extraction from journal articles, we present an efficient method for automatically extracting data from tables in PDF-based journal articles using table texts and border features. All characters and lines in each article are obtained from the text stream of the target PDF file. The table area is then located via the filtering rules and algorithm designed utilizing the obtained features of the table, such as text size, border length, and absolute location of elements. Furthermore, an improved hierarchical clustering algorithm is designed to restore the logical structure of the table, which includes single-linkage clustering and agglomerative nesting based on border constraints. By combining text block layout features, it restores the entire process of character merging, text block clustering, and cell clustering. Finally, by constructing a table structure to restore the correspondence between the header and body, the content output with the desired correct structure is achieved. The table area detection accuracy, logic and content extraction accuracy, information loss rate, extraction efficiency and comprehensive performance were utilized to quantify the performance. Through the extraction experiment of a dataset comprising 500 academic articles with 1157 tables, it indicated the weighted average F1 for table detection achieved 0.963, and the F1 values for logical-structure restoration and content accuracy reached 0.856 and 0.889, respectively. Compared to Tabula, ABBYY FineReader, and TabbyPDF, this method exhibited the highest efficiency, minimal information loss, and best overall performance. The proposed method enables rapid and large-scale acquisition of table data from PDF-based journal articles.