Automatically detecting scientific political science texts from a large general document index

This technical report outlines the filtering approach applied to the collection of the Bielefeld Academic Search Engine (BASE) data to extract articles from the political science domain. We combined hard and soft filters to address entries with different available metadata, e.g. title, abstract or keywords. The hard filter is a weighted keyword-based filter approach. The soft filter uses a multilingual BERT-based classification model, trained to detect scientific articles from the political science domain. We evaluated both approaches using an annotated dataset, consisting of scientific articles from different scientific domains. The weighted keyword-based approach achieved the highest total accuracy of 0.88. The multilingual BERT-based classification model was fine-tuned using a dataset of 14,178 abstracts from scientific articles and reached the highest total accuracy of 0.98.