Adaptive ensemble learning for efficient keyphrase extraction: Diagnosis, aggregation, and distillation

Abstract

Keyphrase extraction (KE) refers to the process of identifying words or phrases that signify the primary themes of a document. Although keyphrase extraction is important in many downstream applications, including scientific document indexing, search, and question answering, the challenge lies in executing this extraction both adaptively and effectively. To this end, we propose a novel Distillation-based Adaptive Ensemble Learning (DAEL) method specifically designed for efficient keyphrase extraction, encompassing diagnosis, aggregation, and distillation processes. Specifically, we initiate with a Cognitive Diagnosis Module (CDM) to evaluate the diverse capabilities of individual KE models. Following this, an Adaptive Aggregation Module (AAM) is employed to create a weight distribution uniquely suited to each data instance. The process concludes with a Knowledge Distillation Module (KDM) to distill the superior performance of the ensemble model into a single model, thereby refining its efficiency and reducing computational cost. Extensive testing on real-world datasets highlights the superior performance of the proposed model. In comparison with leading-edge methods, our approach notably excels in processing text with complex structures or significant noise, marking a substantial advancement in KE effectiveness.