Unsupervised Markdown Feature-Aware Keywords Extraction Towards Technology Blogs

Abstract

A vast amount of blogs are generated from online technology communities every day. Most of them are in Markdown format. The increase of Markdown documents has brought opportunities and challenges to many natural language processing tasks. Extracting keywords from technology blogs is of great value for discovering, retrieving, and sharing knowl-edge about technical blogs. The mainstream keyword extraction algorithms remain to use statistical char-acteristics of words to determine the keywords of a document, seldom considering the structure char-acteristics of the document that potentially express the semantic information. We argue that Markdown markup features as well as the textual content of the document are both concerned with the keywords extraction. In this paper, we propose a novel un-supervised Markdown markup features aware key-words extraction algorithm for technology blogs. The algorithm integrates Markdown markup syntax in-formation with a blog text representation. Through experiments against TF-IDF, TextRank, and PositionRank algorithms on a real Markdown document dataset, our algorithm achieves higher performance with a substantial improvement when the number of keywords extracted is greater than 3.