Automatic extraction of multiword expression candidates for Indonesian language

Abstract

The utilization of dictionary-based multiword expressions (MWEs) has limitation regarding the availability of the word combination, because there are many possible multiword expressions that can be extracted from a text. This research is a preliminary study to extract multiword expressions from a text for Indonesian language. The aim of this study is determining the best method to extract multiword expression candidates for Indonesian language. This research proposed a method to extract multiword expression candidates from texts in a corpus. The text is tokenized and then filtered with stop words to remove unnecessary words. The result of these steps is multiword expression candidates that are still mixed with common and uncommon multiword expressions. To filter uncommon multiword expressions, they are ranked with the other multiword expressions from the other texts within the same corpus by using TF-IDF algorithm. This research evaluates three options for extracting multiword expression candidates. The option which utilizes combination of special characters and stop words to determine word combination is promising because it excels in combining word rate, has more appropriate multiword expression candidates, while it spends almost the same amount of memory usage compared to the others.