Lexical-index-based Japanese syntax matching

Abstract

Syntax matching is a challenging basic issue, and related algorithms can be widely used in natural language processing. This paper addresses the problem of how to efficiently match a sentence with the most similar syntactic structure to a given Japanese sentence from a big set of Japanese sentences, designs a novel lexical index data structure of hiratoken-sentence index (HSI) according to our Japanese syntax identification hypothesis of hiragana token, and proposes a HSI-based Japanese syntax matching (HSIJSM) algorithm. Supported by the HSI data structure, the HSIJSM algorithm can approximately get the syntactic similarity after the fast calculating of formal similarity between two Japanese sentences. The experimental results show that the HSIJSM algorithm can achieve the preferable performance with greatly reduced time costs.