Entity Extraction of Customs Clearance Data Based on CRF and Specific Rules

Abstract

For the problem of entity extraction on dirty data in customs import and export domain, this paper proposes a data cleaning method based on specific rules and machine learning for commodity named entity extraction. First, KNN (k-Nearest Neighbor) classification algorithm is used to solve mismatches of attributes and their values of fields in domain data tuple. second, commodity named entities and sub attributes of fields are extracted by specific rules and CRF (conditional random field algorithm) model, proposed method can extract correct sub-attributes of each entity. Experiment results proved the advantages of proposed method in comparison with other methods in terms of precision and recall.