使用维基百科文章创建印尼语-爪哇语平行语料库

Bayu Distiawan Trisedya, Dyah Inastra
{"title":"使用维基百科文章创建印尼语-爪哇语平行语料库","authors":"Bayu Distiawan Trisedya, Dyah Inastra","doi":"10.1109/ICACSIS.2014.7065828","DOIUrl":null,"url":null,"abstract":"Parallel corpora are necessary for multilingual researches especially in information retrieval (IR) and natural language processing (NLP). However, such corpora are hard to find, specifically for low-resources languages like ethnic languages. Parallel corpora of ethnic languages were usually collected manually. On the other hand, Wikipedia as a free online encyclopedia is supporting more and more languages each year, including ethnic languages in Indonesia. It has become one of the largest multilingual sites in World Wide Web that provides free distributed articles. In this paper, we explore a few sentence alignment methods which have been used before for another domain. We want to check whether Wikipedia can be used as one of the resources for collecting parallel corpora of Indonesian and Javanese, an ethnic language in Indonesia. We used two approaches of sentence alignment by treating Wikipedia as both parallel corpora and comparable corpora. In parallel corpora case, we used sentence length based and word correspondence methods. Meanwhile, we used the characteristics of hypertext links from Wikipedia in comparable corpora case. After the experiments, we can see that Wikipedia is useful enough for our purpose because both approaches gave positive results.","PeriodicalId":443250,"journal":{"name":"2014 International Conference on Advanced Computer Science and Information System","volume":"76 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2014-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"3","resultStr":"{\"title\":\"Creating Indonesian-Javanese parallel corpora using wikipedia articles\",\"authors\":\"Bayu Distiawan Trisedya, Dyah Inastra\",\"doi\":\"10.1109/ICACSIS.2014.7065828\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Parallel corpora are necessary for multilingual researches especially in information retrieval (IR) and natural language processing (NLP). However, such corpora are hard to find, specifically for low-resources languages like ethnic languages. Parallel corpora of ethnic languages were usually collected manually. On the other hand, Wikipedia as a free online encyclopedia is supporting more and more languages each year, including ethnic languages in Indonesia. It has become one of the largest multilingual sites in World Wide Web that provides free distributed articles. In this paper, we explore a few sentence alignment methods which have been used before for another domain. We want to check whether Wikipedia can be used as one of the resources for collecting parallel corpora of Indonesian and Javanese, an ethnic language in Indonesia. We used two approaches of sentence alignment by treating Wikipedia as both parallel corpora and comparable corpora. In parallel corpora case, we used sentence length based and word correspondence methods. Meanwhile, we used the characteristics of hypertext links from Wikipedia in comparable corpora case. After the experiments, we can see that Wikipedia is useful enough for our purpose because both approaches gave positive results.\",\"PeriodicalId\":443250,\"journal\":{\"name\":\"2014 International Conference on Advanced Computer Science and Information System\",\"volume\":\"76 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2014-10-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"3\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2014 International Conference on Advanced Computer Science and Information System\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ICACSIS.2014.7065828\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2014 International Conference on Advanced Computer Science and Information System","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICACSIS.2014.7065828","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 3

摘要

平行语料库是多语言研究,特别是信息检索和自然语言处理研究的重要组成部分。然而,这样的语料库很难找到,特别是对于像民族语言这样资源匮乏的语言。民族语言平行语料库的采集多采用手工方式。另一方面,维基百科作为一个免费的在线百科全书,每年支持越来越多的语言,包括印度尼西亚的少数民族语言。它已成为万维网上最大的多语种网站之一,提供免费分发的文章。在本文中,我们探索了一些在另一个领域中使用过的句子对齐方法。我们想检查维基百科是否可以作为收集印度尼西亚民族语言印度尼西亚语和爪哇语平行语料库的资源之一。我们使用了两种句子对齐方法,将维基百科视为平行语料库和可比语料库。在平行语料库的情况下,我们使用了基于句子长度和单词对应的方法。同时,我们在可比语料库的情况下使用了维基百科超文本链接的特征。经过实验,我们可以看到Wikipedia对我们的目的足够有用,因为两种方法都给出了积极的结果。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Creating Indonesian-Javanese parallel corpora using wikipedia articles
Parallel corpora are necessary for multilingual researches especially in information retrieval (IR) and natural language processing (NLP). However, such corpora are hard to find, specifically for low-resources languages like ethnic languages. Parallel corpora of ethnic languages were usually collected manually. On the other hand, Wikipedia as a free online encyclopedia is supporting more and more languages each year, including ethnic languages in Indonesia. It has become one of the largest multilingual sites in World Wide Web that provides free distributed articles. In this paper, we explore a few sentence alignment methods which have been used before for another domain. We want to check whether Wikipedia can be used as one of the resources for collecting parallel corpora of Indonesian and Javanese, an ethnic language in Indonesia. We used two approaches of sentence alignment by treating Wikipedia as both parallel corpora and comparable corpora. In parallel corpora case, we used sentence length based and word correspondence methods. Meanwhile, we used the characteristics of hypertext links from Wikipedia in comparable corpora case. After the experiments, we can see that Wikipedia is useful enough for our purpose because both approaches gave positive results.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
Model prediction for accreditation of public junior high school in Bogor using spatial decision tree Campaign 2.0: Analysis of social media utilization in 2014 Jakarta legislative election Performance of robust two-dimensional principal component for classification Extending V-model practices to support SRE to build secure web application A comparison of backpropagation and LVQ: A case study of lung sound recognition
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1