H. T. Y. Achsan, W. Wibowo, W. T. Putri, M. Achsan, Quintin Kurnia Dikara Barcah
{"title":"Harvesting Bibliography Multi-thread, Safe and Ethical Web Crawling","authors":"H. T. Y. Achsan, W. Wibowo, W. T. Putri, M. Achsan, Quintin Kurnia Dikara Barcah","doi":"10.1109/ICACSIS.2018.8618262","DOIUrl":null,"url":null,"abstract":"Web mining is an important techniques for it enables extraction of data and information from the web for further needs. Although there are a lot of web databases that holds bibliography data, The Online Computer Library Center (OCLC) owns the largest web databases on bibliography in the world. A large number of bibliography data, surely needs more than manual downloading. In this research we conduct an experimentation to harvest bibliography data using multi-threading process that is fast, safe and ethical. Using C# programming language and Visual Studio IDE, We were successful in harvesting five million bibliography data, without being penalized by the source site.","PeriodicalId":207227,"journal":{"name":"2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS)","volume":"114 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICACSIS.2018.8618262","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1
Abstract
Web mining is an important techniques for it enables extraction of data and information from the web for further needs. Although there are a lot of web databases that holds bibliography data, The Online Computer Library Center (OCLC) owns the largest web databases on bibliography in the world. A large number of bibliography data, surely needs more than manual downloading. In this research we conduct an experimentation to harvest bibliography data using multi-threading process that is fast, safe and ethical. Using C# programming language and Visual Studio IDE, We were successful in harvesting five million bibliography data, without being penalized by the source site.
网络挖掘是一项重要的技术，它可以从网络中提取数据和信息以供进一步需要。尽管保存书目数据的网络数据库很多，但在线计算机图书馆中心(OCLC)拥有世界上最大的书目数据库。大量的书目数据，肯定不需要人工下载。在本研究中，我们进行了一个实验，使用多线程的方法来获取书目数据，这是一种快速、安全、道德的方法。使用c#编程语言和Visual Studio IDE，我们成功地收集了500万个书目数据，而没有受到源站点的惩罚。
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
