收获书目多线程，安全和道德的网络爬行

2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS) Pub Date : 2018-10-01 DOI:10.1109/ICACSIS.2018.8618262

H. T. Y. Achsan, W. Wibowo, W. T. Putri, M. Achsan, Quintin Kurnia Dikara Barcah

{"title":"收获书目多线程，安全和道德的网络爬行","authors":"H. T. Y. Achsan, W. Wibowo, W. T. Putri, M. Achsan, Quintin Kurnia Dikara Barcah","doi":"10.1109/ICACSIS.2018.8618262","DOIUrl":null,"url":null,"abstract":"Web mining is an important techniques for it enables extraction of data and information from the web for further needs. Although there are a lot of web databases that holds bibliography data, The Online Computer Library Center (OCLC) owns the largest web databases on bibliography in the world. A large number of bibliography data, surely needs more than manual downloading. In this research we conduct an experimentation to harvest bibliography data using multi-threading process that is fast, safe and ethical. Using C# programming language and Visual Studio IDE, We were successful in harvesting five million bibliography data, without being penalized by the source site.","PeriodicalId":207227,"journal":{"name":"2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS)","volume":"114 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Harvesting Bibliography Multi-thread, Safe and Ethical Web Crawling\",\"authors\":\"H. T. Y. Achsan, W. Wibowo, W. T. Putri, M. Achsan, Quintin Kurnia Dikara Barcah\",\"doi\":\"10.1109/ICACSIS.2018.8618262\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Web mining is an important techniques for it enables extraction of data and information from the web for further needs. Although there are a lot of web databases that holds bibliography data, The Online Computer Library Center (OCLC) owns the largest web databases on bibliography in the world. A large number of bibliography data, surely needs more than manual downloading. In this research we conduct an experimentation to harvest bibliography data using multi-threading process that is fast, safe and ethical. Using C# programming language and Visual Studio IDE, We were successful in harvesting five million bibliography data, without being penalized by the source site.\",\"PeriodicalId\":207227,\"journal\":{\"name\":\"2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS)\",\"volume\":\"114 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2018-10-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ICACSIS.2018.8618262\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICACSIS.2018.8618262","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 1

摘要

网络挖掘是一项重要的技术，它可以从网络中提取数据和信息以供进一步需要。尽管保存书目数据的网络数据库很多，但在线计算机图书馆中心(OCLC)拥有世界上最大的书目数据库。大量的书目数据，肯定不需要人工下载。在本研究中，我们进行了一个实验，使用多线程的方法来获取书目数据，这是一种快速、安全、道德的方法。使用c#编程语言和Visual Studio IDE，我们成功地收集了500万个书目数据，而没有受到源站点的惩罚。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

Harvesting Bibliography Multi-thread, Safe and Ethical Web Crawling

Web mining is an important techniques for it enables extraction of data and information from the web for further needs. Although there are a lot of web databases that holds bibliography data, The Online Computer Library Center (OCLC) owns the largest web databases on bibliography in the world. A large number of bibliography data, surely needs more than manual downloading. In this research we conduct an experimentation to harvest bibliography data using multi-threading process that is fast, safe and ethical. Using C# programming language and Visual Studio IDE, We were successful in harvesting five million bibliography data, without being penalized by the source site.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS)

自引率

0.00%

发文量