{"title":"Research of Target Characteristics Storage Based on RDBMS and Hadoop","authors":"Yanqi Wang, Yusheng Jia, Xiaodan Xie","doi":"10.1109/IIKI.2016.33","DOIUrl":null,"url":null,"abstract":"As the amount of target characteristics data increasing rapidly, the tradition methods cannot satisfy the need of the storage and management of those data. According to the features of those data, a new storage system is proposed base on RDBMS and Hadoop. The structured data and the metadata of unstructured data is stored in the RDBMS under certain schema, while the large amount of unstructured one allocated among numbers of nodes in the hadoop cluster. In order to maximize the superiority of storage, the HBase is used for storing massive small-size unstructured data and the HDFS is applied for holding the large-scale ones. Meanwhile, the access control and the multi-thread upload and download approach combined with load balancing and caching mechanism is applied for improving the efficiency of data transmission. Experiment results show that the proposed storage system is reasonable and practicable.","PeriodicalId":371106,"journal":{"name":"2016 International Conference on Identification, Information and Knowledge in the Internet of Things (IIKI)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2016-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2016 International Conference on Identification, Information and Knowledge in the Internet of Things (IIKI)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/IIKI.2016.33","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1
Abstract
As the amount of target characteristics data increasing rapidly, the tradition methods cannot satisfy the need of the storage and management of those data. According to the features of those data, a new storage system is proposed base on RDBMS and Hadoop. The structured data and the metadata of unstructured data is stored in the RDBMS under certain schema, while the large amount of unstructured one allocated among numbers of nodes in the hadoop cluster. In order to maximize the superiority of storage, the HBase is used for storing massive small-size unstructured data and the HDFS is applied for holding the large-scale ones. Meanwhile, the access control and the multi-thread upload and download approach combined with load balancing and caching mechanism is applied for improving the efficiency of data transmission. Experiment results show that the proposed storage system is reasonable and practicable.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
