Application Massive Data Processing Platform for Smart Manufacturing Based on Optimization of Data Storage

IF 2.5 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS ACM Transactions on Management Information Systems Pub Date : 2022-04-13 DOI:10.1145/3508395
Bin Ren, Yu-Qiang Chen, Fu-Cheng Wang
{"title":"Application Massive Data Processing Platform for Smart Manufacturing Based on Optimization of Data Storage","authors":"Bin Ren, Yu-Qiang Chen, Fu-Cheng Wang","doi":"10.1145/3508395","DOIUrl":null,"url":null,"abstract":"The aim of smart manufacturing is to reduce manpower requirements of the production line by applying technology of huge amounts of data to the manufacturing industry. Smart manufacturing is also called Industry 4.0, and the platform for processing huge amounts of data has an indispensable role. The massive data processing platform is like the brain of the entire factory, receiving all data from production line sensors via edge computing, processing, and analyzing, and finally making feedback decisions. With the innovation of production technology, the data that the platform needs to process has become diverse and complex, and the amount has become increasingly large. As well, many precision manufacturing industries have begun to enter the field of Industry 4.0. In addition to the accuracy and availability of data processing, there is emphasis on the real-time nature of data processing. After the sensor receives the data, the platform must provide feedback within a short period of time. This article proposes a massive data processing platform based on the Lambda architecture, which has the coexistence of stream processing and batch processing to meet real-time feedback needs of high-precision manufacturing. To verify the effectiveness of the optimization, it is based on real data from the manufacturing industry. To generate a large amount of test data to confirm the optimization of the storage of pictures. The results show that it optimizes the storage and optimization of the image data generated by the Automated Optical Inspection technology used in manufacturing today and optimizes the query for data storage. It also reduces the consumption of a large amount of memory as expected, and the query for Hive reduced the time spent.","PeriodicalId":45274,"journal":{"name":"ACM Transactions on Management Information Systems","volume":null,"pages":null},"PeriodicalIF":2.5000,"publicationDate":"2022-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ACM Transactions on Management Information Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3508395","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 1

Abstract

The aim of smart manufacturing is to reduce manpower requirements of the production line by applying technology of huge amounts of data to the manufacturing industry. Smart manufacturing is also called Industry 4.0, and the platform for processing huge amounts of data has an indispensable role. The massive data processing platform is like the brain of the entire factory, receiving all data from production line sensors via edge computing, processing, and analyzing, and finally making feedback decisions. With the innovation of production technology, the data that the platform needs to process has become diverse and complex, and the amount has become increasingly large. As well, many precision manufacturing industries have begun to enter the field of Industry 4.0. In addition to the accuracy and availability of data processing, there is emphasis on the real-time nature of data processing. After the sensor receives the data, the platform must provide feedback within a short period of time. This article proposes a massive data processing platform based on the Lambda architecture, which has the coexistence of stream processing and batch processing to meet real-time feedback needs of high-precision manufacturing. To verify the effectiveness of the optimization, it is based on real data from the manufacturing industry. To generate a large amount of test data to confirm the optimization of the storage of pictures. The results show that it optimizes the storage and optimization of the image data generated by the Automated Optical Inspection technology used in manufacturing today and optimizes the query for data storage. It also reduces the consumption of a large amount of memory as expected, and the query for Hive reduced the time spent.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
基于数据存储优化的智能制造海量数据处理平台应用
智能制造的目的是通过将大量数据技术应用于制造业来降低生产线的人力需求。智能制造也被称为工业4.0,处理大量数据的平台发挥着不可或缺的作用。海量数据处理平台就像整个工厂的大脑,通过边缘计算、处理和分析接收来自生产线传感器的所有数据,并最终做出反馈决策。随着生产技术的创新,平台需要处理的数据变得多样化和复杂,数量也越来越大。与此同时,许多精密制造业已经开始进入工业4.0领域。除了数据处理的准确性和可用性外,还强调数据处理的实时性。传感器接收到数据后,平台必须在短时间内提供反馈。本文提出了一个基于Lambda架构的海量数据处理平台,该平台具有流处理和批处理共存的特点,以满足高精度制造的实时反馈需求。为了验证优化的有效性,它是基于制造业的真实数据。生成大量的测试数据来确认图片的存储优化。结果表明,它优化了当今制造业使用的自动光学检测技术生成的图像数据的存储和优化,并优化了数据存储的查询。它还如预期的那样减少了大量内存的消耗,并且对Hive的查询减少了所花费的时间。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
ACM Transactions on Management Information Systems
ACM Transactions on Management Information Systems COMPUTER SCIENCE, INFORMATION SYSTEMS-
CiteScore
6.30
自引率
20.00%
发文量
60
期刊最新文献
Interpretable Predictive Models for Healthcare via Rational Multi-Layer Perceptrons Mining Multimorbidity Trajectories and Co-Medication Effects from Patient Data to Predict Post–Hip Fracture Outcomes ShennongMGS: An LLM-based Chinese Medication Guidance System Co-occurrence order-preserving pattern mining with keypoint alignment for time series Estimating Future Financial Development of Urban Areas for Deploying Bank Branches: A Local-Regional Interpretable Model
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1