首页 > 最新文献

2020 16th International Conference on Computational Intelligence and Security (CIS)最新文献

英文 中文
Binary Image Geometric Calculation Algorithm Based on SNAM Representation 基于SNAM表示的二值图像几何计算算法
Pub Date : 2020-11-01 DOI: 10.1109/CIS52066.2020.00012
Caixu Xu, Guo Hui, He Jie
Aiming at problems such as difficulties in determini--ng sub-mode position relations and complexity in image calculations caused by asymmetric segmentation in SNAM image representation, the present study first recovers submodule spatial position relations of SNAM representation with grid arrays and improves the grid arrays satisfying geometric calculation of images. Then, it proposes a geometric calculation algorithm based on SNAM. According to experimental results compared to geometric calculations represented by linear quad-tree and arrays or the like, the proposed algorithm has obvious higher efficiency. It is an eight-neighborhood image geometric calculation algorithm with excellent performance and adaptability to SNAM representation.
针对SNAM图像表示中由于分割不对称导致的子模位置关系难以确定、图像计算复杂等问题,本研究首先利用网格阵列恢复SNAM表示的子模块空间位置关系,并对满足图像几何计算的网格阵列进行改进。然后,提出了一种基于SNAM的几何计算算法。实验结果表明,与线性四叉树和数组等几何计算方式相比,本文算法的效率明显提高。它是一种八邻域图像几何计算算法,具有优异的性能和对SNAM表示的适应性。
{"title":"Binary Image Geometric Calculation Algorithm Based on SNAM Representation","authors":"Caixu Xu, Guo Hui, He Jie","doi":"10.1109/CIS52066.2020.00012","DOIUrl":"https://doi.org/10.1109/CIS52066.2020.00012","url":null,"abstract":"Aiming at problems such as difficulties in determini--ng sub-mode position relations and complexity in image calculations caused by asymmetric segmentation in SNAM image representation, the present study first recovers submodule spatial position relations of SNAM representation with grid arrays and improves the grid arrays satisfying geometric calculation of images. Then, it proposes a geometric calculation algorithm based on SNAM. According to experimental results compared to geometric calculations represented by linear quad-tree and arrays or the like, the proposed algorithm has obvious higher efficiency. It is an eight-neighborhood image geometric calculation algorithm with excellent performance and adaptability to SNAM representation.","PeriodicalId":106959,"journal":{"name":"2020 16th International Conference on Computational Intelligence and Security (CIS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127267810","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Prediction of Air Quality in Major Cities of China by Deep Learning 基于深度学习的中国主要城市空气质量预测
Pub Date : 2020-11-01 DOI: 10.1109/CIS52066.2020.00023
Choujun Zhan, Songyan Li, Jianbin Li, Yijing Guo, Quansi Wen, WeiSheng Wen
With global industrialization, air pollution is becoming a critical issue that threatens human health. The World Health Organization (WHO) estimated that air pollution kills several million people worldwide each year. Researchers from various areas and governments and enterprises have invested many resources in investigating and reducing air pollution. Air Quality Index (AQI) is one of the essential indexes indicating air quality or the level of air pollution. A new dataset, including hourly AQI information recorded by 1,615 observation sites covering China from 2015 to 2019, is constructed. Several methods, including linear model and state-of-art techniques, such as Back Propagation Neural Network (BPNN), Convolutional Neural Networks (CNN), Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), and Bi-directional Long Short-Term Memory (BiLSTM), are adopted to forecast hourly AQI. The performance of these techniques is evaluated, and experiments show that the BiLSTM gives the best performance.
随着全球工业化的发展,空气污染已成为威胁人类健康的重要问题。世界卫生组织(WHO)估计,全球每年有数百万人死于空气污染。各个领域的研究人员以及政府和企业投入了大量资源来调查和减少空气污染。空气质量指数(AQI)是反映空气质量或空气污染程度的重要指标之一。构建了一个新的数据集,包括2015 - 2019年中国1615个观测点记录的每小时AQI信息。采用线性模型和最先进的技术,如反向传播神经网络(BPNN)、卷积神经网络(CNN)、门控循环单元(GRU)、长短期记忆(LSTM)和双向长短期记忆(BiLSTM)来预测每小时的AQI。实验结果表明,BiLSTM的性能最好。
{"title":"Prediction of Air Quality in Major Cities of China by Deep Learning","authors":"Choujun Zhan, Songyan Li, Jianbin Li, Yijing Guo, Quansi Wen, WeiSheng Wen","doi":"10.1109/CIS52066.2020.00023","DOIUrl":"https://doi.org/10.1109/CIS52066.2020.00023","url":null,"abstract":"With global industrialization, air pollution is becoming a critical issue that threatens human health. The World Health Organization (WHO) estimated that air pollution kills several million people worldwide each year. Researchers from various areas and governments and enterprises have invested many resources in investigating and reducing air pollution. Air Quality Index (AQI) is one of the essential indexes indicating air quality or the level of air pollution. A new dataset, including hourly AQI information recorded by 1,615 observation sites covering China from 2015 to 2019, is constructed. Several methods, including linear model and state-of-art techniques, such as Back Propagation Neural Network (BPNN), Convolutional Neural Networks (CNN), Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), and Bi-directional Long Short-Term Memory (BiLSTM), are adopted to forecast hourly AQI. The performance of these techniques is evaluated, and experiments show that the BiLSTM gives the best performance.","PeriodicalId":106959,"journal":{"name":"2020 16th International Conference on Computational Intelligence and Security (CIS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124932973","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
[Copyright notice] (版权)
Pub Date : 2020-11-01 DOI: 10.1109/cis52066.2020.00003
{"title":"[Copyright notice]","authors":"","doi":"10.1109/cis52066.2020.00003","DOIUrl":"https://doi.org/10.1109/cis52066.2020.00003","url":null,"abstract":"","PeriodicalId":106959,"journal":{"name":"2020 16th International Conference on Computational Intelligence and Security (CIS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114457885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Chinese Sentiment Classification Model of Neural Network Based on Particle Swarm Optimization 基于粒子群优化的神经网络中文情感分类模型
Pub Date : 2020-11-01 DOI: 10.1109/CIS52066.2020.00078
Yaling Zhang, Jiale Li, Shibo Bai
Due to the differences in features between different languages, Chinese text is more complicated and difficult in natural language processing tasks than English text. This paper proposes a neural network Chinese sentiment classification model based on particle swarm optimization (PSO-Attention-LSTM), the model uses the Long Short Term Memory Network superimposed attention mechanism to extract information from Chinese review data and determine the sentiment polarity of the sentence; aiming at the problem that parameters such as the number of hidden layer neurons in the LSTM unit and the number of batches of the neural network are difficult to determine, the global optimization capability of the particle swarm optimization (PSO) is used to optimize the parameters. The experimental results show that the neural network Chinese sentiment classification model based on particle swarm optimization has improved the accuracy of the hotel data set by nearly 6 percentage points.
由于不同语言之间的特征差异,汉语文本在自然语言处理任务中比英语文本更为复杂和困难。本文提出了一种基于粒子群优化(PSO-Attention-LSTM)的神经网络中文情感分类模型,该模型利用长短期记忆网络叠加注意机制从中文评论数据中提取信息,确定句子的情感极性;针对LSTM单元中隐含层神经元个数和神经网络批次数等参数难以确定的问题,利用粒子群算法(PSO)的全局优化能力对参数进行优化。实验结果表明,基于粒子群优化的神经网络中文情感分类模型将酒店数据集的准确率提高了近6个百分点。
{"title":"Chinese Sentiment Classification Model of Neural Network Based on Particle Swarm Optimization","authors":"Yaling Zhang, Jiale Li, Shibo Bai","doi":"10.1109/CIS52066.2020.00078","DOIUrl":"https://doi.org/10.1109/CIS52066.2020.00078","url":null,"abstract":"Due to the differences in features between different languages, Chinese text is more complicated and difficult in natural language processing tasks than English text. This paper proposes a neural network Chinese sentiment classification model based on particle swarm optimization (PSO-Attention-LSTM), the model uses the Long Short Term Memory Network superimposed attention mechanism to extract information from Chinese review data and determine the sentiment polarity of the sentence; aiming at the problem that parameters such as the number of hidden layer neurons in the LSTM unit and the number of batches of the neural network are difficult to determine, the global optimization capability of the particle swarm optimization (PSO) is used to optimize the parameters. The experimental results show that the neural network Chinese sentiment classification model based on particle swarm optimization has improved the accuracy of the hotel data set by nearly 6 percentage points.","PeriodicalId":106959,"journal":{"name":"2020 16th International Conference on Computational Intelligence and Security (CIS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129055061","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Electro-optic Combination Coherent Communication System Based On OCDMA Theory 基于OCDMA理论的电光组合相干通信系统
Pub Date : 2020-11-01 DOI: 10.1109/CIS52066.2020.00022
Peng Zhou, Ye Lu, Dong Cheng, Chuanqi Li
With the explosive growth of data and the continuous expansion of the network scale, the spectrum resource problem in the network is becoming more and more serious, which hinders the development of the Internet. Aiming at the problem that the spectrum utilization of the optical code division multiple access (OCDMA) system, we give a possible strategy for communication scheme, analyze them, and derive expressions for coding and decoding states. We design an experimental system with the data-rate of 5Gbit/s, the distance of 620km. The results indicate the high recovery accuracy of the user data. Furthermore, we compare coherent and incoherent systems in terms of transmission distance and bit error rate (BER). The simulation results show that the scheme we proposed can transmit longer distances and the BER is lower than incoherent systems.
随着数据的爆炸式增长和网络规模的不断扩大,网络中的频谱资源问题越来越严重,阻碍了互联网的发展。针对光码分多址(OCDMA)系统的频谱利用问题,给出了一种可能的通信方案策略,并对其进行了分析,推导出了编码和解码状态的表达式。我们设计了一个数据速率为5Gbit/s,传输距离为620km的实验系统。结果表明,该方法具有较高的用户数据恢复精度。此外,我们在传输距离和误码率(BER)方面比较了相干和非相干系统。仿真结果表明,该方案比非相干系统传输距离更远,误码率更低。
{"title":"Electro-optic Combination Coherent Communication System Based On OCDMA Theory","authors":"Peng Zhou, Ye Lu, Dong Cheng, Chuanqi Li","doi":"10.1109/CIS52066.2020.00022","DOIUrl":"https://doi.org/10.1109/CIS52066.2020.00022","url":null,"abstract":"With the explosive growth of data and the continuous expansion of the network scale, the spectrum resource problem in the network is becoming more and more serious, which hinders the development of the Internet. Aiming at the problem that the spectrum utilization of the optical code division multiple access (OCDMA) system, we give a possible strategy for communication scheme, analyze them, and derive expressions for coding and decoding states. We design an experimental system with the data-rate of 5Gbit/s, the distance of 620km. The results indicate the high recovery accuracy of the user data. Furthermore, we compare coherent and incoherent systems in terms of transmission distance and bit error rate (BER). The simulation results show that the scheme we proposed can transmit longer distances and the BER is lower than incoherent systems.","PeriodicalId":106959,"journal":{"name":"2020 16th International Conference on Computational Intelligence and Security (CIS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124515614","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Order of Servers for Periodic Multi-Installment Scheduling 周期性多分期调度的服务器顺序
Pub Date : 2020-11-01 DOI: 10.1109/CIS52066.2020.00065
Hui Wang, Xiaobo Song, Zhihao Tong, Xiaoli Wang
Periodic multi-installment scheduling (PMIS) has been the most effective model for large-scale divisible-load scheduling on distributed systems. In practice, the decreasing order of communication speeds, denoted as IZ, has always been used as the scheduling sequence of servers because it has been proven that IZ is the optimal sequence to achieve minimum makespan for single-installment scheduling and studies available have shown that IZ is the near-optimal sequence for multi-installment scheduling. In this paper, however, we illustrate by an example that IZ unfortunately causes time conflicts for servers between the last installment but one and the last installment, thus it is definitely not a feasible sequence for PMIS, not to mention an optimal or near-optimal sequence. Further, to obtain a feasible order of servers, we provide rigorous proof in this paper that there is no time conflict when servers follow the increasing order of communication speeds.
周期多分期调度(PMIS)是分布式系统大规模可分负荷调度中最有效的模型。在实践中,通信速度的降序表示为IZ,一直被用作服务器的调度序列,因为已经证明IZ是单台调度中实现最小makespan的最优序列,已有研究表明IZ是多台调度的近最优序列。然而,在本文中,我们通过一个示例来说明,IZ不幸导致服务器在第一期和最后一期之间发生时间冲突,因此它绝对不是PMIS的可行序列,更不用说最优或接近最优序列了。进一步,为了获得一个可行的服务器顺序,我们提供了严格的证明,当服务器遵循通信速度的递增顺序时,不存在时间冲突。
{"title":"Order of Servers for Periodic Multi-Installment Scheduling","authors":"Hui Wang, Xiaobo Song, Zhihao Tong, Xiaoli Wang","doi":"10.1109/CIS52066.2020.00065","DOIUrl":"https://doi.org/10.1109/CIS52066.2020.00065","url":null,"abstract":"Periodic multi-installment scheduling (PMIS) has been the most effective model for large-scale divisible-load scheduling on distributed systems. In practice, the decreasing order of communication speeds, denoted as IZ, has always been used as the scheduling sequence of servers because it has been proven that IZ is the optimal sequence to achieve minimum makespan for single-installment scheduling and studies available have shown that IZ is the near-optimal sequence for multi-installment scheduling. In this paper, however, we illustrate by an example that IZ unfortunately causes time conflicts for servers between the last installment but one and the last installment, thus it is definitely not a feasible sequence for PMIS, not to mention an optimal or near-optimal sequence. Further, to obtain a feasible order of servers, we provide rigorous proof in this paper that there is no time conflict when servers follow the increasing order of communication speeds.","PeriodicalId":106959,"journal":{"name":"2020 16th International Conference on Computational Intelligence and Security (CIS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126436267","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Brief Review of Two Classical Models for Asset Allocating 两种经典资产配置模型述评
Pub Date : 2020-11-01 DOI: 10.1109/CIS52066.2020.00042
Ya-juan Yang, Liang Zhang, Yi Niu, Ouan-Ju Zhang
The two most famous models, one is called mean-variance optimization model (in short MVO) which was proposed by Markowiz who won the Nobel prize in 1990 due to his pioneering research in the theory of modern financial economics, and another is named after B-L model proposed by Black-Litterman. This paper introduces the evolution of these two models in asset allocating: MVO model and B-L model. First, the advantages and disadvantages of the two models are described in case of treating a practical investment strategy by the two models being employed. Second, we illustrate that, with a comparison of the mean-variance optimization model, the key ingredients are accurate judgment on the performance and correlation of each asset for Black-Litterman model. Finally, we point out that the Black-Litterman model is not always superior to mean-variance optimization in case of the experiences of an investor are insufficient.
最著名的两种模型,一种是均值方差优化模型(mean-variance optimization model,简称MVO),由1990年因在现代金融经济学理论方面的开创性研究而获得诺贝尔奖的马科维茨提出,另一种是以Black-Litterman提出的B-L模型命名的。本文介绍了MVO模型和B-L模型这两种资产配置模型的演变过程。首先,在使用两种模型处理实际投资策略的情况下,描述了两种模型的优缺点。其次,通过与均值-方差优化模型的比较,说明Black-Litterman模型对各资产的表现和相关性的准确判断是关键。最后,我们指出,在投资者经验不足的情况下,Black-Litterman模型并不总是优于均值方差优化。
{"title":"A Brief Review of Two Classical Models for Asset Allocating","authors":"Ya-juan Yang, Liang Zhang, Yi Niu, Ouan-Ju Zhang","doi":"10.1109/CIS52066.2020.00042","DOIUrl":"https://doi.org/10.1109/CIS52066.2020.00042","url":null,"abstract":"The two most famous models, one is called mean-variance optimization model (in short MVO) which was proposed by Markowiz who won the Nobel prize in 1990 due to his pioneering research in the theory of modern financial economics, and another is named after B-L model proposed by Black-Litterman. This paper introduces the evolution of these two models in asset allocating: MVO model and B-L model. First, the advantages and disadvantages of the two models are described in case of treating a practical investment strategy by the two models being employed. Second, we illustrate that, with a comparison of the mean-variance optimization model, the key ingredients are accurate judgment on the performance and correlation of each asset for Black-Litterman model. Finally, we point out that the Black-Litterman model is not always superior to mean-variance optimization in case of the experiences of an investor are insufficient.","PeriodicalId":106959,"journal":{"name":"2020 16th International Conference on Computational Intelligence and Security (CIS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126512493","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Emotional Analysis on the Public Sentiment of Students Returning to University under COVID-19 新冠肺炎疫情下大学生返校公众情绪的情感分析
Pub Date : 2020-11-01 DOI: 10.1109/CIS52066.2020.00058
Liping Jia, Zhonghua Li
As an enormous epidemic for the global, COVID-19, has become a major threat to the mankind thus influence economy, living activities, especial eduction. The public sentiment has been produced in the internet. In this paper, public emotional sentiment about returning to the university is crawled from the weibo and studied. By analyzing the reviews, the students' emotional analysis is studied by snowNLP-based method and TF-IDF-based method. Numerical experiments and visualization results indicate that the students have positive emotion for returning to the university.
作为一场全球性的巨大流行病,新冠肺炎已成为对人类的重大威胁,影响着经济、生活活动,特别是教育。公众情绪是在互联网上产生的。本文从微博中抓取公众对于返校的情绪并进行研究。通过对综述的分析,采用基于snownlp的方法和基于tf - idf的方法对学生的情绪分析进行研究。数值实验和可视化结果表明,大学生有积极的返校情绪。
{"title":"Emotional Analysis on the Public Sentiment of Students Returning to University under COVID-19","authors":"Liping Jia, Zhonghua Li","doi":"10.1109/CIS52066.2020.00058","DOIUrl":"https://doi.org/10.1109/CIS52066.2020.00058","url":null,"abstract":"As an enormous epidemic for the global, COVID-19, has become a major threat to the mankind thus influence economy, living activities, especial eduction. The public sentiment has been produced in the internet. In this paper, public emotional sentiment about returning to the university is crawled from the weibo and studied. By analyzing the reviews, the students' emotional analysis is studied by snowNLP-based method and TF-IDF-based method. Numerical experiments and visualization results indicate that the students have positive emotion for returning to the university.","PeriodicalId":106959,"journal":{"name":"2020 16th International Conference on Computational Intelligence and Security (CIS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133612621","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Performance Analysis of Scheduling Algorithms in Apache Hadoop Apache Hadoop调度算法的性能分析
Pub Date : 2020-11-01 DOI: 10.1109/CIS52066.2020.00040
Yang Li
Hadoop bundles the two computing resources of memory and CPU in the management resources, and then divides it into two resource models: MapSlot and ReduceSlot according to task types. MapReduce applications will have a large number of sorting operations in operation. Most of these sorts are executed iteratively, which consumes a lot of performance. Chapter 5 of this article takes this as an entry point and reorganizes the execution process of the Shuffle stage. Researched to replace quick sort with more efficient counting sorting. At the same time, the Shuffle execution is branched according to the definition of Combiner. One branch deletes the quick sort in the partition in the spill phase and the merge sort in the combine phase to reduce performance consumption. The other branch executes Combiner in advance to improve data processing efficiency. The two branches processed 21GB of log data on a 7-node PC cluster, and both achieved an efficiency improvement of about half an hour.
Hadoop将内存和CPU这两种计算资源捆绑在管理资源中,根据任务类型划分为MapSlot和ReduceSlot两种资源模型。MapReduce应用会有大量的排序操作在运行。这些类型中的大多数都是迭代执行的,这会消耗很多性能。本文第5章以此为切入点,重新组织Shuffle阶段的执行过程。研究用更有效的计数排序取代快速排序。同时,Shuffle的执行根据Combiner的定义进行了分支。一个分支在溢出阶段删除分区中的快速排序,在合并阶段删除合并排序,以减少性能消耗。另一个分支提前执行Combiner,提高数据处理效率。两个分支在一个7节点的PC集群上处理了21GB的日志数据,都实现了大约半小时的效率提升。
{"title":"Performance Analysis of Scheduling Algorithms in Apache Hadoop","authors":"Yang Li","doi":"10.1109/CIS52066.2020.00040","DOIUrl":"https://doi.org/10.1109/CIS52066.2020.00040","url":null,"abstract":"Hadoop bundles the two computing resources of memory and CPU in the management resources, and then divides it into two resource models: MapSlot and ReduceSlot according to task types. MapReduce applications will have a large number of sorting operations in operation. Most of these sorts are executed iteratively, which consumes a lot of performance. Chapter 5 of this article takes this as an entry point and reorganizes the execution process of the Shuffle stage. Researched to replace quick sort with more efficient counting sorting. At the same time, the Shuffle execution is branched according to the definition of Combiner. One branch deletes the quick sort in the partition in the spill phase and the merge sort in the combine phase to reduce performance consumption. The other branch executes Combiner in advance to improve data processing efficiency. The two branches processed 21GB of log data on a 7-node PC cluster, and both achieved an efficiency improvement of about half an hour.","PeriodicalId":106959,"journal":{"name":"2020 16th International Conference on Computational Intelligence and Security (CIS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132173907","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Research and Application of Multi-Round Dialogue Intent Recognition Method 多轮对话意图识别方法的研究与应用
Pub Date : 2020-11-01 DOI: 10.1109/CIS52066.2020.00036
Jie Song, Qifeng Luo, J. Nie
In the existing dialogue system, there are numerous sentences in non-standardized verbal expression form, which usually is brief and vague. It is a challenging task to identify the intentions through the analysis of these sentences. Considering that the supervised learning approach is the mainstream on multi-intention recognition, an amount of public labeled multi-intention dialogue data is necessary. However, labeling work is costly and time-consuming. In this paper, we put forward a multi-label classification method based on existing mainstream classification algorithms and used for dialogue-level multi-intention recognition to reduce the cost of labeling work. We publish the Chinese Multi-Intention Dialogue (CMID-Transportation) dataset of transportation customer service, which is collected by us in an actual production project. We conduct a series of experiments on the CMID-Transportation corpus by using the mainstream classification algorithms and then produce the basic benchmark performance. We find that BERT achieves the best results. We hope that the CMID-Transportation dataset can promote the research and development of intent recognition tasks in multiple rounds of dialogue.
在现有的对话系统中,有大量的非规范的言语表达形式的句子,这些句子通常是简短而模糊的。通过分析这些句子来识别意图是一项具有挑战性的任务。考虑到监督学习方法是多意图识别的主流,需要大量公开标注的多意图对话数据。然而,标签工作既昂贵又耗时。本文在现有主流分类算法的基础上,提出了一种多标签分类方法,并将其用于对话级多意图识别,以降低标注工作的成本。我们发布了运输客户服务中文多意向对话(CMID-Transportation)数据集,该数据集是我们在实际生产项目中收集的。我们使用主流分类算法在CMID-Transportation语料库上进行了一系列的实验,然后得出了基本的基准性能。我们发现BERT达到了最好的效果。我们希望CMID-Transportation数据集能够在多轮对话中促进意图识别任务的研究和开发。
{"title":"Research and Application of Multi-Round Dialogue Intent Recognition Method","authors":"Jie Song, Qifeng Luo, J. Nie","doi":"10.1109/CIS52066.2020.00036","DOIUrl":"https://doi.org/10.1109/CIS52066.2020.00036","url":null,"abstract":"In the existing dialogue system, there are numerous sentences in non-standardized verbal expression form, which usually is brief and vague. It is a challenging task to identify the intentions through the analysis of these sentences. Considering that the supervised learning approach is the mainstream on multi-intention recognition, an amount of public labeled multi-intention dialogue data is necessary. However, labeling work is costly and time-consuming. In this paper, we put forward a multi-label classification method based on existing mainstream classification algorithms and used for dialogue-level multi-intention recognition to reduce the cost of labeling work. We publish the Chinese Multi-Intention Dialogue (CMID-Transportation) dataset of transportation customer service, which is collected by us in an actual production project. We conduct a series of experiments on the CMID-Transportation corpus by using the mainstream classification algorithms and then produce the basic benchmark performance. We find that BERT achieves the best results. We hope that the CMID-Transportation dataset can promote the research and development of intent recognition tasks in multiple rounds of dialogue.","PeriodicalId":106959,"journal":{"name":"2020 16th International Conference on Computational Intelligence and Security (CIS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124435429","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
2020 16th International Conference on Computational Intelligence and Security (CIS)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1