首页 > 最新文献

2014 IEEE 5th International Conference on Software Engineering and Service Science最新文献

英文 中文
An improvement of embedded zerotree wavelet coding based on compressed sensing 基于压缩感知的嵌入式零树小波编码改进
Pub Date : 2014-06-27 DOI: 10.1109/ICSESS.2014.6933776
Z. Chen, Chenhao Mu, Fan Xu
Embedded zero-tree wavelet (EZW) is a kind of image compression algorithm which provides high compression rates fast. In order to improve the quality of reconstructed image after decoding, in this research, we put forward an improvement of embedded zerotree wavelet coding and simulate this new algorithm. This method takes full advantages of the Compressed Sensing (CS) theory and EZW. Simulation results demonstrated that the proposed method improved the quality of the unzipped image significantly.
嵌入式零树小波(EZW)是一种快速提供高压缩率的图像压缩算法。为了提高解码后重建图像的质量,本研究提出了一种改进的嵌入式零树小波编码算法,并对该算法进行了仿真。该方法充分利用了压缩感知(CS)理论和EZW技术。仿真结果表明,该方法显著提高了解压缩图像的质量。
{"title":"An improvement of embedded zerotree wavelet coding based on compressed sensing","authors":"Z. Chen, Chenhao Mu, Fan Xu","doi":"10.1109/ICSESS.2014.6933776","DOIUrl":"https://doi.org/10.1109/ICSESS.2014.6933776","url":null,"abstract":"Embedded zero-tree wavelet (EZW) is a kind of image compression algorithm which provides high compression rates fast. In order to improve the quality of reconstructed image after decoding, in this research, we put forward an improvement of embedded zerotree wavelet coding and simulate this new algorithm. This method takes full advantages of the Compressed Sensing (CS) theory and EZW. Simulation results demonstrated that the proposed method improved the quality of the unzipped image significantly.","PeriodicalId":6473,"journal":{"name":"2014 IEEE 5th International Conference on Software Engineering and Service Science","volume":"56 ","pages":"1177-1180"},"PeriodicalIF":0.0,"publicationDate":"2014-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91452463","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
The design and implementation of database audit system framework 数据库审计系统框架的设计与实现
Pub Date : 2014-06-27 DOI: 10.1109/ICSESS.2014.6933628
Kehe Wu, Li Hua, Xiaoxiang Wang, Xuewei Ding
For the information system, Database acts an important role in it. How to find a way to audit the operation of the database is becoming more and more important. An effective database auditing system can not only reduce the potential security risk, also make it possible to trace the source when errors happen. In this paper, we design and implement an effective audit framework. To avoid causing any database performance delays, bypass mode is especially useful for the auditing system. Compared with the traditional audit system, our approach has more advantages for example using zero-copy to acquire and reorganize data. In addition, we have a flexible policy to audit more effectively.
在信息系统中,数据库起着重要的作用。如何找到一种审计数据库运行的方法变得越来越重要。一个有效的数据库审计系统不仅可以降低潜在的安全风险,还可以在错误发生时跟踪来源。在本文中,我们设计并实施了一个有效的审计框架。为了避免造成任何数据库性能延迟,旁路模式对审计系统特别有用。与传统的审计系统相比,我们的方法具有更多的优势,例如使用零拷贝来获取和重组数据。此外,我们有一个灵活的政策,以更有效地审计。
{"title":"The design and implementation of database audit system framework","authors":"Kehe Wu, Li Hua, Xiaoxiang Wang, Xuewei Ding","doi":"10.1109/ICSESS.2014.6933628","DOIUrl":"https://doi.org/10.1109/ICSESS.2014.6933628","url":null,"abstract":"For the information system, Database acts an important role in it. How to find a way to audit the operation of the database is becoming more and more important. An effective database auditing system can not only reduce the potential security risk, also make it possible to trace the source when errors happen. In this paper, we design and implement an effective audit framework. To avoid causing any database performance delays, bypass mode is especially useful for the auditing system. Compared with the traditional audit system, our approach has more advantages for example using zero-copy to acquire and reorganize data. In addition, we have a flexible policy to audit more effectively.","PeriodicalId":6473,"journal":{"name":"2014 IEEE 5th International Conference on Software Engineering and Service Science","volume":"5 1","pages":"553-556"},"PeriodicalIF":0.0,"publicationDate":"2014-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87338475","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
Research on architecture of surface vessels decision-making model system based on distributed consultation decision 基于分布式咨询决策的水面舰艇决策模型系统体系结构研究
Pub Date : 2014-06-27 DOI: 10.1109/ICSESS.2014.6933732
Zhengwei Ma, Xu-Wen Sun, Mei Yang, Kan Li, F. Long
According to the fact that the decision-making of surface vessels is more and more complicated, based on the military process of decision-making of surface vessels, using soft engineering, from the view of system science, this article expounds the complexity of the command architecture and decision-making of surface vessels, then analyzes the process of consultation decision, at last, it establishes the architecture of shipboard decision-making model based on the distributed consultation decision,. It is very useful not only for furthering the theoretical system of shipboard decision-making, but also for the research on warfare software.
针对水面舰艇决策越来越复杂的现实,本文以水面舰艇决策的军事过程为基础,运用软件工程技术,从系统科学的角度,阐述了水面舰艇指挥体系结构与决策的复杂性,分析了水面舰艇决策的咨询过程。建立了基于分布式协商决策的舰载决策模型体系结构。这不仅对进一步完善舰载决策理论体系,而且对作战软件的研究具有重要意义。
{"title":"Research on architecture of surface vessels decision-making model system based on distributed consultation decision","authors":"Zhengwei Ma, Xu-Wen Sun, Mei Yang, Kan Li, F. Long","doi":"10.1109/ICSESS.2014.6933732","DOIUrl":"https://doi.org/10.1109/ICSESS.2014.6933732","url":null,"abstract":"According to the fact that the decision-making of surface vessels is more and more complicated, based on the military process of decision-making of surface vessels, using soft engineering, from the view of system science, this article expounds the complexity of the command architecture and decision-making of surface vessels, then analyzes the process of consultation decision, at last, it establishes the architecture of shipboard decision-making model based on the distributed consultation decision,. It is very useful not only for furthering the theoretical system of shipboard decision-making, but also for the research on warfare software.","PeriodicalId":6473,"journal":{"name":"2014 IEEE 5th International Conference on Software Engineering and Service Science","volume":"32 1","pages":"992-995"},"PeriodicalIF":0.0,"publicationDate":"2014-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91169841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A new testing method of randomness for true random sequences 一种新的真随机序列随机性检验方法
Pub Date : 2014-06-27 DOI: 10.1109/ICSESS.2014.6933624
Fang Yutao, Guiping Su
In this paper, based on researches and analysis for some common testing methods of randomness, a new testing method for randomness was proposed. This new method applied theories on kolmogorov complexity and wavelet transform to the randomness testing. This new method can be used to evaluate quality of random sequences generators and security of encryption algorithms.
本文在对几种常用的随机性检验方法进行研究和分析的基础上,提出了一种新的随机性检验方法。该方法将柯尔莫哥洛夫复杂度理论和小波变换理论应用到随机检验中。该方法可用于评价随机序列生成器的质量和加密算法的安全性。
{"title":"A new testing method of randomness for true random sequences","authors":"Fang Yutao, Guiping Su","doi":"10.1109/ICSESS.2014.6933624","DOIUrl":"https://doi.org/10.1109/ICSESS.2014.6933624","url":null,"abstract":"In this paper, based on researches and analysis for some common testing methods of randomness, a new testing method for randomness was proposed. This new method applied theories on kolmogorov complexity and wavelet transform to the randomness testing. This new method can be used to evaluate quality of random sequences generators and security of encryption algorithms.","PeriodicalId":6473,"journal":{"name":"2014 IEEE 5th International Conference on Software Engineering and Service Science","volume":"67 1","pages":"537-540"},"PeriodicalIF":0.0,"publicationDate":"2014-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73347774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
A multi-dimensional ontology-based IoT resource model 基于多维本体的物联网资源模型
Pub Date : 2014-06-27 DOI: 10.1109/ICSESS.2014.6933527
Huijuan Zhang, Chao Meng
Nowadays researches related to resource management and representation of IoT (Internet of Things) emphasize on making identify regulation or describing device working principle. However they could not represent dynamic feature and multi-dimensional feature of interaction. This paper propose a multi-dimensional ontology-based IoT resource model to solve above defects. For resource management, searching and reasoning, this model divides resource into 4 key concepts: general situation, variable feature, function and enforce. Then this paper classify resource into 5 dimensions. Experimental result shows that when IoT resource contains time feature, location feature and priority feature, resource model works well to accomplish management and searching task. In addition, this model lays a foundation for service model, which is in upper layer of resource model.
目前与资源管理和物联网表示相关的研究侧重于制定识别规则或描述设备工作原理。但它们不能代表交互的动态性和多维性。本文提出了一种基于多维本体的物联网资源模型来解决上述缺陷。在资源管理、搜索和推理方面,该模型将资源划分为4个关键概念:一般情况、变量特征、功能和强制。然后将资源划分为5个维度。实验结果表明,当物联网资源包含时间特征、位置特征和优先级特征时,资源模型可以很好地完成管理和搜索任务。此外,该模型还为资源模型上层的服务模型奠定了基础。
{"title":"A multi-dimensional ontology-based IoT resource model","authors":"Huijuan Zhang, Chao Meng","doi":"10.1109/ICSESS.2014.6933527","DOIUrl":"https://doi.org/10.1109/ICSESS.2014.6933527","url":null,"abstract":"Nowadays researches related to resource management and representation of IoT (Internet of Things) emphasize on making identify regulation or describing device working principle. However they could not represent dynamic feature and multi-dimensional feature of interaction. This paper propose a multi-dimensional ontology-based IoT resource model to solve above defects. For resource management, searching and reasoning, this model divides resource into 4 key concepts: general situation, variable feature, function and enforce. Then this paper classify resource into 5 dimensions. Experimental result shows that when IoT resource contains time feature, location feature and priority feature, resource model works well to accomplish management and searching task. In addition, this model lays a foundation for service model, which is in upper layer of resource model.","PeriodicalId":6473,"journal":{"name":"2014 IEEE 5th International Conference on Software Engineering and Service Science","volume":"27 1","pages":"124-127"},"PeriodicalIF":0.0,"publicationDate":"2014-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72683572","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 13
Adopting Text Clustering in web-based application to facilitate searching of education information 在基于web的应用中采用文本聚类,方便教育信息的检索
Pub Date : 2014-06-27 DOI: 10.1109/ICSESS.2014.6933590
N. Nilsson, Yan Liu
Clustering, as a part of the Data Mining field, has been in the center of the research attention for the last decade. It is the task of finding subsets of data that are sharing the same type of attributes. Text Clustering becomes one of the most critical and important solutions in data mining to discover knowledge from fast grow up web data and log files. There are many challenges, algorithms needs to be tailored specific for each domain and scale well with growing data sets. Another interesting aspect is the design of the system. A complex set components need to interact well together. This article proposes an elegant way of clustering university educations based on their text attributes. The solution is integrated directly into a Spring Web Application. A comprehensive architecture is proposed, providing the frameworks needed. Clustering techniques such as Canopy Generation [4] and k-Means are demonstrated.
聚类作为数据挖掘领域的一部分,在过去的十年中一直是研究的焦点。它的任务是查找共享相同类型属性的数据子集。文本聚类从快速增长的web数据和日志文件中发现知识,成为数据挖掘中最关键、最重要的解决方案之一。有许多挑战,算法需要为每个领域量身定制,并随着数据集的增长而良好扩展。另一个有趣的方面是系统的设计。一个复杂的组件集需要很好地相互作用。本文提出了一种基于文本属性对大学教育进行聚类的优雅方法。该解决方案直接集成到Spring Web应用程序中。提出了一个全面的体系结构,提供了所需的框架。本文展示了冠层生成[4]和k-Means等聚类技术。
{"title":"Adopting Text Clustering in web-based application to facilitate searching of education information","authors":"N. Nilsson, Yan Liu","doi":"10.1109/ICSESS.2014.6933590","DOIUrl":"https://doi.org/10.1109/ICSESS.2014.6933590","url":null,"abstract":"Clustering, as a part of the Data Mining field, has been in the center of the research attention for the last decade. It is the task of finding subsets of data that are sharing the same type of attributes. Text Clustering becomes one of the most critical and important solutions in data mining to discover knowledge from fast grow up web data and log files. There are many challenges, algorithms needs to be tailored specific for each domain and scale well with growing data sets. Another interesting aspect is the design of the system. A complex set components need to interact well together. This article proposes an elegant way of clustering university educations based on their text attributes. The solution is integrated directly into a Spring Web Application. A comprehensive architecture is proposed, providing the frameworks needed. Clustering techniques such as Canopy Generation [4] and k-Means are demonstrated.","PeriodicalId":6473,"journal":{"name":"2014 IEEE 5th International Conference on Software Engineering and Service Science","volume":"22 1","pages":"393-396"},"PeriodicalIF":0.0,"publicationDate":"2014-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72784145","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Research on optimization of flight scheduling problem based on the combination of ant colony optimization and genetic algorithm 基于蚁群优化与遗传算法结合的航班调度问题优化研究
Pub Date : 2014-06-27 DOI: 10.1109/ICSESS.2014.6933567
Wenkuai Liang, Yi Li
As the flight scheduling has always been a key technology which reduces flight delays and costs. Rational and efficient flight sorting method not only can effectively improve the utilization of the airport, reduce flight delay, but also ensure flight safety and reduce the incidence of sudden accidents. Firstly, we studied the application of ant colony algorithm in flight sorting, and proposed a new flight sorting method which is combined with ant colony algorithm and genetic algorithm depending on the mutation characteristic of genetic algorithm; Secondly, the minimization objective model that the flight total delay was minimum was established based on the sorting method; Finally, the simulation experiment was done based on the model. The simulation results show that the proposed method can effectively reduce flight delay. Comparing with ant colony algorithm, the global optimal flight sequence can be found in a shorter time.
由于航班调度一直是降低航班延误和成本的关键技术。合理高效的航班分拣方法不仅可以有效提高机场的利用率,减少航班延误,还可以保证飞行安全,减少突发事故的发生。首先,研究了蚁群算法在飞行排序中的应用,根据遗传算法的突变特性,提出了一种将蚁群算法与遗传算法相结合的飞行排序新方法;其次,基于排序法建立了航班总延误最小的最小化目标模型;最后,基于该模型进行了仿真实验。仿真结果表明,该方法可以有效地减少飞行延误。与蚁群算法相比,可以在较短的时间内找到全局最优的飞行序列。
{"title":"Research on optimization of flight scheduling problem based on the combination of ant colony optimization and genetic algorithm","authors":"Wenkuai Liang, Yi Li","doi":"10.1109/ICSESS.2014.6933567","DOIUrl":"https://doi.org/10.1109/ICSESS.2014.6933567","url":null,"abstract":"As the flight scheduling has always been a key technology which reduces flight delays and costs. Rational and efficient flight sorting method not only can effectively improve the utilization of the airport, reduce flight delay, but also ensure flight safety and reduce the incidence of sudden accidents. Firstly, we studied the application of ant colony algorithm in flight sorting, and proposed a new flight sorting method which is combined with ant colony algorithm and genetic algorithm depending on the mutation characteristic of genetic algorithm; Secondly, the minimization objective model that the flight total delay was minimum was established based on the sorting method; Finally, the simulation experiment was done based on the model. The simulation results show that the proposed method can effectively reduce flight delay. Comparing with ant colony algorithm, the global optimal flight sequence can be found in a shorter time.","PeriodicalId":6473,"journal":{"name":"2014 IEEE 5th International Conference on Software Engineering and Service Science","volume":"26 1","pages":"296-299"},"PeriodicalIF":0.0,"publicationDate":"2014-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77263039","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
SoRS: An effective SVD-DWT watermarking algorithm with SVD on the revised singular value 一种有效的基于修正奇异值的SVD- dwt水印算法
Pub Date : 2014-06-27 DOI: 10.1109/ICSESS.2014.6933734
Xiaoyi Zhou, Lingfei Wang
The proposed SoRS algorithm is based on the SoW scheme present in [1]. In SoW algorithm, the watermark is split into four shares, 2-level DWT is applied and SVD is performed on the HL/LH subbands. While in SoRS, all the subbands of 2-level DWT performed on the low subband are selected for embedding watermark. Furthermore, SVD is performed on the singular value revised by the watermark and the carrier image. The experimental results show that SoRS has both the significant improvement in perceptibility and the robustness under various types of image processing attacks. Even though this algorithm is weak against scaling attacks, it extracts watermarks of higher quality than other algorithms under many attacks such as JPEG compression, filtering, histogram equalization, rotation, cropping and noise.
本文提出的sor算法基于[1]中的SoW方案。在SoW算法中,水印被分割成4个份额,应用2级DWT,对HL/LH子带进行奇异值分解。而在sor中,选择在低子带上进行2级DWT的所有子带进行水印嵌入。然后,对水印和载体图像修正后的奇异值进行奇异值分解。实验结果表明,在各种类型的图像处理攻击下,sor在感知性和鲁棒性方面都有显著提高。尽管该算法对缩放攻击的抵抗能力较弱,但在JPEG压缩、滤波、直方图均衡化、旋转、裁剪和噪声等多种攻击下,它提取的水印质量比其他算法更高。
{"title":"SoRS: An effective SVD-DWT watermarking algorithm with SVD on the revised singular value","authors":"Xiaoyi Zhou, Lingfei Wang","doi":"10.1109/ICSESS.2014.6933734","DOIUrl":"https://doi.org/10.1109/ICSESS.2014.6933734","url":null,"abstract":"The proposed SoRS algorithm is based on the SoW scheme present in [1]. In SoW algorithm, the watermark is split into four shares, 2-level DWT is applied and SVD is performed on the HL/LH subbands. While in SoRS, all the subbands of 2-level DWT performed on the low subband are selected for embedding watermark. Furthermore, SVD is performed on the singular value revised by the watermark and the carrier image. The experimental results show that SoRS has both the significant improvement in perceptibility and the robustness under various types of image processing attacks. Even though this algorithm is weak against scaling attacks, it extracts watermarks of higher quality than other algorithms under many attacks such as JPEG compression, filtering, histogram equalization, rotation, cropping and noise.","PeriodicalId":6473,"journal":{"name":"2014 IEEE 5th International Conference on Software Engineering and Service Science","volume":"228 1","pages":"1001-1006"},"PeriodicalIF":0.0,"publicationDate":"2014-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77520289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Fault tolerant data flow using curator — Storm 使用管理员- Storm的容错数据流
Pub Date : 2014-06-27 DOI: 10.1109/ICSESS.2014.6933608
Lavanya Sainik, Dheeraj Khajuria
Driven by the 3GPP (3rd Generation Partnership Project) evolving standards and advent of Big Data technology, to deal with huge volume, velocity and variety of data, various industries like telecommunication, warehousing and storage, financial and many more industries need to be compliant with this evolving technology. There is a huge demand to process both real time and stored data. In this paper we have analyzed an open source framework Storm, which is a real time distributed processing engine and suggesting an improvement on its fault tolerance mechanism so that it can be flawlessly used for any data processing use case. Vanilla storm provides guaranteed message processing however it promises “at least once” level of processing. For perfect fault tolerant system “exactly one” level of processing is required and to achieve this storm provides another framework, Trident which is built on top of it. Trident provides transactional spout where transactional metadata information <; transaction id, data > is stored in zookeeper which provides distributed coordination, thus across node / hardware data can be replayed in case of any failure, timeout, retry. Trident uses zookeeper for coordination of transactional information through apache curator framework. However with current trident framework per activity level (aggregator/reducer) commit can be easily obtained but no direct implementation for single chain level transaction commit. This paper describes an approach where by modifying existing transactional trident, chain level commit can be obtained using curator recipes.
在3GPP(第三代合作伙伴计划)不断发展的标准和大数据技术的推动下,为了处理海量、快速和多样化的数据,电信、仓储、金融等各行各业都需要适应这种不断发展的技术。处理实时和存储数据的需求非常大。在本文中,我们分析了一个开源框架Storm,它是一个实时分布式处理引擎,并建议改进它的容错机制,以便它可以完美地用于任何数据处理用例。Vanilla storm提供了有保证的消息处理,但它承诺“至少一次”级别的处理。对于完美的容错系统,只需要“一个”级别的处理,而为了实现这一目标,风暴提供了另一个框架,即建立在它之上的三叉戟。Trident提供事务性spout,事务性元数据信息存储在zookeeper中,提供分布式协调,因此在任何故障、超时、重试的情况下,跨节点/硬件数据可以重播。Trident使用zookeeper通过apache策展人框架来协调事务信息。然而,在目前的trident框架中,每个活动级别(aggregator/reducer)提交可以很容易地获得,但没有直接实现单链级别的事务提交。本文描述了一种方法,通过修改现有的事务性三叉戟,可以使用管理员食谱获得链级提交。
{"title":"Fault tolerant data flow using curator — Storm","authors":"Lavanya Sainik, Dheeraj Khajuria","doi":"10.1109/ICSESS.2014.6933608","DOIUrl":"https://doi.org/10.1109/ICSESS.2014.6933608","url":null,"abstract":"Driven by the 3GPP (3rd Generation Partnership Project) evolving standards and advent of Big Data technology, to deal with huge volume, velocity and variety of data, various industries like telecommunication, warehousing and storage, financial and many more industries need to be compliant with this evolving technology. There is a huge demand to process both real time and stored data. In this paper we have analyzed an open source framework Storm, which is a real time distributed processing engine and suggesting an improvement on its fault tolerance mechanism so that it can be flawlessly used for any data processing use case. Vanilla storm provides guaranteed message processing however it promises “at least once” level of processing. For perfect fault tolerant system “exactly one” level of processing is required and to achieve this storm provides another framework, Trident which is built on top of it. Trident provides transactional spout where transactional metadata information <; transaction id, data > is stored in zookeeper which provides distributed coordination, thus across node / hardware data can be replayed in case of any failure, timeout, retry. Trident uses zookeeper for coordination of transactional information through apache curator framework. However with current trident framework per activity level (aggregator/reducer) commit can be easily obtained but no direct implementation for single chain level transaction commit. This paper describes an approach where by modifying existing transactional trident, chain level commit can be obtained using curator recipes.","PeriodicalId":6473,"journal":{"name":"2014 IEEE 5th International Conference on Software Engineering and Service Science","volume":"100 1","pages":"472-475"},"PeriodicalIF":0.0,"publicationDate":"2014-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75168195","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
On the quorum requirement and value selection rule for Fast Paxos Fast Paxos的仲裁要求和值选择规则研究
Pub Date : 2014-06-27 DOI: 10.1109/ICSESS.2014.6933593
Wenbing Zhao
Fault tolerance is a crucial requirement for big data and cloud computing because it helps remove single point of failures. The Paxos algorithm has been employed in a number of real-world fault tolerant systems. Fast Paxos is one of the latest algorithms in the Paxos family of algorithms for distributed consensus. In Fast Paxos, an acceptor is allowed to cast a vote for a value of its own choice unilaterally in a fast round, which eliminates a communication step for reaching consensus. As a tradeoff, a bigger quorum than simple majority must be used and a value selection rule is employed to select the right value for agreement to overcome collisions. The derivation of quorum requirements and value selection rule in the original article on Fast Paxos by Lamport, while very rigorous, may be hard to follow and difficult to implement. In this paper, we present a more intuitive approach to deriving the quorum requirements and easy-to-implement value selection rule for Fast Paxos.
容错是大数据和云计算的关键要求,因为它有助于消除单点故障。Paxos算法已经在许多现实世界的容错系统中得到了应用。Fast Paxos是Paxos分布式共识算法家族中最新的算法之一。在Fast Paxos中,允许接受者在快速轮中单方面投票选择自己选择的值,这消除了达成共识的沟通步骤。作为权衡,必须使用比简单多数更大的法定人数,并使用值选择规则来选择正确的值以达成一致,以克服冲突。Lamport在Fast Paxos的原文中对quorum需求和value selection rule的推导虽然非常严谨,但可能难以遵循和实现。在本文中,我们提出了一种更直观的方法来推导快速Paxos的仲裁需求和易于实现的值选择规则。
{"title":"On the quorum requirement and value selection rule for Fast Paxos","authors":"Wenbing Zhao","doi":"10.1109/ICSESS.2014.6933593","DOIUrl":"https://doi.org/10.1109/ICSESS.2014.6933593","url":null,"abstract":"Fault tolerance is a crucial requirement for big data and cloud computing because it helps remove single point of failures. The Paxos algorithm has been employed in a number of real-world fault tolerant systems. Fast Paxos is one of the latest algorithms in the Paxos family of algorithms for distributed consensus. In Fast Paxos, an acceptor is allowed to cast a vote for a value of its own choice unilaterally in a fast round, which eliminates a communication step for reaching consensus. As a tradeoff, a bigger quorum than simple majority must be used and a value selection rule is employed to select the right value for agreement to overcome collisions. The derivation of quorum requirements and value selection rule in the original article on Fast Paxos by Lamport, while very rigorous, may be hard to follow and difficult to implement. In this paper, we present a more intuitive approach to deriving the quorum requirements and easy-to-implement value selection rule for Fast Paxos.","PeriodicalId":6473,"journal":{"name":"2014 IEEE 5th International Conference on Software Engineering and Service Science","volume":"778 1","pages":"406-409"},"PeriodicalIF":0.0,"publicationDate":"2014-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76409385","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
2014 IEEE 5th International Conference on Software Engineering and Service Science
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1