首页 > 最新文献

2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)最新文献

英文 中文
Client side calculation of ‘bacon number’ 客户端计算“培根数”
Thomas Briggs, Aspen Olmsted
This paper presents an architecture for answering the popular ‘bacon number’ game utilizing the IMDB dataset. The unique contribution to this well-known game is this paper presents implementing a breadth traversal algorithm and rendering of the data completely in the client. This paper outlines the architecture and presents evidence of comparable speed in the complete clientside rendering of the computation of large data objects as compared to traditional server-side compiled languages.
本文提出了一种利用IMDB数据集回答流行的“培根数”游戏的架构。对这个著名游戏的独特贡献是本文提出了一种宽度遍历算法并在客户端完全渲染数据。本文概述了该体系结构,并提供了与传统的服务器端编译语言相比,在完整的大型数据对象计算的客户端呈现中具有相当速度的证据。
{"title":"Client side calculation of ‘bacon number’","authors":"Thomas Briggs, Aspen Olmsted","doi":"10.23919/ICITST.2017.8356418","DOIUrl":"https://doi.org/10.23919/ICITST.2017.8356418","url":null,"abstract":"This paper presents an architecture for answering the popular ‘bacon number’ game utilizing the IMDB dataset. The unique contribution to this well-known game is this paper presents implementing a breadth traversal algorithm and rendering of the data completely in the client. This paper outlines the architecture and presents evidence of comparable speed in the complete clientside rendering of the computation of large data objects as compared to traditional server-side compiled languages.","PeriodicalId":440665,"journal":{"name":"2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130801947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Improve CRUD performance on hierarchical data: Nested interval model vs. nested set model 提高分层数据上的CRUD性能:嵌套区间模型与嵌套集模型
Blake Badders, Aspen Olmsted
Hierarchical data models are commonly used in databases to represent data across a multitude of disciplines. In this paper, I will examine two models that can be used to represent hierarchical data in a relational database, the nested interval model, and the nested set model. These models were analyzed on a WAMP server using various queries and CRUD operations. The purpose of this paper is to examine the advantages of using a nested interval model as compared to a nested set model in a database which needs to search quickly, create, update, and destroy large amounts of hierarchical data.
在数据库中,层次数据模型通常用于表示跨多个学科的数据。在本文中,我将研究两种可用于表示关系数据库中的分层数据的模型,即嵌套区间模型和嵌套集模型。在WAMP服务器上使用各种查询和CRUD操作分析这些模型。本文的目的是研究在需要快速搜索、创建、更新和销毁大量分层数据的数据库中,与使用嵌套集模型相比,使用嵌套区间模型的优势。
{"title":"Improve CRUD performance on hierarchical data: Nested interval model vs. nested set model","authors":"Blake Badders, Aspen Olmsted","doi":"10.23919/ICITST.2017.8356423","DOIUrl":"https://doi.org/10.23919/ICITST.2017.8356423","url":null,"abstract":"Hierarchical data models are commonly used in databases to represent data across a multitude of disciplines. In this paper, I will examine two models that can be used to represent hierarchical data in a relational database, the nested interval model, and the nested set model. These models were analyzed on a WAMP server using various queries and CRUD operations. The purpose of this paper is to examine the advantages of using a nested interval model as compared to a nested set model in a database which needs to search quickly, create, update, and destroy large amounts of hierarchical data.","PeriodicalId":440665,"journal":{"name":"2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131068514","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Assessment of fuzzy min-max neural networks for classification tasks 模糊最小-最大神经网络对分类任务的评价
P. Sadeghian, Aspen Olmsted
Statistical methods have been used in order to classify data from random samples. Generally, if we know the statistical distribution of the data, we can utilize specific classifiers designed for that distribution and anticipate good results. This work assesses the accuracy of Fuzzy Min-Max Neural Network (FMM) and Enhanced Fuzzy Min-Max Neural Network (EFMM) classifiers in classification tasks using data from five different statistical distributions: Negative Binomial, Logistic, Log-Normal, Gamma, and Weibull. Results of the assessment are provided and show different accuracies based on the statistical distribution of the data. This study presents a novel approach to the classification of statistical distributions by presenting two classifiers, namely FMM and EFMM Neural Networks, capable of classifying the above statistical distributions.
为了从随机样本中对数据进行分类,已经使用了统计方法。一般来说,如果我们知道数据的统计分布,我们可以利用为该分布设计的特定分类器,并预期良好的结果。这项工作评估了模糊最小-最大神经网络(FMM)和增强模糊最小-最大神经网络(EFMM)分类器在分类任务中的准确性,使用来自五种不同统计分布的数据:负二项分布、Logistic分布、对数正态分布、伽玛分布和威布尔分布。给出了评估结果,并根据数据的统计分布显示出不同的准确性。本研究提出了一种新的统计分布分类方法,通过提出两种分类器,即FMM和EFMM神经网络,能够对上述统计分布进行分类。
{"title":"Assessment of fuzzy min-max neural networks for classification tasks","authors":"P. Sadeghian, Aspen Olmsted","doi":"10.23919/ICITST.2017.8356376","DOIUrl":"https://doi.org/10.23919/ICITST.2017.8356376","url":null,"abstract":"Statistical methods have been used in order to classify data from random samples. Generally, if we know the statistical distribution of the data, we can utilize specific classifiers designed for that distribution and anticipate good results. This work assesses the accuracy of Fuzzy Min-Max Neural Network (FMM) and Enhanced Fuzzy Min-Max Neural Network (EFMM) classifiers in classification tasks using data from five different statistical distributions: Negative Binomial, Logistic, Log-Normal, Gamma, and Weibull. Results of the assessment are provided and show different accuracies based on the statistical distribution of the data. This study presents a novel approach to the classification of statistical distributions by presenting two classifiers, namely FMM and EFMM Neural Networks, capable of classifying the above statistical distributions.","PeriodicalId":440665,"journal":{"name":"2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124109255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A comparative analysis of MANET routing protocols through simulation 通过仿真对MANET路由协议进行了比较分析
Callum Brill, Thomas Nash
In this paper we investigate the performance of various mobile ad hoc network routing protocols. We use AODV and DSDV as our baseline as these are two of the most commonly used protocols in reactive and proactive routing, respectively. We compare these with the performance of an ant colony optimization implementation AntHocNet. This protocol works as a hybrid protocol, using both reactive and proactive elements. We implement the simulations across a variety of scenarios using the NS2 network simulator and present our findings.
本文研究了各种移动自组织网络路由协议的性能。我们使用AODV和DSDV作为基准,因为它们分别是被动路由和主动路由中最常用的两种协议。我们将这些与蚁群优化实现AntHocNet的性能进行比较。该协议作为混合协议工作,同时使用被动和主动元素。我们使用NS2网络模拟器在各种场景中实现了模拟,并展示了我们的发现。
{"title":"A comparative analysis of MANET routing protocols through simulation","authors":"Callum Brill, Thomas Nash","doi":"10.23919/ICITST.2017.8356392","DOIUrl":"https://doi.org/10.23919/ICITST.2017.8356392","url":null,"abstract":"In this paper we investigate the performance of various mobile ad hoc network routing protocols. We use AODV and DSDV as our baseline as these are two of the most commonly used protocols in reactive and proactive routing, respectively. We compare these with the performance of an ant colony optimization implementation AntHocNet. This protocol works as a hybrid protocol, using both reactive and proactive elements. We implement the simulations across a variety of scenarios using the NS2 network simulator and present our findings.","PeriodicalId":440665,"journal":{"name":"2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124871533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 11
RARE: A robust algorithm for rapid encryption 用于快速加密的鲁棒算法
Tasnime Omrani, Rabei Becheikh, Olfa Mannai, Rhouma Rhouma, S. Belghith
Regarding the intensive use of images in the context of IoT, their security becomes crucial. Despite that severals lightweight ciphers have been proposed, they are not suitable for images data due to their features such as correlation, redundancy and voluminosity. In this context, we treat in this paper the weaknesses of the existent lightweight ciphers concerning the constraints of images. Additionally, we propose an appropriate lightweight image cryptosystem that takes all the features of this kind of data. The experimental results show the effectiveness of the new scheme compared to existent ones.
对于在物联网背景下大量使用图像,其安全性变得至关重要。尽管已经提出了几种轻量级密码,但由于其相关性、冗余性和体积性等特点,它们不适合用于图像数据。在此背景下,本文讨论了现有轻量级密码在图像约束方面的弱点。此外,我们提出了一种适当的轻量级图像密码系统,它具有这类数据的所有特征。实验结果表明,与现有方案相比,新方案是有效的。
{"title":"RARE: A robust algorithm for rapid encryption","authors":"Tasnime Omrani, Rabei Becheikh, Olfa Mannai, Rhouma Rhouma, S. Belghith","doi":"10.23919/ICITST.2017.8356339","DOIUrl":"https://doi.org/10.23919/ICITST.2017.8356339","url":null,"abstract":"Regarding the intensive use of images in the context of IoT, their security becomes crucial. Despite that severals lightweight ciphers have been proposed, they are not suitable for images data due to their features such as correlation, redundancy and voluminosity. In this context, we treat in this paper the weaknesses of the existent lightweight ciphers concerning the constraints of images. Additionally, we propose an appropriate lightweight image cryptosystem that takes all the features of this kind of data. The experimental results show the effectiveness of the new scheme compared to existent ones.","PeriodicalId":440665,"journal":{"name":"2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121396961","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Efficient hardware implementation of ITUbee for lightweight application ITUbee轻量级应用的高效硬件实现
Juhua Liu, Wei Li, Guoqiang Bai
Recently, a new lightweight block cryptography algorithm, ITUbee, has been proposed by Ferhat Karakoc in Lightsec 2013. An efficient hardware implementation of ITUbee is presented in this paper. Firstly, we reuse certain module, which takes a big share of hardware resource, to achieve better resource utilization. Secondly, we apply composite field to implement 8-bit S-box instead of the traditional looking up tables(LUTs) to save area requirements. In the end, we conclude that the hardware implementation of ITUbee requires about 6448 GE on 0.18 um technology. The area consumption of ITUbee is roughly 31.2% less than the round-based implementation. And it costs 365.6 GE to implement 8-bit S-box by using composite field, 32.7% less than by using LUTs.
最近,Ferhat Karakoc在Lightsec 2013中提出了一种新的轻量级块加密算法ITUbee。本文提出了一种高效的ITUbee硬件实现方案。首先,我们对某些占用大量硬件资源的模块进行重用,以达到更好的资源利用率。其次,我们采用复合字段来实现8位S-box,而不是传统的查找表(LUTs),以节省面积需求。最后,我们得出结论,ITUbee的硬件实现在0.18 um技术上大约需要6448 GE。ITUbee的面积消耗大约比基于轮询的实现少31.2%。使用复合字段实现8位S-box的成本为365.6 GE,比使用lut节省32.7%。
{"title":"Efficient hardware implementation of ITUbee for lightweight application","authors":"Juhua Liu, Wei Li, Guoqiang Bai","doi":"10.23919/ICITST.2017.8356424","DOIUrl":"https://doi.org/10.23919/ICITST.2017.8356424","url":null,"abstract":"Recently, a new lightweight block cryptography algorithm, ITUbee, has been proposed by Ferhat Karakoc in Lightsec 2013. An efficient hardware implementation of ITUbee is presented in this paper. Firstly, we reuse certain module, which takes a big share of hardware resource, to achieve better resource utilization. Secondly, we apply composite field to implement 8-bit S-box instead of the traditional looking up tables(LUTs) to save area requirements. In the end, we conclude that the hardware implementation of ITUbee requires about 6448 GE on 0.18 um technology. The area consumption of ITUbee is roughly 31.2% less than the round-based implementation. And it costs 365.6 GE to implement 8-bit S-box by using composite field, 32.7% less than by using LUTs.","PeriodicalId":440665,"journal":{"name":"2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)","volume":"80 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116538435","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Considerations for OSV over Linux-based virtual machines 基于linux的虚拟机上的OSV注意事项
Wayne Chen, Aspen Olmsted
The focus of this research work is to explore the operational considerations of using a lightweight operating system (OSV) over standard Linux-based operating systems for virtualization. We focus on three aspects: a review of published literature in regards to performance; a comparison of virtualization footprint for a common application (Apache Tomcat), and an analysis of the possible security advantages. The results provide a rationale as to when lightweight operating systems may be advantageous for virtualization.
这项研究工作的重点是探索使用轻量级操作系统(OSV)而不是标准的基于linux的操作系统进行虚拟化的操作考虑。我们主要关注三个方面:对已发表的有关性能的文献进行综述;比较了一个通用应用程序(Apache Tomcat)的虚拟化占用空间,并分析了可能的安全优势。研究结果为轻量级操作系统何时可能对虚拟化有利提供了理论依据。
{"title":"Considerations for OSV over Linux-based virtual machines","authors":"Wayne Chen, Aspen Olmsted","doi":"10.23919/ICITST.2017.8356415","DOIUrl":"https://doi.org/10.23919/ICITST.2017.8356415","url":null,"abstract":"The focus of this research work is to explore the operational considerations of using a lightweight operating system (OSV) over standard Linux-based operating systems for virtualization. We focus on three aspects: a review of published literature in regards to performance; a comparison of virtualization footprint for a common application (Apache Tomcat), and an analysis of the possible security advantages. The results provide a rationale as to when lightweight operating systems may be advantageous for virtualization.","PeriodicalId":440665,"journal":{"name":"2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)","volume":"96 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122510375","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A secure enterprise architecture focused on security and technology-transformation (SEAST) 专注于安全和技术转换(SEAST)的安全企业架构
Md. Tomig Uddin Ahmed, Nazrul Islam Bhuiya, Md. Mahbubur Rahman
Sustainable and flexible development of an organizational business and achievement of current and future goals of an organization depends on business processes, technologies and information systems. In order to address these issues, there have been a number of enterprise architectures proposed in the literature. Unfortunately, many of the existing architectures pay little attention to security and sustainability. To create an accelerated and flexible IT infrastructure, it is needed to align IT and organizational business demands. Here, we proposed an enterprise architecture which is sustainable, secure and convivial to new technology for a developing or developed organizations. It is a robust and easily understandable architecture which includes single source identification system and a knowledge base with sustainable technology-transformation capabilities. This architecture is based on concepts, modeling and processes which can be used as the framework for both business and IT professionals. The proposed architecture includes kernel based security architecture with risk, incident and audit management systems. It is a high performing architecture which includes all the essential features of modern enterprise architecture (EA) enhanced with security and sustainability.
组织业务的可持续和灵活发展以及组织当前和未来目标的实现取决于业务流程、技术和信息系统。为了解决这些问题,文献中已经提出了许多企业架构。不幸的是,许多现有的架构很少关注安全性和可持续性。要创建一个加速且灵活的IT基础设施,需要将IT和组织业务需求结合起来。在这里,我们为发展中或发达组织提出了一个可持续的、安全的、适合新技术的企业架构。它是一个健壮且易于理解的体系结构,包括单源识别系统和具有可持续技术转换能力的知识库。此体系结构基于概念、建模和流程,可作为业务和IT专业人员的框架。提出的体系结构包括基于内核的安全体系结构,具有风险、事件和审计管理系统。它是一种高性能的体系结构,包含了现代企业体系结构(EA)的所有基本特性,并通过安全性和可持续性进行了增强。
{"title":"A secure enterprise architecture focused on security and technology-transformation (SEAST)","authors":"Md. Tomig Uddin Ahmed, Nazrul Islam Bhuiya, Md. Mahbubur Rahman","doi":"10.23919/ICITST.2017.8356386","DOIUrl":"https://doi.org/10.23919/ICITST.2017.8356386","url":null,"abstract":"Sustainable and flexible development of an organizational business and achievement of current and future goals of an organization depends on business processes, technologies and information systems. In order to address these issues, there have been a number of enterprise architectures proposed in the literature. Unfortunately, many of the existing architectures pay little attention to security and sustainability. To create an accelerated and flexible IT infrastructure, it is needed to align IT and organizational business demands. Here, we proposed an enterprise architecture which is sustainable, secure and convivial to new technology for a developing or developed organizations. It is a robust and easily understandable architecture which includes single source identification system and a knowledge base with sustainable technology-transformation capabilities. This architecture is based on concepts, modeling and processes which can be used as the framework for both business and IT professionals. The proposed architecture includes kernel based security architecture with risk, incident and audit management systems. It is a high performing architecture which includes all the essential features of modern enterprise architecture (EA) enhanced with security and sustainability.","PeriodicalId":440665,"journal":{"name":"2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121242408","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Low-cost detection of backdoor malware 低成本检测后门恶意软件
H. Loi, Aspen Olmsted
Backdoor malware are programs that enable hackers to access unauthorized computer systems by introducing a backdoor. These hackers will use this access to steal company information for personal gain. This malware uses a variety of techniques to hide their presence, and computer security researchers use a growing number of exotic techniques to detect them. However, it is not necessary to expend valuable IT resources on expensive security solutions as most of these backdoors can be detected by simple checks. We tested a wide array of in-the-wild malware to verily the effectiveness of these checks.
后门恶意软件是指通过引入后门使黑客能够访问未经授权的计算机系统的程序。这些黑客将利用这种访问来窃取公司信息以谋取个人利益。这种恶意软件使用各种各样的技术来隐藏它们的存在,计算机安全研究人员使用越来越多的外来技术来检测它们。但是,没有必要在昂贵的安全解决方案上花费宝贵的it资源,因为大多数后门可以通过简单的检查检测到。我们测试了大量的野外恶意软件,以验证这些检查的有效性。
{"title":"Low-cost detection of backdoor malware","authors":"H. Loi, Aspen Olmsted","doi":"10.23919/ICITST.2017.8356377","DOIUrl":"https://doi.org/10.23919/ICITST.2017.8356377","url":null,"abstract":"Backdoor malware are programs that enable hackers to access unauthorized computer systems by introducing a backdoor. These hackers will use this access to steal company information for personal gain. This malware uses a variety of techniques to hide their presence, and computer security researchers use a growing number of exotic techniques to detect them. However, it is not necessary to expend valuable IT resources on expensive security solutions as most of these backdoors can be detected by simple checks. We tested a wide array of in-the-wild malware to verily the effectiveness of these checks.","PeriodicalId":440665,"journal":{"name":"2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123286265","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Spark framework for transcriptomic trimming algorithm reduces cost of reading multiple input files 转录修剪算法的Spark框架减少了读取多个输入文件的成本
W. Blair, Aspen Olmsted, Paul E. Anderson
In this paper, we investigate the feasibility and performance improvement of adapting a common stand-alone bioinformatics trimming tool for in-memory processing on a distributed Spark framework. The rapid and continuous rise of genomics technologies and applications demands fast and efficient genomic data processing pipelines. ADAM has emerged as a successful framework for handling large scientific datasets, and efforts are ongoing to expand its functionality in the bioinformatics pipeline. We hypothesize that executing as much of the pipeline as possible within the ADAM framework will improve the pipeline's time and disk requirements. We compare Trimmomatic, one of the most common raw read trimming algorithms, to our own simple Scala trimmer and show that the distributed framework allows our trimmer to suffer less overhead from increasing the number of input files. We conclude that executing Trimmomatic in Spark will improve performance with multiple file inputs. Future work will investigate the performance benefit of passing the distributed dataset directly to ADAM in memory rather than writing out an intermediate file to disk.
在本文中,我们研究了在分布式Spark框架上采用通用的独立生物信息学修剪工具进行内存处理的可行性和性能改进。基因组学技术和应用的快速和持续的崛起需要快速和高效的基因组数据处理管道。ADAM已经成为处理大型科学数据集的成功框架,并且正在努力扩展其在生物信息学管道中的功能。我们假设在ADAM框架内执行尽可能多的管道将改善管道的时间和磁盘需求。我们将Trimmomatic(最常见的原始读取修剪算法之一)与我们自己的简单Scala修剪器进行比较,并显示分布式框架允许我们的修剪器在增加输入文件数量时承受更少的开销。我们得出结论,在Spark中执行Trimmomatic将提高多文件输入的性能。未来的工作将研究将分布式数据集直接传递给内存中的ADAM而不是将中间文件写入磁盘的性能优势。
{"title":"Spark framework for transcriptomic trimming algorithm reduces cost of reading multiple input files","authors":"W. Blair, Aspen Olmsted, Paul E. Anderson","doi":"10.23919/ICITST.2017.8356451","DOIUrl":"https://doi.org/10.23919/ICITST.2017.8356451","url":null,"abstract":"In this paper, we investigate the feasibility and performance improvement of adapting a common stand-alone bioinformatics trimming tool for in-memory processing on a distributed Spark framework. The rapid and continuous rise of genomics technologies and applications demands fast and efficient genomic data processing pipelines. ADAM has emerged as a successful framework for handling large scientific datasets, and efforts are ongoing to expand its functionality in the bioinformatics pipeline. We hypothesize that executing as much of the pipeline as possible within the ADAM framework will improve the pipeline's time and disk requirements. We compare Trimmomatic, one of the most common raw read trimming algorithms, to our own simple Scala trimmer and show that the distributed framework allows our trimmer to suffer less overhead from increasing the number of input files. We conclude that executing Trimmomatic in Spark will improve performance with multiple file inputs. Future work will investigate the performance benefit of passing the distributed dataset directly to ADAM in memory rather than writing out an intermediate file to disk.","PeriodicalId":440665,"journal":{"name":"2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121760131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
期刊
2017 12th International Conference for Internet Technology and Secured Transactions (ICITST)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1