首页 > 最新文献

2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)最新文献

英文 中文
Efficient Multi-GPU Memory Management for Deep Learning Acceleration
Youngrang Kim, J. Lee, Jik-Soo Kim, Hyunseung Jei, Hongchan Roh
In this paper, we propose a new optimized memory management scheme that can improve the overall GPU memory utilization in multi-GPU systems for deep learning application acceleration. We extend the Nvidia's vDNN concept (a hybrid utilization of GPU and CPU memories) in a multi-GPU environment by effectively addressing PCIe-bus contention problems. In addition, we designed and implemented an intelligent prefetching algorithm (from CPU memory to GPU) that can achieve the highest processing throughput while sustaining a large min-batch size. For evaluation, we have implemented our memory usage optimization scheme on Tensorflow, the well-known machine learning library from Google, and performed extensive experiments in a multi-GPU testbed. Our evaluation results show that the proposed scheme can increase the mini-batch size by up to 60%, and improve the training throughput by up to 46.6% in a multi-GPU system.
在本文中,我们提出了一种新的优化内存管理方案,该方案可以提高多GPU系统中深度学习应用程序加速的GPU内存利用率。我们通过有效地解决pcie总线争用问题,在多GPU环境中扩展了Nvidia的vDNN概念(GPU和CPU内存的混合利用)。此外,我们设计并实现了一种智能预取算法(从CPU内存到GPU),可以实现最高的处理吞吐量,同时保持较大的最小批大小。为了进行评估,我们在谷歌著名的机器学习库Tensorflow上实现了我们的内存使用优化方案,并在多gpu测试平台上进行了广泛的实验。我们的评估结果表明,在多gpu系统中,提出的方案可以将小批大小增加多达60%,并将训练吞吐量提高多达46.6%。
{"title":"Efficient Multi-GPU Memory Management for Deep Learning Acceleration","authors":"Youngrang Kim, J. Lee, Jik-Soo Kim, Hyunseung Jei, Hongchan Roh","doi":"10.1109/FAS-W.2018.00023","DOIUrl":"https://doi.org/10.1109/FAS-W.2018.00023","url":null,"abstract":"In this paper, we propose a new optimized memory management scheme that can improve the overall GPU memory utilization in multi-GPU systems for deep learning application acceleration. We extend the Nvidia's vDNN concept (a hybrid utilization of GPU and CPU memories) in a multi-GPU environment by effectively addressing PCIe-bus contention problems. In addition, we designed and implemented an intelligent prefetching algorithm (from CPU memory to GPU) that can achieve the highest processing throughput while sustaining a large min-batch size. For evaluation, we have implemented our memory usage optimization scheme on Tensorflow, the well-known machine learning library from Google, and performed extensive experiments in a multi-GPU testbed. Our evaluation results show that the proposed scheme can increase the mini-batch size by up to 60%, and improve the training throughput by up to 46.6% in a multi-GPU system.","PeriodicalId":164903,"journal":{"name":"2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129401146","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 10
Engineering Self-Adaptive Software Systems – An Organized Tour 工程自适应软件系统-组织之旅
Danny Weyns
Engineering software that is subject to uncertainties that are difficult to anticipate before deployment is challenging. Self-adaptation extends a software system with an external feedback loop system that monitors the system and adapts its configuration or architecture to ensure that its qualities are met under uncertain operating conditions. In this tutorial, we provide a particular perspective on the evolution of the field of selfadaptation in six waves. These waves put complementary aspects of engineering self-adaptive systems in focus that synergetically have contributed to the current body of knowledge in the field.
工程软件受到不确定性的影响,在部署之前很难预测,这是具有挑战性的。自适应扩展了软件系统的外部反馈循环系统,该系统监视系统并调整其配置或架构,以确保在不确定的操作条件下满足其质量。在本教程中,我们提供了一个特定的观点,在六个波自适应领域的演变。这些浪潮将工程自适应系统的互补方面放在了焦点上,这些方面共同促进了该领域当前的知识体系。
{"title":"Engineering Self-Adaptive Software Systems – An Organized Tour","authors":"Danny Weyns","doi":"10.1109/FAS-W.2018.00012","DOIUrl":"https://doi.org/10.1109/FAS-W.2018.00012","url":null,"abstract":"Engineering software that is subject to uncertainties that are difficult to anticipate before deployment is challenging. Self-adaptation extends a software system with an external feedback loop system that monitors the system and adapts its configuration or architecture to ensure that its qualities are met under uncertain operating conditions. In this tutorial, we provide a particular perspective on the evolution of the field of selfadaptation in six waves. These waves put complementary aspects of engineering self-adaptive systems in focus that synergetically have contributed to the current body of knowledge in the field.","PeriodicalId":164903,"journal":{"name":"2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)","volume":"122 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133665012","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 21
Toward an Adaptive Fair GPU Sharing Scheme in Container-Based Clusters 基于容器集群的自适应公平GPU共享方案研究
Jisun Oh, Seoyoung Kim, Yoonhee Kim
Virtualization is an innovative technology that accelerates software development by providing portability and maintainability of applications. However, it often leads underperformance especially caused by overheads from managing virtual machines. To address the limitation of virtual machines, container technology has emerged to deploy and operate distributed applications without launching entire virtual machines. Unfortunately, resources contention issues in container-based clusters, bringing substantial performance loss are still challenging. This paper proposes an adaptive fair-share method to share effectively in container-based virtualization environment. In particular, we focus on enabling GPU sharing between multiple concurrent containers without lack of GPU memory. We demonstrate that our approach contributes to overall performance improvement as well as higher resource utilization compared to default and static fair-share methods with homogeneous and heterogeneous workloads. Compared to two other conditions, their results show that the proposed method reduces by 16.37%, 15.61% in average execution time and boosts approximately by 52.46%, 10.3% in average GPU memory utilization, respectively.
虚拟化是一种创新技术,通过提供应用程序的可移植性和可维护性来加速软件开发。但是,它经常导致性能不佳,特别是由于管理虚拟机的开销造成的。为了解决虚拟机的局限性,出现了容器技术,可以在不启动整个虚拟机的情况下部署和操作分布式应用程序。不幸的是,在基于容器的集群中,资源争用问题带来的巨大性能损失仍然具有挑战性。为了在基于容器的虚拟化环境中实现有效的共享,提出了一种自适应公平共享方法。特别是,我们专注于在不缺乏GPU内存的情况下在多个并发容器之间实现GPU共享。我们证明,与具有同构和异构工作负载的默认和静态公平共享方法相比,我们的方法有助于整体性能改进以及更高的资源利用率。与其他两种情况相比,他们的结果表明,该方法的平均执行时间分别减少了16.37%,15.61%,平均GPU内存利用率分别提高了52.46%,10.3%。
{"title":"Toward an Adaptive Fair GPU Sharing Scheme in Container-Based Clusters","authors":"Jisun Oh, Seoyoung Kim, Yoonhee Kim","doi":"10.1109/FAS-W.2018.00029","DOIUrl":"https://doi.org/10.1109/FAS-W.2018.00029","url":null,"abstract":"Virtualization is an innovative technology that accelerates software development by providing portability and maintainability of applications. However, it often leads underperformance especially caused by overheads from managing virtual machines. To address the limitation of virtual machines, container technology has emerged to deploy and operate distributed applications without launching entire virtual machines. Unfortunately, resources contention issues in container-based clusters, bringing substantial performance loss are still challenging. This paper proposes an adaptive fair-share method to share effectively in container-based virtualization environment. In particular, we focus on enabling GPU sharing between multiple concurrent containers without lack of GPU memory. We demonstrate that our approach contributes to overall performance improvement as well as higher resource utilization compared to default and static fair-share methods with homogeneous and heterogeneous workloads. Compared to two other conditions, their results show that the proposed method reduces by 16.37%, 15.61% in average execution time and boosts approximately by 52.46%, 10.3% in average GPU memory utilization, respectively.","PeriodicalId":164903,"journal":{"name":"2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)","volume":"81 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133083346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Aspects of Measuring and Evaluating the Integration Status of a (Sub-)System at Runtime 运行时测量和评价(子系统)集成状态的几个方面
Christian Gruhl, Sven Tomforde, B. Sick
The basic idea of self-improving system integration (SISSY) is that the system itself autonomously decides about its own integration status and combines this maintenance with mechanisms to optimise it. Such an approach requires a quantifiable basis of all necessary system attributes and a set of goal functions defining desired solutions. In this article, we define a basic model of such a SISSY system and use this as basis to revisit possible aspects for measuring and evaluating the integration status. Different assessment criteria are suggested, based on performance, cost, security, behaviour, and more.
自完善系统集成(SISSY)的基本思想是系统自身自主决定自身的集成状态,并将这种维护与优化机制相结合。这种方法需要所有必要的系统属性和一组定义所需解决方案的目标函数的可量化基础。在本文中,我们定义了这样一个SISSY系统的基本模型,并以此为基础重新审视测量和评估集成状态的可能方面。根据性能、成本、安全性、行为等,提出了不同的评估标准。
{"title":"Aspects of Measuring and Evaluating the Integration Status of a (Sub-)System at Runtime","authors":"Christian Gruhl, Sven Tomforde, B. Sick","doi":"10.1109/FAS-W.2018.00047","DOIUrl":"https://doi.org/10.1109/FAS-W.2018.00047","url":null,"abstract":"The basic idea of self-improving system integration (SISSY) is that the system itself autonomously decides about its own integration status and combines this maintenance with mechanisms to optimise it. Such an approach requires a quantifiable basis of all necessary system attributes and a set of goal functions defining desired solutions. In this article, we define a basic model of such a SISSY system and use this as basis to revisit possible aspects for measuring and evaluating the integration status. Different assessment criteria are suggested, based on performance, cost, security, behaviour, and more.","PeriodicalId":164903,"journal":{"name":"2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)","volume":"82 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115150723","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Design and Implementation of Software-Based Dynamically Connected Transport 基于软件的动态连接传输的设计与实现
Jiwoong Park, H. Yeom
Most RDMA applications are based on either RC QP(Reliable Connected Queue Pair) or UD QP(Unreliable Datagram Queue Pair). RC QP has advantages on usability but disadvantages on scalability while UD QP is scalable but hard to use. Dynamically Connected (DC) transport has been introduced to address the limitations when using one of the existing QPs. However, DC transport is only available with hardware support. In this paper, we design and implement a totally software-based DCT prototype to eliminate hardware dependency. Our experimental results show that although our prototype adds a first-time connection overhead it could provide the advantages of both RC QP and UD QP. We also write and evaluate a DC QP-based application to prove its effectiveness compared to RC QP-based one.
大多数RDMA应用程序基于RC QP(可靠连接队列对)或UD QP(不可靠数据报队列对)。RC QP在可用性上有优势,但在可扩展性上有劣势;UD QP在可扩展性上有优势,但难以使用。引入动态连接(DC)传输是为了解决使用现有qp之一时的限制。但是,只有在硬件支持的情况下才能使用DC传输。在本文中,我们设计并实现了一个完全基于软件的DCT原型,以消除硬件依赖。我们的实验结果表明,虽然我们的原型增加了首次连接开销,但它可以提供RC QP和UD QP的优点。我们还编写并评估了一个基于DC qp的应用程序,以证明其与基于RC qp的应用程序相比的有效性。
{"title":"Design and Implementation of Software-Based Dynamically Connected Transport","authors":"Jiwoong Park, H. Yeom","doi":"10.1109/FAS-W.2018.00026","DOIUrl":"https://doi.org/10.1109/FAS-W.2018.00026","url":null,"abstract":"Most RDMA applications are based on either RC QP(Reliable Connected Queue Pair) or UD QP(Unreliable Datagram Queue Pair). RC QP has advantages on usability but disadvantages on scalability while UD QP is scalable but hard to use. Dynamically Connected (DC) transport has been introduced to address the limitations when using one of the existing QPs. However, DC transport is only available with hardware support. In this paper, we design and implement a totally software-based DCT prototype to eliminate hardware dependency. Our experimental results show that although our prototype adds a first-time connection overhead it could provide the advantages of both RC QP and UD QP. We also write and evaluate a DC QP-based application to prove its effectiveness compared to RC QP-based one.","PeriodicalId":164903,"journal":{"name":"2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)","volume":"76 5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127393401","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Title Page i 第1页
{"title":"Title Page i","authors":"","doi":"10.1109/fas-w.2018.00001","DOIUrl":"https://doi.org/10.1109/fas-w.2018.00001","url":null,"abstract":"","PeriodicalId":164903,"journal":{"name":"2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125338691","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Tool for Online Experiment-Driven Adaptation 在线实验驱动适应工具
I. Gerostathopoulos, Ali Naci Uysal, C. Prehofer, T. Bures
In this paper, we present Online Experiment-Driven Adaptation (OEDA), a tool for performing end-to-end optimization of a target system abstracted as a black-box by combining statistical and optimization methods and providing statistical guarantees along the optimization process. We present the requirements and architecture of OEDA and describe its built-in optimization process that chains together factorial design, Bayesian optimization, and t-test. OEDA allows the user to create reusable abstractions of systems-to-be-optimized and specify, run and observe the execution of end-to-end experiments. For instance, we support data exchange with common tools like Kafka, MQTT and HTTP. We show the benefits of OEDA in a web server application example. OEDA can be a useful vehicle for research in the area of automated experimentation, an emerging challenge where systems are capable of performing experiments (akin to A/B testing) to themselves in order to self-optimize.
在本文中,我们提出了在线实验驱动自适应(Online experimental - driven Adaptation, OEDA),这是一种将统计方法和优化方法相结合,对抽象为黑盒的目标系统进行端到端优化的工具,并在优化过程中提供统计保证。我们介绍了OEDA的需求和体系结构,并描述了其内置的优化过程,该过程将析因设计、贝叶斯优化和t检验联系在一起。OEDA允许用户创建待优化系统的可重用抽象,并指定、运行和观察端到端实验的执行。例如,我们支持与Kafka、MQTT和HTTP等常用工具进行数据交换。我们在一个web服务器应用程序示例中展示了OEDA的好处。OEDA可以成为自动化实验领域研究的有用工具,这是一个新兴的挑战,系统能够对自己执行实验(类似于a /B测试)以实现自我优化。
{"title":"A Tool for Online Experiment-Driven Adaptation","authors":"I. Gerostathopoulos, Ali Naci Uysal, C. Prehofer, T. Bures","doi":"10.1109/FAS-W.2018.00032","DOIUrl":"https://doi.org/10.1109/FAS-W.2018.00032","url":null,"abstract":"In this paper, we present Online Experiment-Driven Adaptation (OEDA), a tool for performing end-to-end optimization of a target system abstracted as a black-box by combining statistical and optimization methods and providing statistical guarantees along the optimization process. We present the requirements and architecture of OEDA and describe its built-in optimization process that chains together factorial design, Bayesian optimization, and t-test. OEDA allows the user to create reusable abstractions of systems-to-be-optimized and specify, run and observe the execution of end-to-end experiments. For instance, we support data exchange with common tools like Kafka, MQTT and HTTP. We show the benefits of OEDA in a web server application example. OEDA can be a useful vehicle for research in the area of automated experimentation, an emerging challenge where systems are capable of performing experiments (akin to A/B testing) to themselves in order to self-optimize.","PeriodicalId":164903,"journal":{"name":"2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121969427","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Reactive and Adaptive Security Monitoring in Cloud Computing 云计算中的响应式和自适应安全监控
Clément Elbaz, Louis Rilling, C. Morin
Cloud computing enabled service-level agreements (SLAs) to gain widespread use among information systems stakeholders. It is now normal for performance and availability of such systems to be carefully measured and evaluated. Contracts that include financial penalties in case of breach are now common. However security is lagging behind this trend; it is as important to stakeholders as performance and availability, but is generally not included in the scope of service-level agreements between stakeholders, and handled instead on a best-effort basis, without any transparency nor SLA with their clients. One reason for this is the difficulty of objectively measuring security. Indeed, the actual security level of a system is dependent on a wide range of factors, some intrinsic to the system - such as a design or implementation mistake resulting in a vulnerability - and some extrinsic to it. For instance, an external event such as the publication of a vulnerability in an open-source software dependency or a change of political context in a country can widely impact the risks faced by an information system even if no actual change were made to the system. These factors are even more numerous in multi-tenant cloud infrastructures because of the sheer number of actors involved - and their sometimes conflicting incentives - and opportunities for both attack and defense at scale. Security monitoring aims to detect and react to attacks in real time; Reactive security monitoring intends to take external events into account while doing so. Improving the accuracy of a system's overall security assessment can help transitioning security to a SLA paradigm and enable better transparency for all stakeholders.
云计算使服务水平协议(sla)在信息系统涉众中得到广泛使用。现在,仔细测量和评估这些系统的性能和可用性是正常的。如果违约,包括经济处罚的合同现在很常见。然而,安全性落后于这一趋势;对于涉众来说,它与性能和可用性一样重要,但通常不包括在涉众之间的服务水平协议范围内,而是在尽最大努力的基础上处理,没有任何透明度,也没有与客户的SLA。原因之一是难以客观地衡量安全性。实际上,系统的实际安全级别取决于一系列因素,其中一些是系统固有的因素(例如导致漏洞的设计或实现错误),还有一些是系统外在的因素。例如,公开开源软件依赖项中的漏洞或一国政治环境的变化等外部事件,即使没有对信息系统进行实际更改,也可能对信息系统面临的风险产生广泛影响。在多租户云基础设施中,这些因素甚至更多,因为涉及的参与者数量非常多——有时他们的动机相互冲突——以及大规模攻击和防御的机会。安全监控旨在实时检测和应对攻击;响应式安全监视打算在执行此操作时考虑外部事件。提高系统整体安全评估的准确性可以帮助将安全性转换为SLA范例,并为所有涉众提供更好的透明度。
{"title":"Reactive and Adaptive Security Monitoring in Cloud Computing","authors":"Clément Elbaz, Louis Rilling, C. Morin","doi":"10.1109/FAS-W.2018.00014","DOIUrl":"https://doi.org/10.1109/FAS-W.2018.00014","url":null,"abstract":"Cloud computing enabled service-level agreements (SLAs) to gain widespread use among information systems stakeholders. It is now normal for performance and availability of such systems to be carefully measured and evaluated. Contracts that include financial penalties in case of breach are now common. However security is lagging behind this trend; it is as important to stakeholders as performance and availability, but is generally not included in the scope of service-level agreements between stakeholders, and handled instead on a best-effort basis, without any transparency nor SLA with their clients. One reason for this is the difficulty of objectively measuring security. Indeed, the actual security level of a system is dependent on a wide range of factors, some intrinsic to the system - such as a design or implementation mistake resulting in a vulnerability - and some extrinsic to it. For instance, an external event such as the publication of a vulnerability in an open-source software dependency or a change of political context in a country can widely impact the risks faced by an information system even if no actual change were made to the system. These factors are even more numerous in multi-tenant cloud infrastructures because of the sheer number of actors involved - and their sometimes conflicting incentives - and opportunities for both attack and defense at scale. Security monitoring aims to detect and react to attacks in real time; Reactive security monitoring intends to take external events into account while doing so. Improving the accuracy of a system's overall security assessment can help transitioning security to a SLA paradigm and enable better transparency for all stakeholders.","PeriodicalId":164903,"journal":{"name":"2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114294754","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Design of Skiplist Based Key-Value Store on Non-Volatile Memory 非易失性存储器中基于Skiplist键值存储的设计
Qichen Chen, H. Yeom
Non-volatile Memory is promising to persistent data storage, which has outstanding advantages against traditional storage devices such as HDD and SSD. One of its hugest advantages is its DRAM-like read latency and micro second level write latency, which is several hundred times faster than the original block device. However, one of the issues on using non-volatile memory as storage device is designing a suitable indexing system for data stores there, in which the characteristics of non-volatile memory can be able to make full use of. The state of the art indexing systems of non-volatile key-value stores are usually based on B+-tree or its variant, which are originally designed for mechanic hard disk and volatile memory. The semantics of B+-tree requires inside data being sorted and frequent split and merge for keeping its balance. However, both of sorting and splitting will cause extra write to non-volatile memory, which will downgrade the performance. As a result, B+-tree and its variant may not be naturally suitable for non-volatile memory. In this paper, we proposed a skiplist based indexing system for non-volatile memory key-value store(NV-Skiplist), which can take fully use of the features of both the non-volatile memory and DRAM. NV-Skiplist constructs its bottom layer in the non-volatile memory for data persistence and supporting range scan, it also builds its upper layers in the DRAM to retain fast index searching and prevent large consistent overhead. We also introduced a multi-ranged variant to increase the search performance. We evaluate the performance of NV-Skiplist on a non-volatile memory emulator with a server that has Intel Xeon E5-2620 v2 processor. The experimental results show that our design outperforms the original tree-based non-volatile key-value store on both insertion and search performance by both 27% and 12% in each case.
非易失性存储器(Non-volatile Memory)有望实现持久的数据存储,与HDD、SSD等传统存储设备相比,具有明显的优势。它最大的优点之一是具有类似dram的读取延迟和微秒级的写入延迟,比原始块设备快数百倍。然而,使用非易失性存储器作为存储设备的问题之一是如何设计适合于非易失性存储器的数据存储索引系统,使其能够充分利用非易失性存储器的特性。非易失性键值存储的最新索引系统通常基于B+树或其变体,它们最初是为机械硬盘和易失性存储器设计的。B+-tree的语义要求对内部数据进行排序,并经常拆分和合并以保持其平衡。但是,排序和拆分都会导致对非易失性内存的额外写入,从而降低性能。因此,B+-tree及其变体可能不适合非易失性存储器。本文提出了一种基于skiplist的非易失性存储器键值存储索引系统(NV-Skiplist),该系统可以充分利用非易失性存储器和DRAM的特点。NV-Skiplist在非易失性存储器中构建其底层,用于数据持久性和支持范围扫描,它还在DRAM中构建其上层,以保持快速索引搜索并防止大的一致性开销。我们还引入了一个多范围变体来提高搜索性能。我们在具有Intel至强E5-2620 v2处理器的服务器上评估了NV-Skiplist在非易失性存储器模拟器上的性能。实验结果表明,我们的设计在插入和搜索性能上分别比原始的基于树的非易失性键值存储高出27%和12%。
{"title":"Design of Skiplist Based Key-Value Store on Non-Volatile Memory","authors":"Qichen Chen, H. Yeom","doi":"10.1109/FAS-W.2018.00024","DOIUrl":"https://doi.org/10.1109/FAS-W.2018.00024","url":null,"abstract":"Non-volatile Memory is promising to persistent data storage, which has outstanding advantages against traditional storage devices such as HDD and SSD. One of its hugest advantages is its DRAM-like read latency and micro second level write latency, which is several hundred times faster than the original block device. However, one of the issues on using non-volatile memory as storage device is designing a suitable indexing system for data stores there, in which the characteristics of non-volatile memory can be able to make full use of. The state of the art indexing systems of non-volatile key-value stores are usually based on B+-tree or its variant, which are originally designed for mechanic hard disk and volatile memory. The semantics of B+-tree requires inside data being sorted and frequent split and merge for keeping its balance. However, both of sorting and splitting will cause extra write to non-volatile memory, which will downgrade the performance. As a result, B+-tree and its variant may not be naturally suitable for non-volatile memory. In this paper, we proposed a skiplist based indexing system for non-volatile memory key-value store(NV-Skiplist), which can take fully use of the features of both the non-volatile memory and DRAM. NV-Skiplist constructs its bottom layer in the non-volatile memory for data persistence and supporting range scan, it also builds its upper layers in the DRAM to retain fast index searching and prevent large consistent overhead. We also introduced a multi-ranged variant to increase the search performance. We evaluate the performance of NV-Skiplist on a non-volatile memory emulator with a server that has Intel Xeon E5-2620 v2 processor. The experimental results show that our design outperforms the original tree-based non-volatile key-value store on both insertion and search performance by both 27% and 12% in each case.","PeriodicalId":164903,"journal":{"name":"2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)","volume":"96 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128136798","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
AMGCC 2018 Foreword
H. Yeom, Kento Aida, Jaeyoung Choi, Young-ri Choi, E. Deelman, Sandro Fiore, R. G. Diaz, Hyeonsang Eom, E. Heien, J. Jensen, Jik-Soo Kim, Yoonhee Kim, Youngjae Kim, Jaewook Lee, K. Lee, J. Lee, Myungho Lee, Young-Choon Lee, R. Montella, Taiga Nakamura, Beom-Il Nam, S. Pallickara, Seung-Jong Park, Sungyong Park, R. Quick, Ilkyeun Ra, D. Sarramia, Yoshio Tanaka, Taro Tezuka, S. Timm, Ananta Tiwari, J. L. Vázquez-Poletti, J. Wozniak
{"title":"AMGCC 2018 Foreword","authors":"H. Yeom, Kento Aida, Jaeyoung Choi, Young-ri Choi, E. Deelman, Sandro Fiore, R. G. Diaz, Hyeonsang Eom, E. Heien, J. Jensen, Jik-Soo Kim, Yoonhee Kim, Youngjae Kim, Jaewook Lee, K. Lee, J. Lee, Myungho Lee, Young-Choon Lee, R. Montella, Taiga Nakamura, Beom-Il Nam, S. Pallickara, Seung-Jong Park, Sungyong Park, R. Quick, Ilkyeun Ra, D. Sarramia, Yoshio Tanaka, Taro Tezuka, S. Timm, Ananta Tiwari, J. L. Vázquez-Poletti, J. Wozniak","doi":"10.1109/fas-w.2018.00007","DOIUrl":"https://doi.org/10.1109/fas-w.2018.00007","url":null,"abstract":"","PeriodicalId":164903,"journal":{"name":"2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121641815","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1