Anomalous events can signal a variety of problems in any system. As such, robust, fast detection of anomalies is important so issues can be fixed before they cascade to create larger problems. In this paper we focus on IBM Z mainframes, although most of the problems addressed and techniques used are broadly applicable. For example, anomalies can signal issues such as disk malfunctions, slow or unresponsive modules, crashes and latent bugs, lock contention, excessive retries, the need to allocate more resources to reduce contention, etc. Although there are specific techniques for addressing individual issues, anomaly detection is useful in its broad spectrum utility, and its ability to identify combinations of problems for which there may not be a specific approach implemented. In addition, anomaly detection serves as a backstop: truly anomalous events suggest that normal mechanisms did not work. Our input for detecting anomalies is low-level, summarized information available in time series to the zOS operating system. Although such information lacks some high-level context, it does provide an operating system awareness that benefits from universal applicability to any zOS system and any code running on such a system. The data is also quite rich with 100 - 100,000 metrics per sample depending on how "metric" is defined. As might be expected the data contains metrics such as CPU utilization, execution priorities, internal locking behavior, bytes read and written by an executing process, etc. It also contains higher-level information such as the executing process names or transactional identifiers from online transactional processing facilities. Names are useful not only in detecting anomalies, but in conveying context to users trying to isolate and fix problems. Our techniques build on KL divergence [21] and learn continuously without supervision and with low overhead. Continuous learning is important. The first instance of an aberrant behavior is an anomaly. The 10th instance probably is not. This point also illustrates the utility of anomaly detection in pinpointing root cause: early detection is essential and broad-spectrum anomaly detection provides excellent capability to do just that. This paper outlines these techniques and demonstrates their efficacy in detecting and resolving key problems.
{"title":"Anomaly Detection on IBM Z Mainframes: Performance Analysis and More","authors":"Erik Altman, Benjamin Segal","doi":"10.1145/3579370.3594770","DOIUrl":"https://doi.org/10.1145/3579370.3594770","url":null,"abstract":"Anomalous events can signal a variety of problems in any system. As such, robust, fast detection of anomalies is important so issues can be fixed before they cascade to create larger problems. In this paper we focus on IBM Z mainframes, although most of the problems addressed and techniques used are broadly applicable. For example, anomalies can signal issues such as disk malfunctions, slow or unresponsive modules, crashes and latent bugs, lock contention, excessive retries, the need to allocate more resources to reduce contention, etc. Although there are specific techniques for addressing individual issues, anomaly detection is useful in its broad spectrum utility, and its ability to identify combinations of problems for which there may not be a specific approach implemented. In addition, anomaly detection serves as a backstop: truly anomalous events suggest that normal mechanisms did not work. Our input for detecting anomalies is low-level, summarized information available in time series to the zOS operating system. Although such information lacks some high-level context, it does provide an operating system awareness that benefits from universal applicability to any zOS system and any code running on such a system. The data is also quite rich with 100 - 100,000 metrics per sample depending on how \"metric\" is defined. As might be expected the data contains metrics such as CPU utilization, execution priorities, internal locking behavior, bytes read and written by an executing process, etc. It also contains higher-level information such as the executing process names or transactional identifiers from online transactional processing facilities. Names are useful not only in detecting anomalies, but in conveying context to users trying to isolate and fix problems. Our techniques build on KL divergence [21] and learn continuously without supervision and with low overhead. Continuous learning is important. The first instance of an aberrant behavior is an anomaly. The 10th instance probably is not. This point also illustrates the utility of anomaly detection in pinpointing root cause: early detection is essential and broad-spectrum anomaly detection provides excellent capability to do just that. This paper outlines these techniques and demonstrates their efficacy in detecting and resolving key problems.","PeriodicalId":180024,"journal":{"name":"Proceedings of the 16th ACM International Conference on Systems and Storage","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121098981","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alex Merenstein, Vasily Tarasov, Ali Anwar, S. Guthridge, E. Zadok
Serverless platforms offer on-demand computation and represent a significant shift from previous platforms that typically required resources to be pre-allocated (e.g., virtual machines). As serverless platforms have evolved, they have become suitable for a much wider range of applications than their original use cases. However, storage access remains a pain point that holds serverless back from becoming a completely generic computation platform. Existing storage for serverless typically uses an object interface. Although object APIs are simple to use, they lack the richness, versatility, and performance of file based APIs. Additionally, there is a large body of existing applications that relies on file-based interfaces. The lack of file based storage options prevents these applications from being ported to serverless environments. In this paper, we present F3, a file system that offers features to improve file access in serverless platforms: (1) efficient handling of ephemeral data, by placing ephemeral and non-ephemeral data on storage that exists at a different points along the durability-performance tradeoff continuum, (2) locality-aware data scheduling, and (3) efficient reading while writing. We modified OpenWhisk to support attaching file-based storage and to leverage F3's features using hints. Our prototype evaluation of F3 shows improved performance of up to 1.5--6.5× compared to existing storage systems.
{"title":"F3: Serving Files Efficiently in Serverless Computing","authors":"Alex Merenstein, Vasily Tarasov, Ali Anwar, S. Guthridge, E. Zadok","doi":"10.1145/3579370.3594771","DOIUrl":"https://doi.org/10.1145/3579370.3594771","url":null,"abstract":"Serverless platforms offer on-demand computation and represent a significant shift from previous platforms that typically required resources to be pre-allocated (e.g., virtual machines). As serverless platforms have evolved, they have become suitable for a much wider range of applications than their original use cases. However, storage access remains a pain point that holds serverless back from becoming a completely generic computation platform. Existing storage for serverless typically uses an object interface. Although object APIs are simple to use, they lack the richness, versatility, and performance of file based APIs. Additionally, there is a large body of existing applications that relies on file-based interfaces. The lack of file based storage options prevents these applications from being ported to serverless environments. In this paper, we present F3, a file system that offers features to improve file access in serverless platforms: (1) efficient handling of ephemeral data, by placing ephemeral and non-ephemeral data on storage that exists at a different points along the durability-performance tradeoff continuum, (2) locality-aware data scheduling, and (3) efficient reading while writing. We modified OpenWhisk to support attaching file-based storage and to leverage F3's features using hints. Our prototype evaluation of F3 shows improved performance of up to 1.5--6.5× compared to existing storage systems.","PeriodicalId":180024,"journal":{"name":"Proceedings of the 16th ACM International Conference on Systems and Storage","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115576327","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The disregarded performance of erase operation in low-latency NAND flash memory relatively increases the impact of garbage collection interferences, which is a challenge when considering the use for storage class memory applications. Conventional erase-related techniques do not sufficiently conceal the impact of erase operation and incur a performance trade-off between throughput and latency. In this paper, we propose a boost and stop erase (BOOSTER) technique consisting of a multi-block erase technique and an adaptive erase suspension technique. The proposed technique improves both throughput and latency of low-latency NAND flash-based SSDs. Our experiments show 1.15× higher throughput and 27.2% lower latency than the conventional techniques in the small random workload. The proposed technique also passes the certification test with 1.8 to 4.9× higher load in Aerospike Certification Tool evaluations.
低延迟NAND闪存中被忽视的擦除操作性能相对增加了垃圾收集干扰的影响，当考虑使用存储类内存应用程序时，这是一个挑战。传统的擦除相关技术不能充分隐藏擦除操作的影响，并导致吞吐量和延迟之间的性能权衡。本文提出了一种由多块擦除技术和自适应擦除悬浮技术组成的boost and stop erase (BOOSTER)技术。该技术提高了基于NAND闪存的低延迟ssd的吞吐量和延迟。我们的实验表明，在小的随机工作负载下，吞吐量比传统技术提高了1.15倍，延迟降低了27.2%。该技术还通过了Aerospike认证工具评估的认证测试，负载提高了1.8到4.9倍。
{"title":"BOOSTER: Rethinking the erase operation of low-latency SSDs to achieve high throughput and less long latency","authors":"Takumi Fujimori, Shuou Nomura","doi":"10.1145/3579370.3594774","DOIUrl":"https://doi.org/10.1145/3579370.3594774","url":null,"abstract":"The disregarded performance of erase operation in low-latency NAND flash memory relatively increases the impact of garbage collection interferences, which is a challenge when considering the use for storage class memory applications. Conventional erase-related techniques do not sufficiently conceal the impact of erase operation and incur a performance trade-off between throughput and latency. In this paper, we propose a boost and stop erase (BOOSTER) technique consisting of a multi-block erase technique and an adaptive erase suspension technique. The proposed technique improves both throughput and latency of low-latency NAND flash-based SSDs. Our experiments show 1.15× higher throughput and 27.2% lower latency than the conventional techniques in the small random workload. The proposed technique also passes the certification test with 1.8 to 4.9× higher load in Aerospike Certification Tool evaluations.","PeriodicalId":180024,"journal":{"name":"Proceedings of the 16th ACM International Conference on Systems and Storage","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130479325","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Proceedings of the 16th ACM International Conference on Systems and Storage","authors":"","doi":"10.1145/3579370","DOIUrl":"https://doi.org/10.1145/3579370","url":null,"abstract":"","PeriodicalId":180024,"journal":{"name":"Proceedings of the 16th ACM International Conference on Systems and Storage","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128918480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：