Performance of parallel I/O is critical for large-scale scientific applications to store and access data from parallel file systems on high-performance computing (HPC) systems. These applications use HPC systems often to generate and analyze large amounts of data. They use the parallel I/O software stack for accessing and retrieving data. This stack includes several layers of software libraries - high-level I/O libraries such as HDF5, middleware (MPI-IO), and low-level I/O libraries (POSIX, STD-IO). Each of these layers have complex inter-dependencies among them that impact the I/O performance significantly. As a result, scientific applications frequently spend a large fraction of their execution time in reading and writing data on parallel file systems. These inter-dependencies also complicate tuning parallel I/O performance. A typical parallel I/O performance tuning approach includes collecting performance logs or traces, identifying performance bottlenecks, attributing root causes, and devising optimization strategies. Toward this systematic process, we have done research in collecting Darshan traces for I/O, studying logs on production supercomputing systems, attributing root cause analysis by zooming into application I/O performance, visualizing parallel I/O performance, and applying performance tuning. We will introduce parallel I/O basics, I/O monitoring using various profiling tools, analysis of logs collected on production class supercomputers to identify performance bottlenecks, and application of performance tuning options.We will also describe numerous application use cases and performance improvements.
{"title":"Understanding Parallel I/O Performance and Tuning","authors":"S. Byna","doi":"10.1145/3526064.3534114","DOIUrl":"https://doi.org/10.1145/3526064.3534114","url":null,"abstract":"Performance of parallel I/O is critical for large-scale scientific applications to store and access data from parallel file systems on high-performance computing (HPC) systems. These applications use HPC systems often to generate and analyze large amounts of data. They use the parallel I/O software stack for accessing and retrieving data. This stack includes several layers of software libraries - high-level I/O libraries such as HDF5, middleware (MPI-IO), and low-level I/O libraries (POSIX, STD-IO). Each of these layers have complex inter-dependencies among them that impact the I/O performance significantly. As a result, scientific applications frequently spend a large fraction of their execution time in reading and writing data on parallel file systems. These inter-dependencies also complicate tuning parallel I/O performance. A typical parallel I/O performance tuning approach includes collecting performance logs or traces, identifying performance bottlenecks, attributing root causes, and devising optimization strategies. Toward this systematic process, we have done research in collecting Darshan traces for I/O, studying logs on production supercomputing systems, attributing root cause analysis by zooming into application I/O performance, visualizing parallel I/O performance, and applying performance tuning. We will introduce parallel I/O basics, I/O monitoring using various profiling tools, analysis of logs collected on production class supercomputers to identify performance bottlenecks, and application of performance tuning options.We will also describe numerous application use cases and performance improvements.","PeriodicalId":183096,"journal":{"name":"Fifth International Workshop on Systems and Network Telemetry and Analytics","volume":"79 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125052748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ning Li, Xin Yuan, José-Fernán Martínez, Zhaoxin Zhang
In wireless scenarios, the multi-attribute decision-making (MADM) algorithm has been widely used. It can address the multi-objective decision-making issues effectively. However, considering the data flow in wireless network is high-dynamic, continuous, and large-scale, the traditional MADM algorithms are not accurate anymore and the computational complexity is extremely high. To address this problem, in this paper, we propose the variable-weight MADM (vw-MADM) algorithm, which is simple but more effective than previous works. In vw-MADM, when one of the parameters changes, different from the traditional MADM algorithm, only the utility of this parameter needs to be recalculated, the utilities of other candidates are not affected. Based on this innovation, the accuracy is improved while the computational complexity is reduced. Moreover, we also prove the correctness of vw-MADM algorithm, i.e., it is reasonable and effective. Finally, we analyze the computational complexity of both vw-MADM algorithm and traditional MADM algorithm. All the conclusions demonstrate that the proposed vw-MADM algorithm has better performance than the traditional MADM algorithm on accuracy and complexity.
{"title":"The Variable-Weight MADM Algorithm for Wireless Network","authors":"Ning Li, Xin Yuan, José-Fernán Martínez, Zhaoxin Zhang","doi":"10.1145/3526064.3534115","DOIUrl":"https://doi.org/10.1145/3526064.3534115","url":null,"abstract":"In wireless scenarios, the multi-attribute decision-making (MADM) algorithm has been widely used. It can address the multi-objective decision-making issues effectively. However, considering the data flow in wireless network is high-dynamic, continuous, and large-scale, the traditional MADM algorithms are not accurate anymore and the computational complexity is extremely high. To address this problem, in this paper, we propose the variable-weight MADM (vw-MADM) algorithm, which is simple but more effective than previous works. In vw-MADM, when one of the parameters changes, different from the traditional MADM algorithm, only the utility of this parameter needs to be recalculated, the utilities of other candidates are not affected. Based on this innovation, the accuracy is improved while the computational complexity is reduced. Moreover, we also prove the correctness of vw-MADM algorithm, i.e., it is reasonable and effective. Finally, we analyze the computational complexity of both vw-MADM algorithm and traditional MADM algorithm. All the conclusions demonstrate that the proposed vw-MADM algorithm has better performance than the traditional MADM algorithm on accuracy and complexity.","PeriodicalId":183096,"journal":{"name":"Fifth International Workshop on Systems and Network Telemetry and Analytics","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130538950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Session details: Technical Session 1","authors":"Jerry Chou","doi":"10.1145/3545158","DOIUrl":"https://doi.org/10.1145/3545158","url":null,"abstract":"","PeriodicalId":183096,"journal":{"name":"Fifth International Workshop on Systems and Network Telemetry and Analytics","volume":"108 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126937348","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Session details: Keynote","authors":"Jinoh Kim","doi":"10.1145/3545157","DOIUrl":"https://doi.org/10.1145/3545157","url":null,"abstract":"","PeriodicalId":183096,"journal":{"name":"Fifth International Workshop on Systems and Network Telemetry and Analytics","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129358680","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ghazanfar Ali, Jon R. Hass, A. Sill, E. Hojati, Tommy Dang, Yong Chen
Current monitoring tools for high-performance computing (HPC) systems are often inefficient in terms of scalability and interfacing with modern data center management APIs. This inefficiency leads to a lack of effective management of infrastructure of modern data centers. Nagios is one of the widely used industry-standard tools for data center infrastructure monitoring, which mainly include monitoring of nodes and associated hardware and software components. However, current Nagios monitoring has special requirements that introduce several limitations. First, a significant human effort is needed for the configuration of monitored nodes in the Nagios server. Second, the Nagios Remote Plugin Executor and the Nagios Service Check Acceptor are required on the Nagios server and each monitored node for active and passive monitoring, respectively. Third, Nagios monitoring also requires monitoring-specific agents on each monitored node. These shortcomings are inherently due to Nagios' in-band implementation nature. To overcome these limitations, we introduced Redfish-Nagios, a scalable out-of-band monitoring tool for modern HPC systems. It integrates the Nagios server with the out-of-band Distributed Management Task Force's Redfish telemetry model, which is implemented in the baseboard management controller of the nodes. This integration eliminates the requirements of any agent, plugin, hardware component, or configuration on the monitored nodes. It is potentially a paradigm shift in Nagios-based monitoring for two reasons. First, it simplifies communication between the Nagios server and monitored nodes. Second, it saves the computational cost by removing the requirements of running complex Nagios-native protocols and agents on the monitored nodes. The Redfish-Nagios integration methodology enables monitoring of next-generation HPC systems using the scalable and modern Redfish telemetry model and interface.
{"title":"Redfish-Nagios: A Scalable Out-of-Band Data Center Monitoring Framework Based on Redfish Telemetry Model","authors":"Ghazanfar Ali, Jon R. Hass, A. Sill, E. Hojati, Tommy Dang, Yong Chen","doi":"10.1145/3526064.3534108","DOIUrl":"https://doi.org/10.1145/3526064.3534108","url":null,"abstract":"Current monitoring tools for high-performance computing (HPC) systems are often inefficient in terms of scalability and interfacing with modern data center management APIs. This inefficiency leads to a lack of effective management of infrastructure of modern data centers. Nagios is one of the widely used industry-standard tools for data center infrastructure monitoring, which mainly include monitoring of nodes and associated hardware and software components. However, current Nagios monitoring has special requirements that introduce several limitations. First, a significant human effort is needed for the configuration of monitored nodes in the Nagios server. Second, the Nagios Remote Plugin Executor and the Nagios Service Check Acceptor are required on the Nagios server and each monitored node for active and passive monitoring, respectively. Third, Nagios monitoring also requires monitoring-specific agents on each monitored node. These shortcomings are inherently due to Nagios' in-band implementation nature. To overcome these limitations, we introduced Redfish-Nagios, a scalable out-of-band monitoring tool for modern HPC systems. It integrates the Nagios server with the out-of-band Distributed Management Task Force's Redfish telemetry model, which is implemented in the baseboard management controller of the nodes. This integration eliminates the requirements of any agent, plugin, hardware component, or configuration on the monitored nodes. It is potentially a paradigm shift in Nagios-based monitoring for two reasons. First, it simplifies communication between the Nagios server and monitored nodes. Second, it saves the computational cost by removing the requirements of running complex Nagios-native protocols and agents on the monitored nodes. The Redfish-Nagios integration methodology enables monitoring of next-generation HPC systems using the scalable and modern Redfish telemetry model and interface.","PeriodicalId":183096,"journal":{"name":"Fifth International Workshop on Systems and Network Telemetry and Analytics","volume":"140 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115560226","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Data access throughput is one of the key performance metrics in scientific computing, particularly for distributed data-intensive applications. While there has been a body of studies focusing on elephant connections that consume a significant fraction of network bandwidth, this study focuses on predicting slow connections that create bottlenecks in distributed workflows. In this study, we analyze network traffic logs collected between January 2019 and May 2021 at National Energy Research Scientific Computing Center (NERSC). Based on the observed patterns from this data collection, we define a set of features to be used for identifying low-performing data transfers. Through extensive feature engineering and feature selection, we identify a number of new features to significantly enhance the prediction performance. With these new features, even the relatively simple decision tree model could predict slow connections with a F1 score as high as 0.945.
{"title":"Predicting Slow Network Transfers in Scientific Computing","authors":"Robin Shao, Jinoh Kim, A. Sim, K. Wu","doi":"10.1145/3526064.3534112","DOIUrl":"https://doi.org/10.1145/3526064.3534112","url":null,"abstract":"Data access throughput is one of the key performance metrics in scientific computing, particularly for distributed data-intensive applications. While there has been a body of studies focusing on elephant connections that consume a significant fraction of network bandwidth, this study focuses on predicting slow connections that create bottlenecks in distributed workflows. In this study, we analyze network traffic logs collected between January 2019 and May 2021 at National Energy Research Scientific Computing Center (NERSC). Based on the observed patterns from this data collection, we define a set of features to be used for identifying low-performing data transfers. Through extensive feature engineering and feature selection, we identify a number of new features to significantly enhance the prediction performance. With these new features, even the relatively simple decision tree model could predict slow connections with a F1 score as high as 0.945.","PeriodicalId":183096,"journal":{"name":"Fifth International Workshop on Systems and Network Telemetry and Analytics","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115341815","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Session details: Technical Session 2","authors":"M. Cafaro","doi":"10.1145/3545159","DOIUrl":"https://doi.org/10.1145/3545159","url":null,"abstract":"","PeriodicalId":183096,"journal":{"name":"Fifth International Workshop on Systems and Network Telemetry and Analytics","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115988297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We introduce Janus as a framework designed around exposing container configuration and tuning for high-performance data mover applications. A goal is to minimize complexity and offer a path to rapid service deployment on data transfer node hardware. In this preliminary work, we briefly describe the Janus architecture and use a 100Gbps network testbed deployment to perform a series of disk-to-disk transfers. The results show that Janus provides an optimized container environment for networked applications, and that configuration choices may have a significant impact on the overall observed transfer rate and CPU utilization.
{"title":"Janus","authors":"E. Kissel","doi":"10.1145/3526064.3534113","DOIUrl":"https://doi.org/10.1145/3526064.3534113","url":null,"abstract":"We introduce Janus as a framework designed around exposing container configuration and tuning for high-performance data mover applications. A goal is to minimize complexity and offer a path to rapid service deployment on data transfer node hardware. In this preliminary work, we briefly describe the Janus architecture and use a 100Gbps network testbed deployment to perform a series of disk-to-disk transfers. The results show that Janus provides an optimized container environment for networked applications, and that configuration choices may have a significant impact on the overall observed transfer rate and CPU utilization.","PeriodicalId":183096,"journal":{"name":"Fifth International Workshop on Systems and Network Telemetry and Analytics","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115762757","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Eric Chan-Tin, Loretta J. Stalans, Spencer Johnston, D. Reyes, S. Kennison
Phishing is a common vector for cybercrime and hacking. This research examines participants' personality styles (e.g. decision-making styles, self-control) and the likelihood of falling victim to phishing attacks. Over 300 participants completed an online survey assessing protective and vulnerable strategies, personality styles, trust in people, prior victimization from catphishing or identity theft, and demographics information. Unbeknownst to the participants, 2 to 4 weeks after completing the survey they received a phishing e-mail asking them to click on a link. Individuals with a stronger systematic decision-making style were more likely to have a greater number of protective strategies, and those with greater protective strategies were less likely to be a victim of catphishing and identity theft. Individuals with low avoidant decision-making styles and prior vulnerable strategies were more likely to be phished. These findings suggest that learning protective strategies and not using vulnerable strategies are insufficient to lower substantially the risk of being phished. Training might be improved through considering the match between decision-making styles and the content of the training.
{"title":"Predicting Phishing Victimization: Roles of Protective and Vulnerable Strategies and Decision-Making Styles","authors":"Eric Chan-Tin, Loretta J. Stalans, Spencer Johnston, D. Reyes, S. Kennison","doi":"10.1145/3526064.3534107","DOIUrl":"https://doi.org/10.1145/3526064.3534107","url":null,"abstract":"Phishing is a common vector for cybercrime and hacking. This research examines participants' personality styles (e.g. decision-making styles, self-control) and the likelihood of falling victim to phishing attacks. Over 300 participants completed an online survey assessing protective and vulnerable strategies, personality styles, trust in people, prior victimization from catphishing or identity theft, and demographics information. Unbeknownst to the participants, 2 to 4 weeks after completing the survey they received a phishing e-mail asking them to click on a link. Individuals with a stronger systematic decision-making style were more likely to have a greater number of protective strategies, and those with greater protective strategies were less likely to be a victim of catphishing and identity theft. Individuals with low avoidant decision-making styles and prior vulnerable strategies were more likely to be phished. These findings suggest that learning protective strategies and not using vulnerable strategies are insufficient to lower substantially the risk of being phished. Training might be improved through considering the match between decision-making styles and the content of the training.","PeriodicalId":183096,"journal":{"name":"Fifth International Workshop on Systems and Network Telemetry and Analytics","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123161342","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ruize Han, A. Sim, K. Wu, I. Monga, C. Guok, F. Würthwein, Diego Davila, J. Balcas, Harvey Newman
Scientific collaborations are increasingly relying on large volumes of data for their work and many of them employ tiered systems to replicate the data to their worldwide user communities. Each user in the community often selects a different subset of data for their analysis tasks; however, members of a research group often are working on related research topics that require similar data objects. Thus, there is a significant amount of data sharing possible. In this work, we study the access traces of a federated storage cache known as the Southern California Petabyte Scale Cache. By studying the access patterns and potential for network traffic reduction by this caching system, we aim to explore the predictability of the cache uses and the potential for a more general in-network data caching. Our study shows that this distributed storage cache is able to reduce the network traffic volume by a factor of 2.35 during a part of the study period. We further show that machine learning models could predict cache utilization with an accuracy of 0.88. This demonstrates that such cache usage is predictable, which could be useful for managing complex networking resources such as in-network caching.
{"title":"Access Trends of In-network Cache for Scientific Data","authors":"Ruize Han, A. Sim, K. Wu, I. Monga, C. Guok, F. Würthwein, Diego Davila, J. Balcas, Harvey Newman","doi":"10.1145/3526064.3534110","DOIUrl":"https://doi.org/10.1145/3526064.3534110","url":null,"abstract":"Scientific collaborations are increasingly relying on large volumes of data for their work and many of them employ tiered systems to replicate the data to their worldwide user communities. Each user in the community often selects a different subset of data for their analysis tasks; however, members of a research group often are working on related research topics that require similar data objects. Thus, there is a significant amount of data sharing possible. In this work, we study the access traces of a federated storage cache known as the Southern California Petabyte Scale Cache. By studying the access patterns and potential for network traffic reduction by this caching system, we aim to explore the predictability of the cache uses and the potential for a more general in-network data caching. Our study shows that this distributed storage cache is able to reduce the network traffic volume by a factor of 2.35 during a part of the study period. We further show that machine learning models could predict cache utilization with an accuracy of 0.88. This demonstrates that such cache usage is predictable, which could be useful for managing complex networking resources such as in-network caching.","PeriodicalId":183096,"journal":{"name":"Fifth International Workshop on Systems and Network Telemetry and Analytics","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125406155","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：