Pub Date : 2021-10-01DOI: 10.1109/LDAV53230.2021.00017
D. Semeraro, Leigh Orf
Lossy compression is a data compression technique that sacrifices precision for the sake of higher compression rates. While loss of precision is unacceptable when storing simulation data for check pointing, it has little discernable impact on visualization. Saving simulation output for later examination is still a prevalent workflow. Domain scientists often return to data from older runs to examine the data in a new context. Storage of visualization data at full precision is not necessary for this purpose. The use of lossy compression can therefore relieve the pressure on HPC storage equipment or be used to store data at higher temporal resolution than without compression. In this poster we show how lossy compression was used to store visualization data for the analysis of a supercell thunderstorm. The visual results will be shown as well as details of how the compression was used in the workflow.
{"title":"Lossy Compression for Visualization of Atmospheric Data","authors":"D. Semeraro, Leigh Orf","doi":"10.1109/LDAV53230.2021.00017","DOIUrl":"https://doi.org/10.1109/LDAV53230.2021.00017","url":null,"abstract":"Lossy compression is a data compression technique that sacrifices precision for the sake of higher compression rates. While loss of precision is unacceptable when storing simulation data for check pointing, it has little discernable impact on visualization. Saving simulation output for later examination is still a prevalent workflow. Domain scientists often return to data from older runs to examine the data in a new context. Storage of visualization data at full precision is not necessary for this purpose. The use of lossy compression can therefore relieve the pressure on HPC storage equipment or be used to store data at higher temporal resolution than without compression. In this poster we show how lossy compression was used to store visualization data for the analysis of a supercell thunderstorm. The visual results will be shown as well as details of how the compression was used in the workflow.","PeriodicalId":441438,"journal":{"name":"2021 IEEE 11th Symposium on Large Data Analysis and Visualization (LDAV)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121412084","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/LDAV53230.2021.00009
D. Morozov, T. Peterka, Hanqi Guo, Mukund Raj, Jiayi Xu, Han-Wei Shen
Iterative parallel algorithms can be implemented by synchronizing after each round. This bulk-synchronous parallel (BSP) pattern is inefficient when strict synchronization is not required: global synchronization is costly at scale and prohibits amortizing load imbalance over the entire execution, and termination detection is challenging with irregular data-dependent communication. We present an asynchronous communication protocol that efficiently interleaves communication with computation. The protocol includes global termination detection without obstructing computation and communication between nodes. The user's computational primitive only needs to indicate when local work is done; our algorithm detects when all processors reach this state. We do not assume that global work decreases monotonically, allowing processors to createnew work. We illustrate the utility of our solution through experiments, including two large data analysis and visualization codes: parallel particle advection and distributed union-find. Our asynchronous algorithm is several times faster with better strong scaling efficiency than the synchronous approach.
{"title":"IExchange: Asynchronous Communication and Termination Detection for Iterative Algorithms","authors":"D. Morozov, T. Peterka, Hanqi Guo, Mukund Raj, Jiayi Xu, Han-Wei Shen","doi":"10.1109/LDAV53230.2021.00009","DOIUrl":"https://doi.org/10.1109/LDAV53230.2021.00009","url":null,"abstract":"Iterative parallel algorithms can be implemented by synchronizing after each round. This bulk-synchronous parallel (BSP) pattern is inefficient when strict synchronization is not required: global synchronization is costly at scale and prohibits amortizing load imbalance over the entire execution, and termination detection is challenging with irregular data-dependent communication. We present an asynchronous communication protocol that efficiently interleaves communication with computation. The protocol includes global termination detection without obstructing computation and communication between nodes. The user's computational primitive only needs to indicate when local work is done; our algorithm detects when all processors reach this state. We do not assume that global work decreases monotonically, allowing processors to createnew work. We illustrate the utility of our solution through experiments, including two large data analysis and visualization codes: parallel particle advection and distributed union-find. Our asynchronous algorithm is several times faster with better strong scaling efficiency than the synchronous approach.","PeriodicalId":441438,"journal":{"name":"2021 IEEE 11th Symposium on Large Data Analysis and Visualization (LDAV)","volume":"104 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114613207","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/LDAV53230.2021.00015
Nicole Marsaglia, Yuya Kawakami, Samuel D. Schwartz, Stefan Fields, H. Childs
Viewpoint Quality (VQ) metrics have the potential to predict human preferences for camera placement. With this study, we introduce new VQ metrics that incorporate entropy, and explore how they can be used in combination. Our evaluation involves three phases: (1) creating a database of isosurface imagery from ten large, scientific data sets, (2) conducting a user study with approximately 30 large data visualization experts who provided over 1000 responses, and (3) analyzing how our entropy-based VQ metrics compared with existing VQ metrics in predicting expert preference. In terms of findings, we find that our entropy-based metrics are able to predict expert preferences 68% of the time, while existing VQ metrics perform much worse (52%). This finding, while valuable on its own, also opens the door for future work on in situ camera placement. Finally, as another important contribution, this work has the most extensive evaluation to date of existing VQ metrics to predict expert preference for visualizations of large, scientific data sets.
{"title":"An Entropy-Based Approach for Identifying User-Preferred Camera Positions","authors":"Nicole Marsaglia, Yuya Kawakami, Samuel D. Schwartz, Stefan Fields, H. Childs","doi":"10.1109/LDAV53230.2021.00015","DOIUrl":"https://doi.org/10.1109/LDAV53230.2021.00015","url":null,"abstract":"Viewpoint Quality (VQ) metrics have the potential to predict human preferences for camera placement. With this study, we introduce new VQ metrics that incorporate entropy, and explore how they can be used in combination. Our evaluation involves three phases: (1) creating a database of isosurface imagery from ten large, scientific data sets, (2) conducting a user study with approximately 30 large data visualization experts who provided over 1000 responses, and (3) analyzing how our entropy-based VQ metrics compared with existing VQ metrics in predicting expert preference. In terms of findings, we find that our entropy-based metrics are able to predict expert preferences 68% of the time, while existing VQ metrics perform much worse (52%). This finding, while valuable on its own, also opens the door for future work on in situ camera placement. Finally, as another important contribution, this work has the most extensive evaluation to date of existing VQ metrics to predict expert preference for visualizations of large, scientific data sets.","PeriodicalId":441438,"journal":{"name":"2021 IEEE 11th Symposium on Large Data Analysis and Visualization (LDAV)","volume":"85 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116107364","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/LDAV53230.2021.00010
Matthew Larsen, Lawrence Livermore, C. Harrison, Terece L. Turton, S. Sane, S. Brink, H. Childs
Triggers are an emerging strategy for optimizing execution time for in situ analysis. However, their performance characteristics are complex, making it difficult to decide if a particular trigger-based approach is viable. With this study, we propose a cost model for trigger-based in situ analysis that can assess viability, and we also validate the model's efficacy. Then, once the cost model is established, we apply the model to inform the space of viable approaches, considering variation in simulation code, trigger techniques, and analyses, as well as trigger inspection and fire rates. Real-world values are needed both to validate the model and to use the model to inform the space of viable approaches. We obtain these values by surveying science application teams and by performing runs as large as 2,040 GPUs and 32 billion cells.
{"title":"Trigger Happy: Assessing the Viability of Trigger-Based In Situ Analysis","authors":"Matthew Larsen, Lawrence Livermore, C. Harrison, Terece L. Turton, S. Sane, S. Brink, H. Childs","doi":"10.1109/LDAV53230.2021.00010","DOIUrl":"https://doi.org/10.1109/LDAV53230.2021.00010","url":null,"abstract":"Triggers are an emerging strategy for optimizing execution time for in situ analysis. However, their performance characteristics are complex, making it difficult to decide if a particular trigger-based approach is viable. With this study, we propose a cost model for trigger-based in situ analysis that can assess viability, and we also validate the model's efficacy. Then, once the cost model is established, we apply the model to inform the space of viable approaches, considering variation in simulation code, trigger techniques, and analyses, as well as trigger inspection and fire rates. Real-world values are needed both to validate the model and to use the model to inform the space of viable approaches. We obtain these values by surveying science application teams and by performing runs as large as 2,040 GPUs and 32 billion cells.","PeriodicalId":441438,"journal":{"name":"2021 IEEE 11th Symposium on Large Data Analysis and Visualization (LDAV)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128048355","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/LDAV53230.2021.00014
Sergei Shudler, Steve Petruzza, Valerio Pascucci, P. Bremer
Existing data analysis and visualization algorithms are used in a wide range of simulations that strive to support an increasing number of runtime systems. The BabelFlow framework has been designed to address this situation by providing users with a simple interface to implement analysis algorithms as dataflow graphs portable across different runtimes. The limitation in BabelFlow, however, is that the graphs are not easily reusable. Plugging them into existing in situ workflows and constructing more complex graphs is difficult. In this paper, we introduce LegoFlow, an extension to BabelFlow that addresses these challenges. Specifically, we integrate LegoFlow into Ascent, a flyweight framework for large scale in situ analytics, and provide a graph composability mechanism. This mechanism is an intuitive approach to link an arbitrary number of graphs together to create more complex patterns, as well as avoid costly reimple-mentations for minor modifications. Without sacrificing portability, LegoFlow introduces complete flexibility that maximizes the productivity of in situ analytics workflows. Furthermore, we demonstrate a complete LULESH simulation with LegoFlow-based in situ visualization running on top of Charm++. It is a novel approach for in situ analytics, whereby the asynchronous tasking runtime allows routines for computation and analysis to overlap. Finally, we evaluate a number of LegoFlow-based filters and extracts in Ascent, as well as the scaling behavior of a LegoFlow graph for Radix-k based image compositing.
{"title":"Portable and Composable Flow Graphs for In Situ Analytics","authors":"Sergei Shudler, Steve Petruzza, Valerio Pascucci, P. Bremer","doi":"10.1109/LDAV53230.2021.00014","DOIUrl":"https://doi.org/10.1109/LDAV53230.2021.00014","url":null,"abstract":"Existing data analysis and visualization algorithms are used in a wide range of simulations that strive to support an increasing number of runtime systems. The BabelFlow framework has been designed to address this situation by providing users with a simple interface to implement analysis algorithms as dataflow graphs portable across different runtimes. The limitation in BabelFlow, however, is that the graphs are not easily reusable. Plugging them into existing in situ workflows and constructing more complex graphs is difficult. In this paper, we introduce LegoFlow, an extension to BabelFlow that addresses these challenges. Specifically, we integrate LegoFlow into Ascent, a flyweight framework for large scale in situ analytics, and provide a graph composability mechanism. This mechanism is an intuitive approach to link an arbitrary number of graphs together to create more complex patterns, as well as avoid costly reimple-mentations for minor modifications. Without sacrificing portability, LegoFlow introduces complete flexibility that maximizes the productivity of in situ analytics workflows. Furthermore, we demonstrate a complete LULESH simulation with LegoFlow-based in situ visualization running on top of Charm++. It is a novel approach for in situ analytics, whereby the asynchronous tasking runtime allows routines for computation and analysis to overlap. Finally, we evaluate a number of LegoFlow-based filters and extracts in Ascent, as well as the scaling behavior of a LegoFlow graph for Radix-k based image compositing.","PeriodicalId":441438,"journal":{"name":"2021 IEEE 11th Symposium on Large Data Analysis and Visualization (LDAV)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121166702","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/LDAV53230.2021.00011
D. Hoang, H. Bhatia, P. Lindstrom, Valerio Pascucci
Particle representations are used often in large-scale simulations and observations, frequently creating datasets containing several millions of particles or more. Due to their sheer size, such datasets are difficult to store, transfer, and analyze efficiently. Data compression is a promising solution; however, effective approaches to compress particle data are lacking and no community-standard and accepted techniques exist. Current techniques are designed either to compress small data very well but require high computational resources when applied to large data, or to work with large data but without a focus on compression, resulting in low reconstruction quality per bit stored. In this paper, we present innovations targeting tree-based particle compression approaches that improve the tradeoff between high quality and low memory-footprint for compression and decompression of large particle datasets. Inspired by the lazy wavelet transform, we introduce a new way of partitioning space, which allows a low-cost depth-first traversal of a particle hierarchy to cover the space broadly. We also devise novel data-adaptive traversal orders that significantly reduce reconstruction error compared to traditional data-agnostic orders such as breadth-first and depth-first traversals. The new partitioning and traversal schemes are used to build novel particle hierarchies that can be traversed with asymptotically constant memory footprint while incurring low reconstruction error. Our solution to encoding and (lossy) decoding of large particle data is a flexible block-based hierarchy that supports progressive, random-access, and error-driven decoding, where error heuristics can be supplied by the user. Finally, through extensive experimentation, we demonstrate the efficacy and the flexibility of the proposed techniques when combined as well as when used independently with existing approaches on a wide range of scientific particle datasets.
{"title":"High-Quality and Low-Memory-Footprint Progressive Decoding of Large-Scale Particle Data","authors":"D. Hoang, H. Bhatia, P. Lindstrom, Valerio Pascucci","doi":"10.1109/LDAV53230.2021.00011","DOIUrl":"https://doi.org/10.1109/LDAV53230.2021.00011","url":null,"abstract":"Particle representations are used often in large-scale simulations and observations, frequently creating datasets containing several millions of particles or more. Due to their sheer size, such datasets are difficult to store, transfer, and analyze efficiently. Data compression is a promising solution; however, effective approaches to compress particle data are lacking and no community-standard and accepted techniques exist. Current techniques are designed either to compress small data very well but require high computational resources when applied to large data, or to work with large data but without a focus on compression, resulting in low reconstruction quality per bit stored. In this paper, we present innovations targeting tree-based particle compression approaches that improve the tradeoff between high quality and low memory-footprint for compression and decompression of large particle datasets. Inspired by the lazy wavelet transform, we introduce a new way of partitioning space, which allows a low-cost depth-first traversal of a particle hierarchy to cover the space broadly. We also devise novel data-adaptive traversal orders that significantly reduce reconstruction error compared to traditional data-agnostic orders such as breadth-first and depth-first traversals. The new partitioning and traversal schemes are used to build novel particle hierarchies that can be traversed with asymptotically constant memory footprint while incurring low reconstruction error. Our solution to encoding and (lossy) decoding of large particle data is a flexible block-based hierarchy that supports progressive, random-access, and error-driven decoding, where error heuristics can be supplied by the user. Finally, through extensive experimentation, we demonstrate the efficacy and the flexibility of the proposed techniques when combined as well as when used independently with existing approaches on a wide range of scientific particle datasets.","PeriodicalId":441438,"journal":{"name":"2021 IEEE 11th Symposium on Large Data Analysis and Visualization (LDAV)","volume":"192 3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131583200","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/LDAV53230.2021.00018
Anthony Bucaro, Connor Murphy, N. Ferrier, J. Insley, V. Mateevitsi, M. Papka, S. Rizzi, Jifu Tan
Blood flow simulations have important applications in engineering and medicine, requiring visualization and analysis for both fluid (blood plasma) and solid (cells). Recent advances in blood flow simulations highlight the need of a more efficient analysis of large data sets. Traditionally, analysis is performed after a simulation is completed, and any changes of simulation settings require running the simulation again. With bi-directional in situ analysis we aim to solve this problem by allowing manipulation of simulation parameters in run time. In this project, we describe our early steps toward this goal and present the in situ instrumentation of two coupled codes for blood flow simulation using the SENSEI in situ framework.
{"title":"Instrumenting Multiphysics Blood Flow Simulation Codes for In Situ Visualization and Analysis","authors":"Anthony Bucaro, Connor Murphy, N. Ferrier, J. Insley, V. Mateevitsi, M. Papka, S. Rizzi, Jifu Tan","doi":"10.1109/LDAV53230.2021.00018","DOIUrl":"https://doi.org/10.1109/LDAV53230.2021.00018","url":null,"abstract":"Blood flow simulations have important applications in engineering and medicine, requiring visualization and analysis for both fluid (blood plasma) and solid (cells). Recent advances in blood flow simulations highlight the need of a more efficient analysis of large data sets. Traditionally, analysis is performed after a simulation is completed, and any changes of simulation settings require running the simulation again. With bi-directional in situ analysis we aim to solve this problem by allowing manipulation of simulation parameters in run time. In this project, we describe our early steps toward this goal and present the in situ instrumentation of two coupled codes for blood flow simulation using the SENSEI in situ framework.","PeriodicalId":441438,"journal":{"name":"2021 IEEE 11th Symposium on Large Data Analysis and Visualization (LDAV)","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132900831","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/LDAV53230.2021.00016
Nafiul Nipu, Carla Floricel, Negar Naghashzadeh, R. Paoli, G. Marai
Aircraft engines emit particulates that alter the chemical composition of the atmosphere and perturb the Earth's radiation budget by creating additional ice clouds in the form of condensation trails called contrails. We propose a multi-scale visual computing system that will assist in defining contrail features with parameter analysis for computer-generated aircraft engine simulations. These simulations are computationally intensive and rely on high performance computing (HPC) solutions. Our multi-linked visual system seeks to help in the identification of the formation and evolution of contrails and in the identification of contrail-related spatial features from the simulation workflow.
{"title":"Parameter Analysis and Contrail Detection of Aircraft Engine Simulations","authors":"Nafiul Nipu, Carla Floricel, Negar Naghashzadeh, R. Paoli, G. Marai","doi":"10.1109/LDAV53230.2021.00016","DOIUrl":"https://doi.org/10.1109/LDAV53230.2021.00016","url":null,"abstract":"Aircraft engines emit particulates that alter the chemical composition of the atmosphere and perturb the Earth's radiation budget by creating additional ice clouds in the form of condensation trails called contrails. We propose a multi-scale visual computing system that will assist in defining contrail features with parameter analysis for computer-generated aircraft engine simulations. These simulations are computationally intensive and rely on high performance computing (HPC) solutions. Our multi-linked visual system seeks to help in the identification of the formation and evolution of contrails and in the identification of contrail-related spatial features from the simulation workflow.","PeriodicalId":441438,"journal":{"name":"2021 IEEE 11th Symposium on Large Data Analysis and Visualization (LDAV)","volume":"63 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115044988","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/ldav53230.2021.00001
{"title":"2021 IEEE 11th Symposium on Large Data Analysis and Visualization LDAV 2021","authors":"","doi":"10.1109/ldav53230.2021.00001","DOIUrl":"https://doi.org/10.1109/ldav53230.2021.00001","url":null,"abstract":"","PeriodicalId":441438,"journal":{"name":"2021 IEEE 11th Symposium on Large Data Analysis and Visualization (LDAV)","volume":"78 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127771444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/LDAV53230.2021.00020
Pambayun Savira, T. Marrinan, M. Papka
Large-scale scientific simulations typically output massive amounts of data that must be later read in for post-hoc visualization and analysis. With codes simulating complex phenomena at ever-increasing fidelity, writing data to disk during this traditional high-performance computing workflow has become a significant bottleneck. In situ workflows offer a solution to this bottleneck, whereby data is simultaneously produced and analyzed without involving disk storage. In situ analysis can increase efficiency for domain scientists who are exploring a data set or fine-tuning visualization and analysis parameters. Our work seeks to enable researchers to easily create and interactively analyze large-scale simulations through the use of Jupyter Notebooks without requiring application developers to explicitly integrate in situ libraries.
{"title":"Writing, Running, and Analyzing Large-scale Scientific Simulations with Jupyter Notebooks","authors":"Pambayun Savira, T. Marrinan, M. Papka","doi":"10.1109/LDAV53230.2021.00020","DOIUrl":"https://doi.org/10.1109/LDAV53230.2021.00020","url":null,"abstract":"Large-scale scientific simulations typically output massive amounts of data that must be later read in for post-hoc visualization and analysis. With codes simulating complex phenomena at ever-increasing fidelity, writing data to disk during this traditional high-performance computing workflow has become a significant bottleneck. In situ workflows offer a solution to this bottleneck, whereby data is simultaneously produced and analyzed without involving disk storage. In situ analysis can increase efficiency for domain scientists who are exploring a data set or fine-tuning visualization and analysis parameters. Our work seeks to enable researchers to easily create and interactively analyze large-scale simulations through the use of Jupyter Notebooks without requiring application developers to explicitly integrate in situ libraries.","PeriodicalId":441438,"journal":{"name":"2021 IEEE 11th Symposium on Large Data Analysis and Visualization (LDAV)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132706217","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：