BenchCouncil Transactions on Benchmarks, Standards and Evaluations最新文献

英文中文

Benchmarking feature selection methods with different prediction models on large-scale healthcare event data 对大规模医疗事件数据上不同预测模型的特征选择方法进行基准测试

BenchCouncil Transactions on Benchmarks, Standards and Evaluations

Pub Date : 2021-10-01 DOI: 10.1016/j.tbench.2021.100004

Fan Zhang , Chunjie Luo , Chuanxin Lan , Jianfeng Zhan

With the development of the Electronic Health Record (EHR) technique, vast volumes of digital clinical data are generated. Based on the data, many methods are developed to improve the performance of clinical predictions. Among those methods, Deep Neural Networks (DNN) have been proven outstanding with respect to accuracy by employing many patient instances and events (features). However, each patient-specific event requires time and money. Collecting too many features before making a decision is insufferable, especially for time-critical tasks such as mortality prediction. So it is essential to predict with high accuracy using as minimal clinical events as possible, which makes feature selection a critical question. This paper presents detailed benchmarking results of various feature selection methods, applying different classification and regression algorithms for clinical prediction tasks, including mortality prediction, length of stay prediction, and ICD-9 code group prediction. We use the publicly available dataset, Medical Information Mart for Intensive Care III (MIMIC-III), in our experiments. Our results show that Genetic Algorithm (GA) based methods perform well with only a few features and outperform others. Besides, for the mortality prediction task, the feature subset selected by GA for one classifier can also be used to others while achieving good performance.

随着电子病历(EHR)技术的发展，产生了大量的数字化临床数据。基于这些数据，开发了许多方法来提高临床预测的性能。在这些方法中，深度神经网络(DNN)通过使用许多患者实例和事件(特征)来证明其准确性。然而，每个针对患者的事件都需要时间和金钱。在做出决定之前收集太多的特征是令人难以忍受的，特别是对于时间紧迫的任务，如死亡率预测。因此，使用尽可能少的临床事件进行高精度预测是至关重要的，这使得特征选择成为一个关键问题。本文详细介绍了各种特征选择方法的基准测试结果，将不同的分类和回归算法应用于临床预测任务，包括死亡率预测、住院时间预测和ICD-9代码组预测。我们在实验中使用了公开可用的数据集，重症监护医疗信息市场III (MIMIC-III)。我们的研究结果表明，基于遗传算法(GA)的方法仅在少数特征上表现良好，并且优于其他方法。此外，对于死亡率预测任务，遗传算法为一个分类器选择的特征子集也可以用于其他分类器，同时获得良好的性能。

{"title":"Benchmarking feature selection methods with different prediction models on large-scale healthcare event data","authors":"Fan Zhang , Chunjie Luo , Chuanxin Lan , Jianfeng Zhan","doi":"10.1016/j.tbench.2021.100004","DOIUrl":"10.1016/j.tbench.2021.100004","url":null,"abstract":"<div><p>With the development of the Electronic Health Record (EHR) technique, vast volumes of digital clinical data are generated. Based on the data, many methods are developed to improve the performance of clinical predictions. Among those methods, Deep Neural Networks (DNN) have been proven outstanding with respect to accuracy by employing many patient instances and events (features). However, each patient-specific event requires time and money. Collecting too many features before making a decision is insufferable, especially for time-critical tasks such as mortality prediction. So it is essential to predict with high accuracy using as minimal clinical events as possible, which makes feature selection a critical question. This paper presents detailed benchmarking results of various feature selection methods, applying different classification and regression algorithms for clinical prediction tasks, including mortality prediction, length of stay prediction, and ICD-9 code group prediction. We use the publicly available dataset, Medical Information Mart for Intensive Care III (MIMIC-III), in our experiments. Our results show that Genetic Algorithm (GA) based methods perform well with only a few features and outperform others. Besides, for the mortality prediction task, the feature subset selected by GA for one classifier can also be used to others while achieving good performance.</p></div>","PeriodicalId":100155,"journal":{"name":"BenchCouncil Transactions on Benchmarks, Standards and Evaluations","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2772485921000041/pdfft?md5=99fde57d63abff83585ab50c968fd9b0&pid=1-s2.0-S2772485921000041-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81823992","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Workflow Critical Path: A data-oriented critical path metric for Holistic HPC Workflows 工作流关键路径:面向数据的整体HPC工作流关键路径度量

BenchCouncil Transactions on Benchmarks, Standards and Evaluations

Pub Date : 2021-10-01 DOI: 10.15760/etd.7369

Daniel D. Nguyen, K. Karavanic

............................................................................................................................... i List of Tables ...................................................................................................................... iv List of Figures ..................................................................................................................... v Chapter 1: Introduction ....................................................................................................... 1 1.1 Motivation ................................................................................................................. 3 1.2 Definitions ................................................................................................................. 5 1.3 Thesis Statement ....................................................................................................... 6 1.4 Contributions ............................................................................................................. 6 Chapter 2: Background ........................................................................................................ 9 2.1 Parallel Computing .................................................................................................... 9 2.2 Critical Path Analysis ................................................................................................ 9 2.3 High Performance Computing ................................................................................ 11 2.4 Holistic HPC Workflows ........................................................................................ 12 2.5 Instrumentation, Profiling, and Tracing .................................................................. 12 Chapter 3: Related Work ................................................................................................... 14 3.1 Workflow Management Systems ............................................................................ 14 3.2 Distributed Systems Tracing Tools ......................................................................... 17 3.3 HPC Performance Measurement Tools ................................................................... 20 3.4 Performance Analysis of Scientific Workflows ...................................................... 21 Chapter 4: Architecture ..................................................................................................... 22 4.1 Data State ................................................................................................................ 22 4.2 Crux UI .................................................................................................................... 24 4.3 Crux API ................................................................................................................. 24 4.4 Crux Database ......................................................................................................... 27

...............................................................................................................................我的表列表 ......................................................................................................................第四列数据 .....................................................................................................................第五章1:介绍 .......................................................................................................1 1.1动机 .................................................................................................................3 1.2定义 .................................................................................................................5 1.3论文声明 .......................................................................................................6 1.4贡献 .............................................................................................................6第二章:背景 ........................................................................................................9 2.1并行计算 ....................................................................................................9 2.2关键路径分析 ................................................................................................9 2.3高性能计算 ................................................................................11 2.4整体HPC工作流 ........................................................................................12个2.5仪器、分析和跟踪 ..................................................................12章3:相关工作 ...................................................................................................14个3.1工作流管理系统 ............................................................................14个3.2分布式系统跟踪工具 .........................................................................17 3.3 HPC性能测量工具 ...................................................................20 3.4科学工作流的性能分析 ......................................................第四章:体系结构 .....................................................................................................22 4.1数据状态 ................................................................................................................22 4.2关键用户界面 ....................................................................................................................24个4.3关键API .................................................................................................................24个4.4关键数据库 .........................................................................................................27 4.5关键关键路径算法 ..................................................................................29日4.6部署在HPC集群 ...................................................................................31日4.7 HPC应用程序的工具 .....................................................................33

{"title":"Workflow Critical Path: A data-oriented critical path metric for Holistic HPC Workflows","authors":"Daniel D. Nguyen, K. Karavanic","doi":"10.15760/etd.7369","DOIUrl":"https://doi.org/10.15760/etd.7369","url":null,"abstract":"............................................................................................................................... i List of Tables ...................................................................................................................... iv List of Figures ..................................................................................................................... v Chapter 1: Introduction ....................................................................................................... 1 1.1 Motivation ................................................................................................................. 3 1.2 Definitions ................................................................................................................. 5 1.3 Thesis Statement ....................................................................................................... 6 1.4 Contributions ............................................................................................................. 6 Chapter 2: Background ........................................................................................................ 9 2.1 Parallel Computing .................................................................................................... 9 2.2 Critical Path Analysis ................................................................................................ 9 2.3 High Performance Computing ................................................................................ 11 2.4 Holistic HPC Workflows ........................................................................................ 12 2.5 Instrumentation, Profiling, and Tracing .................................................................. 12 Chapter 3: Related Work ................................................................................................... 14 3.1 Workflow Management Systems ............................................................................ 14 3.2 Distributed Systems Tracing Tools ......................................................................... 17 3.3 HPC Performance Measurement Tools ................................................................... 20 3.4 Performance Analysis of Scientific Workflows ...................................................... 21 Chapter 4: Architecture ..................................................................................................... 22 4.1 Data State ................................................................................................................ 22 4.2 Crux UI .................................................................................................................... 24 4.3 Crux API ................................................................................................................. 24 4.4 Crux Database ......................................................................................................... 27","PeriodicalId":100155,"journal":{"name":"BenchCouncil Transactions on Benchmarks, Standards and Evaluations","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77251801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Revisiting the effects of the Spectre and Meltdown patches using the top-down microarchitectural method and purchasing power parity theory 使用自上而下的微架构方法和购买力平价理论重新审视Spectre和Meltdown补丁的影响

BenchCouncil Transactions on Benchmarks, Standards and Evaluations

Pub Date : 2021-10-01 DOI: 10.1016/j.tbench.2021.100011

Yectli A. Huerta , David J. Lilja

Software patches are made available to fix security vulnerabilities, enhance performance, and usability. Previous works focused on measuring the performance effect of patches on benchmark runtimes. In this study, we used the Top-Down microarchitecture analysis method to understand how pipeline bottlenecks were affected by the application of the Spectre and Meltdown security patches. Bottleneck analysis makes it possible to better understand how different hardware resources are being utilized, highlighting portions of the pipeline where possible improvements could be achieved. We complement the Top-Down analysis technique with the use a normalization technique from the field of economics, purchasing power parity (PPP), to better understand the relative difference between patched and unpatched runs. In this study, we showed that security patches had an effect that was reflected on the corresponding Top-Down metrics. We showed that recent compilers are not as negatively affected as previously reported. Out of the 14 benchmarks that make up the SPEC OMP2012 suite, three had noticeable slowdowns when the patches were applied. We also found that Top-Down metrics had large relative differences when the security patches were applied, differences that standard techniques based in absolute, non-normalized, metrics failed to highlight.

提供软件补丁以修复安全漏洞、增强性能和可用性。以前的工作主要集中在衡量补丁对基准运行时的性能影响。在本研究中，我们使用自顶向下的微架构分析方法来了解Spectre和Meltdown安全补丁的应用如何影响管道瓶颈。瓶颈分析可以更好地理解不同的硬件资源是如何被利用的，突出显示可以实现改进的管道部分。我们通过使用经济学领域的标准化技术，即购买力平价(PPP)来补充自上而下的分析技术，以更好地理解修补和未修补运行之间的相对差异。在这项研究中，我们展示了安全补丁具有反映在相应的自上而下度量标准上的效果。我们表明，最近的编译器并没有像之前报道的那样受到负面影响。在构成SPEC OMP2012套件的14个基准测试中，有3个在应用补丁时出现了明显的减速。我们还发现，当应用安全补丁时，自上而下的度量具有较大的相对差异，基于绝对的、非规范化的度量的标准技术无法突出这些差异。

{"title":"Revisiting the effects of the Spectre and Meltdown patches using the top-down microarchitectural method and purchasing power parity theory","authors":"Yectli A. Huerta , David J. Lilja","doi":"10.1016/j.tbench.2021.100011","DOIUrl":"10.1016/j.tbench.2021.100011","url":null,"abstract":"<div><p>Software patches are made available to fix security vulnerabilities, enhance performance, and usability. Previous works focused on measuring the performance effect of patches on benchmark runtimes. In this study, we used the Top-Down microarchitecture analysis method to understand how pipeline bottlenecks were affected by the application of the Spectre and Meltdown security patches. Bottleneck analysis makes it possible to better understand how different hardware resources are being utilized, highlighting portions of the pipeline where possible improvements could be achieved. We complement the Top-Down analysis technique with the use a normalization technique from the field of economics, purchasing power parity (PPP), to better understand the relative difference between patched and unpatched runs. In this study, we showed that security patches had an effect that was reflected on the corresponding Top-Down metrics. We showed that recent compilers are not as negatively affected as previously reported. Out of the 14 benchmarks that make up the SPEC OMP2012 suite, three had noticeable slowdowns when the patches were applied. We also found that Top-Down metrics had large relative differences when the security patches were applied, differences that standard techniques based in absolute, non-normalized, metrics failed to highlight.</p></div>","PeriodicalId":100155,"journal":{"name":"BenchCouncil Transactions on Benchmarks, Standards and Evaluations","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2772485921000119/pdfft?md5=6e1139ebac69c084c6fe482b82fdd42c&pid=1-s2.0-S2772485921000119-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91428938","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

首页上一页

类型

全部化学•材料生命科学医学物理工程技术环境•农林材料科学地球科学法学管理学化学环境科学与生态学计算机科学教育学经济学农林科学人文科学生物学数学物理与天体物理心理学综合性期刊其他工业工程理学历史学农学文学信息工程

数据库

全部 ACS Publications Elsevier ieeexplore Springer The Royal Society of Chemistry Wiley

期刊

BenchCouncil Transactions on Benchmarks, Standards and Evaluations

全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.

﹀