首页 > 最新文献

Meta: Avaliacao最新文献

英文 中文
Metamer mismatch volume calculation method based on high-dimensional spherical sampling 基于高维球面采样的Metamer失配体积计算方法
Q4 Social Sciences Pub Date : 2023-08-11 DOI: 10.1117/12.2687942
Kuan Xu, Long Ma, Peng Li
The metamer mismatch volume has important applications in color correction, camera design, and light source design. The method based on spherical sampling to calculate the metamer mismatch volume has a long computation time, a large number of duplicated boundary points, too few effective vertices, and the dimension of its metamer set will appear lower than the theoretical dimension. In this paper, we propose a high-dimensional spherical sampling method that samples the metamer set directly, and find all boundary points by selecting direction vectors and polarizing all directions. The experimental results show that our method improves the above problems, the computational speed is faster, the computational results are close, the repetition rate of boundary points is greatly reduced, and the actual dimensionality of the corresponding metamer set is consistent with the theoretical dimensionality.
超材料失配体积在色彩校正、相机设计和光源设计中有着重要的应用。基于球形采样计算超模型失配体积的方法计算时间长,重复边界点数量多,有效顶点过少,其超模型集的维数会低于理论维数。在本文中,我们提出了一种高维球面采样方法,该方法直接对超模型集进行采样,并通过选择方向向量和极化所有方向来找到所有边界点。实验结果表明,我们的方法改进了上述问题,计算速度更快,计算结果接近,边界点的重复率大大降低,相应元模型集的实际维数与理论维数一致。
{"title":"Metamer mismatch volume calculation method based on high-dimensional spherical sampling","authors":"Kuan Xu, Long Ma, Peng Li","doi":"10.1117/12.2687942","DOIUrl":"https://doi.org/10.1117/12.2687942","url":null,"abstract":"The metamer mismatch volume has important applications in color correction, camera design, and light source design. The method based on spherical sampling to calculate the metamer mismatch volume has a long computation time, a large number of duplicated boundary points, too few effective vertices, and the dimension of its metamer set will appear lower than the theoretical dimension. In this paper, we propose a high-dimensional spherical sampling method that samples the metamer set directly, and find all boundary points by selecting direction vectors and polarizing all directions. The experimental results show that our method improves the above problems, the computational speed is faster, the computational results are close, the repetition rate of boundary points is greatly reduced, and the actual dimensionality of the corresponding metamer set is consistent with the theoretical dimensionality.","PeriodicalId":38836,"journal":{"name":"Meta: Avaliacao","volume":"12785 1","pages":"1278503 - 1278503-9"},"PeriodicalIF":0.0,"publicationDate":"2023-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44355698","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Study on spectral adaptive transformation based on chromatic aberration 基于色差的光谱自适应变换研究
Q4 Social Sciences Pub Date : 2023-08-11 DOI: 10.1117/12.2687939
Long Ma, Haitang Chen
The spectral reflectance of multispectral images can provide more valuable information about object characteristics. In order to improve the utilization of the spectrum, the reflectance reconstruction requires the same system calibration and illumination of the image acquisition. Therefore, Khan proposed the concept of multispectral constancy, which is to transform the multispectral image data into a standard representation through spectral adaptive transformation. Khan used the linear mapping method to solve SAT to convert the multispectral image data obtained under unknown illumination into the image data under standard light source. In order to further improve the spectral utilization rate and expand the application range of multispectral cameras, an algorithm to improve multispectral constancy based on chromatic aberration index is proposed in this paper. The algorithm uses chromatic aberration as the objective function to solve the spectral adaptive transformation. In this paper, ten light sources are used as unknown light sources, SFU and X-rite are used as training and testing datasets, and multispectral camera channels are simulated by Equi-Gaussian and Equi-Energy filters with different number of channels to train and test 5, 6, 8, and 10 channels of data. In this paper, the color difference under different light sources is used as the evaluation index to test the performance of the proposed algorithm, and compared with the Khan method for calculating SAT multispectral constancy. The experimental results show that the spectral constancy algorithm based on color difference can perform better, and expand the application of different kinds of unknown light sources in multispectral constancy.
多光谱图像的光谱反射率可以提供更有价值的目标特征信息。为了提高光谱的利用率,反射率重建需要与图像采集相同的系统校准和照明。因此,Khan提出了多光谱恒常性的概念,即通过光谱自适应变换将多光谱图像数据转换成标准表示。Khan采用线性映射法求解SAT,将未知照度下获得的多光谱图像数据转换为标准光源下的图像数据。为了进一步提高多光谱相机的光谱利用率,扩大多光谱相机的应用范围,本文提出了一种基于色差指数提高多光谱常数的算法。该算法以色差为目标函数求解光谱自适应变换。本文以10个光源作为未知光源,以SFU和X-rite作为训练和测试数据集,采用不同通道数的等高斯滤波器和等能量滤波器模拟多光谱相机通道,对5、6、8、10通道数据进行训练和测试。本文以不同光源下的色差作为评价指标,测试了所提算法的性能,并与计算SAT多光谱常数的Khan方法进行了比较。实验结果表明,基于色差的光谱常数算法具有较好的性能,扩展了不同类型未知光源在多光谱常数中的应用。
{"title":"Study on spectral adaptive transformation based on chromatic aberration","authors":"Long Ma, Haitang Chen","doi":"10.1117/12.2687939","DOIUrl":"https://doi.org/10.1117/12.2687939","url":null,"abstract":"The spectral reflectance of multispectral images can provide more valuable information about object characteristics. In order to improve the utilization of the spectrum, the reflectance reconstruction requires the same system calibration and illumination of the image acquisition. Therefore, Khan proposed the concept of multispectral constancy, which is to transform the multispectral image data into a standard representation through spectral adaptive transformation. Khan used the linear mapping method to solve SAT to convert the multispectral image data obtained under unknown illumination into the image data under standard light source. In order to further improve the spectral utilization rate and expand the application range of multispectral cameras, an algorithm to improve multispectral constancy based on chromatic aberration index is proposed in this paper. The algorithm uses chromatic aberration as the objective function to solve the spectral adaptive transformation. In this paper, ten light sources are used as unknown light sources, SFU and X-rite are used as training and testing datasets, and multispectral camera channels are simulated by Equi-Gaussian and Equi-Energy filters with different number of channels to train and test 5, 6, 8, and 10 channels of data. In this paper, the color difference under different light sources is used as the evaluation index to test the performance of the proposed algorithm, and compared with the Khan method for calculating SAT multispectral constancy. The experimental results show that the spectral constancy algorithm based on color difference can perform better, and expand the application of different kinds of unknown light sources in multispectral constancy.","PeriodicalId":38836,"journal":{"name":"Meta: Avaliacao","volume":"12785 1","pages":"1278505 - 1278505-16"},"PeriodicalIF":0.0,"publicationDate":"2023-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46590887","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Improved color accuracy of the camera using optimized matching illumination method 采用优化匹配光照方法,提高了相机的色彩精度
Q4 Social Sciences Pub Date : 2023-08-11 DOI: 10.1117/12.2687940
Long Ma, Jing Chen
To improve the color reproduction and realism of digital cameras and to promote the development of computer vision. Camera colorimetry is conditioned on the spectral sensitivity response of the camera being a linear transformation of the color matching function of the human visual system. Previous methods have proposed placing well-designed filters in front of the camera to produce a sensitivity that well matches the Luther condition. In this paper, we optimize the latest matching illumination method (by using a spectral-tunable illumination system to modulate the spectrum of certain light sources), improve the method of designing filters and add new constraints. Experiments demonstrate that the matching illumination method using new objective functions give a 5% improvement over the original method, and the optimization of the filter using a gradient ascent algorithm and a genetic algorithm gives a 10% improvement in chromaticity over the original method. The method of limiting the average transmittance also has a 10% improvement over the previous one. As a result, these methods can make the imaging of digital cameras more accurate and realistic.
提高数码相机的色彩再现性和真实感,促进计算机视觉的发展。相机色度学的条件是相机的光谱灵敏度响应是人类视觉系统的颜色匹配函数的线性变换。以前的方法建议在相机前面放置精心设计的滤波器,以产生与Luther条件完全匹配的灵敏度。在本文中,我们优化了最新的匹配照明方法(通过使用光谱可调照明系统来调制某些光源的光谱),改进了滤波器的设计方法,并添加了新的约束。实验表明,使用新目标函数的匹配照明方法比原始方法提高了5%,使用梯度上升算法和遗传算法的滤波器优化比原始方法的色度提高了10%。限制平均透射率的方法也比之前的方法提高了10%。因此,这些方法可以使数码相机的成像更加准确和逼真。
{"title":"Improved color accuracy of the camera using optimized matching illumination method","authors":"Long Ma, Jing Chen","doi":"10.1117/12.2687940","DOIUrl":"https://doi.org/10.1117/12.2687940","url":null,"abstract":"To improve the color reproduction and realism of digital cameras and to promote the development of computer vision. Camera colorimetry is conditioned on the spectral sensitivity response of the camera being a linear transformation of the color matching function of the human visual system. Previous methods have proposed placing well-designed filters in front of the camera to produce a sensitivity that well matches the Luther condition. In this paper, we optimize the latest matching illumination method (by using a spectral-tunable illumination system to modulate the spectrum of certain light sources), improve the method of designing filters and add new constraints. Experiments demonstrate that the matching illumination method using new objective functions give a 5% improvement over the original method, and the optimization of the filter using a gradient ascent algorithm and a genetic algorithm gives a 10% improvement in chromaticity over the original method. The method of limiting the average transmittance also has a 10% improvement over the previous one. As a result, these methods can make the imaging of digital cameras more accurate and realistic.","PeriodicalId":38836,"journal":{"name":"Meta: Avaliacao","volume":"12785 1","pages":"1278506 - 1278506-10"},"PeriodicalIF":0.0,"publicationDate":"2023-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46596809","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Machine vision-based portable track inspection system 基于机器视觉的便携式轨道检测系统
Q4 Social Sciences Pub Date : 2023-08-11 DOI: 10.1117/12.2687936
Qing Li, L. Wei, Xin Qu, Kai Cheng, Yanbo Chang, Houle Zhou
With the rapid development of the transportation industry, railway transportation plays a crucial role. Manual inspection methods are time-consuming, labor-intensive, and highly subjective. Therefore, there is an urgent need for a more efficient and accurate flaw detection method. This system is a portable rail flaw detection device based on machine vision, with YOLOv5 as its core deep learning algorithm. The system captures surface images of the rail through a camera and transmits them in real-time to the host computer for analysis. Leveraging the powerful real-time object detection capability of YOLOv5s, the system can accurately identify and locate various types of rail surface damages, such as cracks, fractures, and wear. Compared to traditional manual inspection, this system is more efficient and greatly improves the accuracy and efficiency of rail flaw detection. It has a smaller size and is convenient to carry, making it suitable for working in various environments and conditions, greatly enhancing the practicality and flexibility of the device.
随着交通运输业的快速发展,铁路运输发挥着至关重要的作用。手动检查方法耗时、劳动密集,而且主观性很强。因此,迫切需要一种更高效、更准确的探伤方法。该系统是一款基于机器视觉的便携式钢轨探伤设备,以YOLOv5为核心的深度学习算法。该系统通过摄像头捕捉铁轨的表面图像,并将其实时传输到主机进行分析。利用YOLOv5s强大的实时物体检测能力,该系统可以准确识别和定位各种类型的钢轨表面损伤,如裂纹、断裂和磨损。与传统的手工检测相比,该系统效率更高,大大提高了钢轨探伤的准确性和效率。它体积更小,携带方便,适合在各种环境和条件下工作,大大增强了设备的实用性和灵活性。
{"title":"Machine vision-based portable track inspection system","authors":"Qing Li, L. Wei, Xin Qu, Kai Cheng, Yanbo Chang, Houle Zhou","doi":"10.1117/12.2687936","DOIUrl":"https://doi.org/10.1117/12.2687936","url":null,"abstract":"With the rapid development of the transportation industry, railway transportation plays a crucial role. Manual inspection methods are time-consuming, labor-intensive, and highly subjective. Therefore, there is an urgent need for a more efficient and accurate flaw detection method. This system is a portable rail flaw detection device based on machine vision, with YOLOv5 as its core deep learning algorithm. The system captures surface images of the rail through a camera and transmits them in real-time to the host computer for analysis. Leveraging the powerful real-time object detection capability of YOLOv5s, the system can accurately identify and locate various types of rail surface damages, such as cracks, fractures, and wear. Compared to traditional manual inspection, this system is more efficient and greatly improves the accuracy and efficiency of rail flaw detection. It has a smaller size and is convenient to carry, making it suitable for working in various environments and conditions, greatly enhancing the practicality and flexibility of the device.","PeriodicalId":38836,"journal":{"name":"Meta: Avaliacao","volume":"12785 1","pages":"1278502 - 1278502-8"},"PeriodicalIF":0.0,"publicationDate":"2023-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45148061","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Optimization of RGB image spectral reconstruction based on radial basis function networks 基于径向基函数网络的RGB图像光谱重建优化
Q4 Social Sciences Pub Date : 2023-08-11 DOI: 10.1117/12.2687949
Long Ma, Zhipeng Qian
The Spectral Reconstruction (SR) algorithm attempts to recover hyperspectral information from RGB camera responses. This estimation problem is usually formulated as a least squares regression, and because the data is noisy, Tikhonov regularization is reconsidered. The degree of regularization is controlled by a single penalty parameter. This paper improves the traditional cross validation experiment method for the optimization of this parameter. In addition, this article also proposes an improved SR model. Unlike common SR models, our method divides the processed RGB space into different numbers of neighborhoods and determines the center point of each neighborhood. Finally, the adjacent RGB data and spectral data of each center point are used as input and output data for the Radial Basis Function Network (RBFN) model to train the SR regression of each RGB neighborhood. This article selects MRAE and RMSE to evaluate the performance of the SR algorithm. Through comparison with different SR models, the methods proposed in this article have significant performance improvements.
光谱重建(SR)算法试图从RGB相机响应中恢复高光谱信息。这种估计问题通常被表述为最小二乘回归,由于数据是有噪声的,Tikhonov正则化被重新考虑。正则化程度由单个惩罚参数控制。本文改进了传统的交叉验证实验方法,对该参数进行了优化。此外,本文还提出了一种改进的SR模型。与普通的SR模型不同,我们的方法将处理后的RGB空间划分为不同数量的邻域,并确定每个邻域的中心点。最后,将每个中心点的相邻RGB数据和光谱数据作为径向基函数网络(RBFN)模型的输入和输出数据,训练每个RGB邻域的SR回归。本文选取MRAE和RMSE来评价SR算法的性能。通过与不同SR模型的比较,本文提出的方法具有显著的性能改进。
{"title":"Optimization of RGB image spectral reconstruction based on radial basis function networks","authors":"Long Ma, Zhipeng Qian","doi":"10.1117/12.2687949","DOIUrl":"https://doi.org/10.1117/12.2687949","url":null,"abstract":"The Spectral Reconstruction (SR) algorithm attempts to recover hyperspectral information from RGB camera responses. This estimation problem is usually formulated as a least squares regression, and because the data is noisy, Tikhonov regularization is reconsidered. The degree of regularization is controlled by a single penalty parameter. This paper improves the traditional cross validation experiment method for the optimization of this parameter. In addition, this article also proposes an improved SR model. Unlike common SR models, our method divides the processed RGB space into different numbers of neighborhoods and determines the center point of each neighborhood. Finally, the adjacent RGB data and spectral data of each center point are used as input and output data for the Radial Basis Function Network (RBFN) model to train the SR regression of each RGB neighborhood. This article selects MRAE and RMSE to evaluate the performance of the SR algorithm. Through comparison with different SR models, the methods proposed in this article have significant performance improvements.","PeriodicalId":38836,"journal":{"name":"Meta: Avaliacao","volume":"12785 1","pages":"1278507 - 1278507-11"},"PeriodicalIF":0.0,"publicationDate":"2023-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46538733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Camera spectral sensitivity estimation based on spectrally tunable LED illumination 基于光谱可调LED照明的相机光谱灵敏度估计
Q4 Social Sciences Pub Date : 2023-08-11 DOI: 10.1117/12.2687937
Long Ma, Bowen Xu
The choice of light source affects the accuracy of the spectral sensitivity estimation. In this paper, we propose to estimate the spectral sensitivity function of digital camera using spectrally tunable LED light sources. The spectral power distribution of the LED light source is determined by a combination of multiple LEDs and their weight coefficients. The method of tuning the weight coefficients of the LEDs includes Monte Carlo method and particle swarm optimization algorithm, so that the LED light source with the smallest estimation error is defined as the optimal light source. Experimental results show that the particle swarm algorithm gives the best estimation results. The relative error of estimation using LED light sources is significantly reduced when compared with the results when using a single light source for estimation (e.g., D65 light source).
光源的选择影响光谱灵敏度估计的准确性。在本文中,我们建议使用光谱可调LED光源来估计数码相机的光谱灵敏度函数。LED光源的光谱功率分布由多个LED及其权重系数的组合确定。调整LED权重系数的方法包括蒙特卡罗方法和粒子群优化算法,从而将估计误差最小的LED光源定义为最佳光源。实验结果表明,粒子群算法给出了最好的估计结果。与使用单个光源(例如D65光源)进行估计时的结果相比,使用LED光源进行估计的相对误差显著降低。
{"title":"Camera spectral sensitivity estimation based on spectrally tunable LED illumination","authors":"Long Ma, Bowen Xu","doi":"10.1117/12.2687937","DOIUrl":"https://doi.org/10.1117/12.2687937","url":null,"abstract":"The choice of light source affects the accuracy of the spectral sensitivity estimation. In this paper, we propose to estimate the spectral sensitivity function of digital camera using spectrally tunable LED light sources. The spectral power distribution of the LED light source is determined by a combination of multiple LEDs and their weight coefficients. The method of tuning the weight coefficients of the LEDs includes Monte Carlo method and particle swarm optimization algorithm, so that the LED light source with the smallest estimation error is defined as the optimal light source. Experimental results show that the particle swarm algorithm gives the best estimation results. The relative error of estimation using LED light sources is significantly reduced when compared with the results when using a single light source for estimation (e.g., D65 light source).","PeriodicalId":38836,"journal":{"name":"Meta: Avaliacao","volume":"12785 1","pages":"1278504 - 1278504-13"},"PeriodicalIF":0.0,"publicationDate":"2023-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44279416","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Pikachu
Q4 Social Sciences Pub Date : 2022-11-07 DOI: 10.1145/3560829.3563563
Sarah Azouvi, M. Vukolic
Blockchain systems based on a reusable resource, such as proof-of-stake (PoS), provide weaker security guarantees than those based on proof-of-work. Specifically, they are vulnerable to long-range attacks, where an adversary can corrupt prior participants in order to rewrite the full history of the chain. To prevent this attack on a PoS chain, we propose a protocol that checkpoints the state of the PoS chain to a proof-of-work blockchain such as Bitcoin. Our checkpointing protocol hence does not rely on any central authority. Our work uses Schnorr signatures and leverages Bitcoin recent Taproot upgrade, allowing us to create a checkpointing transaction of constant size. We argue for the security of our protocol and present an open-source implementation that was tested on the Bitcoin testnet.
{"title":"Pikachu","authors":"Sarah Azouvi, M. Vukolic","doi":"10.1145/3560829.3563563","DOIUrl":"https://doi.org/10.1145/3560829.3563563","url":null,"abstract":"Blockchain systems based on a reusable resource, such as proof-of-stake (PoS), provide weaker security guarantees than those based on proof-of-work. Specifically, they are vulnerable to long-range attacks, where an adversary can corrupt prior participants in order to rewrite the full history of the chain. To prevent this attack on a PoS chain, we propose a protocol that checkpoints the state of the PoS chain to a proof-of-work blockchain such as Bitcoin. Our checkpointing protocol hence does not rely on any central authority. Our work uses Schnorr signatures and leverages Bitcoin recent Taproot upgrade, allowing us to create a checkpointing transaction of constant size. We argue for the security of our protocol and present an open-source implementation that was tested on the Bitcoin testnet.","PeriodicalId":38836,"journal":{"name":"Meta: Avaliacao","volume":"68 37","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72445261","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 10
Exascale potholes for HPC: Execution performance and variability analysis of the flagship application code HemeLB 高性能计算的百亿亿级漏洞:旗舰应用程序代码HemeLB的执行性能和可变性分析
Q4 Social Sciences Pub Date : 2020-11-01 DOI: 10.1109/HUSTProtools51951.2020.00014
B. Wylie
Performance measurement and analysis of parallel applications is often challenging, despite many excellent commercial and open-source tools being available. Currently envisaged exascale computer systems exacerbate matters by requiring extremely high scalability to effectively exploit millions of processor cores. Unfortunately, significant application execution performance variability arising from increasingly complex interactions between hardware and system software makes this situation much more difficult for application developers and performance analysts alike. This work considers the performance assessment of the HemeLB exascale flagship application code from the EU HPC Centre of Excellence (CoE) for Computational Biomedicine (CompBioMed) running on the SuperMUC-NG Tier-0 leadership system, using the methodology of the Performance Optimisation and Productivity (POP) CoE. Although 80% scaling efficiency is maintained to over 100,000 MPI processes, disappointing initial performance with more processes and corresponding poor strong scaling was identified to originate from the same few compute nodes in multiple runs, which later system diagnostic checks found had faulty DIMMs and lacklustre performance. Excluding these compute nodes from subsequent runs improved performance of executions with over 300,000 MPI processes by a factor of five, resulting in 190 x speed-up compared to 864 MPI processes. While communication efficiency remains very good up to the largest scale, parallel efficiency is primarily limited by load balance found to be largely due to core-to-core and run-to-run variability from excessive stalls for memory accesses, that affect many HPC systems with Intel Xeon Scalable processors. The POP methodology for this performance diagnosis is demonstrated via a detailed exposition with widely deployed ‘standard’ measurement and analysis tools.
尽管有许多优秀的商业和开源工具可用,但并行应用程序的性能度量和分析通常是具有挑战性的。目前设想的百亿亿次计算机系统需要极高的可扩展性来有效地利用数百万个处理器核心,这使问题更加严重。不幸的是,由于硬件和系统软件之间日益复杂的交互而产生的显著的应用程序执行性能可变性,使得应用程序开发人员和性能分析人员都更加困难。这项工作考虑了在supermu - ng Tier-0领导系统上运行的欧盟高性能计算卓越中心(CoE)计算生物医学(CompBioMed)的HemeLB百亿级旗舰应用程序代码的性能评估,使用了性能优化和生产力(POP) CoE的方法。尽管在超过100,000个MPI进程的情况下保持了80%的扩展效率,但是随着更多的进程和相应的较弱的扩展,令人失望的初始性能被确定为源于多次运行中相同的几个计算节点,后来系统诊断检查发现这些节点有故障的内存和低迷的性能。将这些计算节点从后续运行中排除后,超过30万个MPI进程的执行性能提高了5倍,与864个MPI进程相比,速度提高了190倍。虽然通信效率在最大规模上仍然非常好,但并行效率主要受到负载平衡的限制,这主要是由于内核对内核和运行对运行的可变性,这些可变性来自内存访问的过度停滞,这影响了许多使用英特尔至强可扩展处理器的HPC系统。通过对广泛部署的“标准”测量和分析工具的详细阐述,演示了这种性能诊断的POP方法。
{"title":"Exascale potholes for HPC: Execution performance and variability analysis of the flagship application code HemeLB","authors":"B. Wylie","doi":"10.1109/HUSTProtools51951.2020.00014","DOIUrl":"https://doi.org/10.1109/HUSTProtools51951.2020.00014","url":null,"abstract":"Performance measurement and analysis of parallel applications is often challenging, despite many excellent commercial and open-source tools being available. Currently envisaged exascale computer systems exacerbate matters by requiring extremely high scalability to effectively exploit millions of processor cores. Unfortunately, significant application execution performance variability arising from increasingly complex interactions between hardware and system software makes this situation much more difficult for application developers and performance analysts alike. This work considers the performance assessment of the HemeLB exascale flagship application code from the EU HPC Centre of Excellence (CoE) for Computational Biomedicine (CompBioMed) running on the SuperMUC-NG Tier-0 leadership system, using the methodology of the Performance Optimisation and Productivity (POP) CoE. Although 80% scaling efficiency is maintained to over 100,000 MPI processes, disappointing initial performance with more processes and corresponding poor strong scaling was identified to originate from the same few compute nodes in multiple runs, which later system diagnostic checks found had faulty DIMMs and lacklustre performance. Excluding these compute nodes from subsequent runs improved performance of executions with over 300,000 MPI processes by a factor of five, resulting in 190 x speed-up compared to 864 MPI processes. While communication efficiency remains very good up to the largest scale, parallel efficiency is primarily limited by load balance found to be largely due to core-to-core and run-to-run variability from excessive stalls for memory accesses, that affect many HPC systems with Intel Xeon Scalable processors. The POP methodology for this performance diagnosis is demonstrated via a detailed exposition with widely deployed ‘standard’ measurement and analysis tools.","PeriodicalId":38836,"journal":{"name":"Meta: Avaliacao","volume":"85 1","pages":"59-70"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75823804","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Integrating Science Gateways with Secure Cloud Computing Resources: An Examination of Two Deployment Patterns and Their Requirements 集成科学网关与安全云计算资源:两种部署模式及其需求分析
Q4 Social Sciences Pub Date : 2020-11-01 DOI: 10.1109/HUSTProtools51951.2020.00010
M. Pierce, S. Marru
This paper examines scenarios in which science gateways can facilitate access to cloud computing resources to support scientific research using regulated or protected data stored on clouds. Specifically, we discuss the use of science gateways to access Controlled Unclassified Information (CUI), a US regulatory standard that covers a broad range of US federal government-owned or regulated data, and that also provides a useful proxy for other types of sensitive data, such as private sector intellectual property. We focus on the impact of CUI requirements on science gateway platforms that can be used to create and manage science gateway instances. Gateway platforms are centrally operated by gateway platform providers who create and control gateway instances on behalf of gateway providers. Broadly, platforms operate following either a multi-tenant or else a multi-instance pattern. Multi-tenanted science gateway platforms are designed to support multiple science gateways simultaneously, with each gateway as a tenant to a single operational instance of the platform middleware. Multi-instance platforms, on the other hand, provide and manage an entire instance of the science gateway software for each gateway. This paper reviews these two scenarios from the perspective of the Science Gateways Platform as a service (SciGaP), a multitenanted gateway platform based on the open-source Apache Airavata software. We examine requirements for providing multitenanted platforms for CUI gateways and also the requirements for providing the same software as a multi-instance platform. In both cases, we assume the use of CUI-compatible resources from commercial cloud providers. Both approaches are technically feasible but have trade-offs that must be considered.
本文研究了科学网关可以促进访问云计算资源的场景,以便使用存储在云上的受监管或受保护的数据来支持科学研究。具体来说,我们讨论了科学网关访问受控非机密信息(CUI)的使用,这是一项美国监管标准,涵盖了广泛的美国联邦政府拥有或监管的数据,并且还为其他类型的敏感数据(如私营部门知识产权)提供了有用的代理。我们将重点关注CUI需求对科学网关平台的影响,这些平台可用于创建和管理科学网关实例。网关平台由网关平台提供商集中操作,网关平台提供商代表网关提供商创建和控制网关实例。一般来说,平台要么遵循多租户模式,要么遵循多实例模式。多租户科学网关平台旨在同时支持多个科学网关,每个网关作为平台中间件的单个操作实例的租户。另一方面,多实例平台为每个网关提供和管理科学网关软件的完整实例。本文从基于开源Apache Airavata软件的多租户网关平台科学网关平台即服务(sciigap)的角度对这两种场景进行了回顾。我们研究了为CUI网关提供多租户平台的需求,以及为多实例平台提供相同软件的需求。在这两种情况下,我们都假设使用来自商业云提供商的gui兼容资源。这两种方法在技术上都是可行的,但必须考虑折衷。
{"title":"Integrating Science Gateways with Secure Cloud Computing Resources: An Examination of Two Deployment Patterns and Their Requirements","authors":"M. Pierce, S. Marru","doi":"10.1109/HUSTProtools51951.2020.00010","DOIUrl":"https://doi.org/10.1109/HUSTProtools51951.2020.00010","url":null,"abstract":"This paper examines scenarios in which science gateways can facilitate access to cloud computing resources to support scientific research using regulated or protected data stored on clouds. Specifically, we discuss the use of science gateways to access Controlled Unclassified Information (CUI), a US regulatory standard that covers a broad range of US federal government-owned or regulated data, and that also provides a useful proxy for other types of sensitive data, such as private sector intellectual property. We focus on the impact of CUI requirements on science gateway platforms that can be used to create and manage science gateway instances. Gateway platforms are centrally operated by gateway platform providers who create and control gateway instances on behalf of gateway providers. Broadly, platforms operate following either a multi-tenant or else a multi-instance pattern. Multi-tenanted science gateway platforms are designed to support multiple science gateways simultaneously, with each gateway as a tenant to a single operational instance of the platform middleware. Multi-instance platforms, on the other hand, provide and manage an entire instance of the science gateway software for each gateway. This paper reviews these two scenarios from the perspective of the Science Gateways Platform as a service (SciGaP), a multitenanted gateway platform based on the open-source Apache Airavata software. We examine requirements for providing multitenanted platforms for CUI gateways and also the requirements for providing the same software as a multi-instance platform. In both cases, we assume the use of CUI-compatible resources from commercial cloud providers. Both approaches are technically feasible but have trade-offs that must be considered.","PeriodicalId":38836,"journal":{"name":"Meta: Avaliacao","volume":"15 1","pages":"19-26"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81986651","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
[Title page] (标题页)
Q4 Social Sciences Pub Date : 2020-11-01 DOI: 10.1109/hustprotools51951.2020.00001
{"title":"[Title page]","authors":"","doi":"10.1109/hustprotools51951.2020.00001","DOIUrl":"https://doi.org/10.1109/hustprotools51951.2020.00001","url":null,"abstract":"","PeriodicalId":38836,"journal":{"name":"Meta: Avaliacao","volume":"4 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86221372","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Meta: Avaliacao
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1