{"title":"A framework for assessing the computational reproducibility of geo-simulation experiments","authors":"Zhiyi Zhu, Min Chen, Guangjin Ren, Yuanqing He, Lingzhi Sun, Fengyuan Zhang, Yongning Wen, Songshan Yue, Guonian Lü","doi":"10.1016/j.envsoft.2025.106323","DOIUrl":null,"url":null,"abstract":"Recent advances in computational technologies have enhanced geo-simulation experiments (GSEs), making computational reproducibility assessments increasingly critical. However, existing methods often focus on isolated aspects, lacking a comprehensive framework. This study proposes an integrated framework for assessing reproducibility in GSEs, structured into two parts: (1) evaluating overall computational workflows, and (2) investigating individual processes to identify inconsistencies. The framework employs a detailed assessment model using hierarchical dimensions and metrics that combine quantitative measures (e.g., output consistency) and qualitative evaluations (e.g., clarity of descriptions). These components address both broad and granular aspects of computational processes. The framework is implemented in a prototype system to support reproducibility assessments and demonstrated through practical applications. This systematic approach provides a robust and adaptable method for assessing reproducibility, promoting the resolution of challenges in existing methods.","PeriodicalId":310,"journal":{"name":"Environmental Modelling & Software","volume":"38 1","pages":""},"PeriodicalIF":4.8000,"publicationDate":"2025-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Environmental Modelling & Software","FirstCategoryId":"93","ListUrlMain":"https://doi.org/10.1016/j.envsoft.2025.106323","RegionNum":2,"RegionCategory":"环境科学与生态学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0
Abstract
Recent advances in computational technologies have enhanced geo-simulation experiments (GSEs), making computational reproducibility assessments increasingly critical. However, existing methods often focus on isolated aspects, lacking a comprehensive framework. This study proposes an integrated framework for assessing reproducibility in GSEs, structured into two parts: (1) evaluating overall computational workflows, and (2) investigating individual processes to identify inconsistencies. The framework employs a detailed assessment model using hierarchical dimensions and metrics that combine quantitative measures (e.g., output consistency) and qualitative evaluations (e.g., clarity of descriptions). These components address both broad and granular aspects of computational processes. The framework is implemented in a prototype system to support reproducibility assessments and demonstrated through practical applications. This systematic approach provides a robust and adaptable method for assessing reproducibility, promoting the resolution of challenges in existing methods.
期刊介绍:
Environmental Modelling & Software publishes contributions, in the form of research articles, reviews and short communications, on recent advances in environmental modelling and/or software. The aim is to improve our capacity to represent, understand, predict or manage the behaviour of environmental systems at all practical scales, and to communicate those improvements to a wide scientific and professional audience.