加强藻华预测:特殊时间模式时期机器学习性能评估的新框架

IF 4.8 2区 环境科学与生态学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Environmental Modelling & Software Pub Date : 2024-07-25 DOI:10.1016/j.envsoft.2024.106164
Wei Xia , Ilija Ilievski , Christine Ann Shoemaker
{"title":"加强藻华预测:特殊时间模式时期机器学习性能评估的新框架","authors":"Wei Xia ,&nbsp;Ilija Ilievski ,&nbsp;Christine Ann Shoemaker","doi":"10.1016/j.envsoft.2024.106164","DOIUrl":null,"url":null,"abstract":"<div><p>The evaluation of algal bloom forecasting models typically relies on error metrics that quantify the forecasting performance over the whole test set as a single number. Furthermore, the comparison with simple baseline methods is often omitted. To address this, we introduce a novel framework for Model performance Analysis and Visualization of time series forecasting (MAVts). MAVts incorporates novel algorithms for the automatic identification and visualization of time series periods of interest where the forecasting models are evaluated and compared with simple baseline methods. The application of MAVts on evaluating algal bloom forecasting models composed of sophisticated machine learning (ML) methods, reveals that in 85% of experiments a single error metric is not enough and only in 12.5% of experiments a ML model outperforms all baselines on all metrics and periods of interest. Thus, MAVts emerges as a valuable tool for analyzing and comparing ML models, advancing environmental management and protection.</p></div>","PeriodicalId":310,"journal":{"name":"Environmental Modelling & Software","volume":"180 ","pages":"Article 106164"},"PeriodicalIF":4.8000,"publicationDate":"2024-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Enhancing algal bloom forecasting: A novel framework for machine learning performance evaluation during periods of special temporal patterns\",\"authors\":\"Wei Xia ,&nbsp;Ilija Ilievski ,&nbsp;Christine Ann Shoemaker\",\"doi\":\"10.1016/j.envsoft.2024.106164\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>The evaluation of algal bloom forecasting models typically relies on error metrics that quantify the forecasting performance over the whole test set as a single number. Furthermore, the comparison with simple baseline methods is often omitted. To address this, we introduce a novel framework for Model performance Analysis and Visualization of time series forecasting (MAVts). MAVts incorporates novel algorithms for the automatic identification and visualization of time series periods of interest where the forecasting models are evaluated and compared with simple baseline methods. The application of MAVts on evaluating algal bloom forecasting models composed of sophisticated machine learning (ML) methods, reveals that in 85% of experiments a single error metric is not enough and only in 12.5% of experiments a ML model outperforms all baselines on all metrics and periods of interest. Thus, MAVts emerges as a valuable tool for analyzing and comparing ML models, advancing environmental management and protection.</p></div>\",\"PeriodicalId\":310,\"journal\":{\"name\":\"Environmental Modelling & Software\",\"volume\":\"180 \",\"pages\":\"Article 106164\"},\"PeriodicalIF\":4.8000,\"publicationDate\":\"2024-07-25\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Environmental Modelling & Software\",\"FirstCategoryId\":\"93\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1364815224002251\",\"RegionNum\":2,\"RegionCategory\":\"环境科学与生态学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Environmental Modelling & Software","FirstCategoryId":"93","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1364815224002251","RegionNum":2,"RegionCategory":"环境科学与生态学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0

摘要

藻华预报模型的评估通常依赖于误差指标,即用一个数字量化整个测试集的预报性能。此外,与简单基线方法的比较往往被忽略。为了解决这个问题,我们引入了一个新颖的时间序列预测模型性能分析和可视化框架(MAVts)。MAVts 采用了新颖的算法,用于自动识别和可视化感兴趣的时间序列期,在此基础上对预测模型进行评估,并与简单的基线方法进行比较。在评估由复杂的机器学习(ML)方法组成的藻华预测模型时,MAVts 的应用表明,在 85% 的实验中,单一误差指标是不够的,只有 12.5% 的实验中,ML 模型在所有指标和相关时段上都优于所有基线方法。因此,MAVts 成为分析和比较 ML 模型的重要工具,推动了环境管理和保护。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Enhancing algal bloom forecasting: A novel framework for machine learning performance evaluation during periods of special temporal patterns

The evaluation of algal bloom forecasting models typically relies on error metrics that quantify the forecasting performance over the whole test set as a single number. Furthermore, the comparison with simple baseline methods is often omitted. To address this, we introduce a novel framework for Model performance Analysis and Visualization of time series forecasting (MAVts). MAVts incorporates novel algorithms for the automatic identification and visualization of time series periods of interest where the forecasting models are evaluated and compared with simple baseline methods. The application of MAVts on evaluating algal bloom forecasting models composed of sophisticated machine learning (ML) methods, reveals that in 85% of experiments a single error metric is not enough and only in 12.5% of experiments a ML model outperforms all baselines on all metrics and periods of interest. Thus, MAVts emerges as a valuable tool for analyzing and comparing ML models, advancing environmental management and protection.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Environmental Modelling & Software
Environmental Modelling & Software 工程技术-工程:环境
CiteScore
9.30
自引率
8.20%
发文量
241
审稿时长
60 days
期刊介绍: Environmental Modelling & Software publishes contributions, in the form of research articles, reviews and short communications, on recent advances in environmental modelling and/or software. The aim is to improve our capacity to represent, understand, predict or manage the behaviour of environmental systems at all practical scales, and to communicate those improvements to a wide scientific and professional audience.
期刊最新文献
QGIS Shoreline Change Analysis Tool (QSCAT): A fast, open-source shoreline change analysis plugin for QGIS Probability analysis of shallow landslides in varying vegetation zones with random soil grain-size distribution Variable sensitivity analysis in groundwater level projections under climate change adopting a hybrid machine learning algorithm Taxonomy of purposes, methods, and recommendations for vulnerability analysis Canopy height Mapper: A google earth engine application for predicting global canopy heights combining GEDI with multi-source data
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1