Pub Date : 2024-08-01Epub Date: 2024-02-08DOI: 10.1111/risa.14277
Raphaël Trouvé, Andrew P Robinson
Biological invasions are a growing threat to biodiversity, food security, and economies. Rising pressure from increased global trade requires improving border inspection efficiency. Here, we depart from the conventional consignment-by-consignment approach advocated in current inspection standards. Instead, we suggest a broader perspective: evaluating border inspection regimes based on their ability to reduce propagule pressure across entire pathways. Additionally, we demonstrate that most biosecurity pathways exhibit superspreading behavior, that is, consignments from the same pathway have varying infestation rates and contain rare right-tail events (also called overdispersion). We show that greater overdispersion leads to more pronounced diminishing returns, with consequences on the optimal allocation of sampling effort. We leverage these two insights to develop a simple and efficient border inspection regime that can significantly reduce propagule pressure compared to current standards. Our analysis revealed that consignment size is a key driver of biosecurity risk and that sampling proportional to the square root of consignment size is near optimal. In testing, our framework reduced propagule pressure by 31 to 38% compared to current standards. We also identified opportunities to further improve inspection efficiency by considering additional pathway characteristics (i.e., overdispersion parameters, zero inflation, relative risk, sampling cost, detectability) and developed solutions for these more complex scenarios. We anticipate our result will mitigate biological invasion risk with significant implications for biodiversity conservation, food security, and economies worldwide.
{"title":"Efficient border biosecurity inspection leverages superspreading to reduce biological invasion risk.","authors":"Raphaël Trouvé, Andrew P Robinson","doi":"10.1111/risa.14277","DOIUrl":"10.1111/risa.14277","url":null,"abstract":"<p><p>Biological invasions are a growing threat to biodiversity, food security, and economies. Rising pressure from increased global trade requires improving border inspection efficiency. Here, we depart from the conventional consignment-by-consignment approach advocated in current inspection standards. Instead, we suggest a broader perspective: evaluating border inspection regimes based on their ability to reduce propagule pressure across entire pathways. Additionally, we demonstrate that most biosecurity pathways exhibit superspreading behavior, that is, consignments from the same pathway have varying infestation rates and contain rare right-tail events (also called overdispersion). We show that greater overdispersion leads to more pronounced diminishing returns, with consequences on the optimal allocation of sampling effort. We leverage these two insights to develop a simple and efficient border inspection regime that can significantly reduce propagule pressure compared to current standards. Our analysis revealed that consignment size is a key driver of biosecurity risk and that sampling proportional to the square root of consignment size is near optimal. In testing, our framework reduced propagule pressure by 31 to 38% compared to current standards. We also identified opportunities to further improve inspection efficiency by considering additional pathway characteristics (i.e., overdispersion parameters, zero inflation, relative risk, sampling cost, detectability) and developed solutions for these more complex scenarios. We anticipate our result will mitigate biological invasion risk with significant implications for biodiversity conservation, food security, and economies worldwide.</p>","PeriodicalId":21472,"journal":{"name":"Risk Analysis","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139707767","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-01Epub Date: 2024-02-14DOI: 10.1111/risa.14282
Jing Zeng, Hongyu Duan, Zhonglin Zhou, Jingyan Song
Although waste-to-energy (WtE) incineration projects have boosted the economic effectiveness of the waste management system, locals frequently view them with suspicion, opposition, or even outright rejection because of potential environmental and health risks. In this study, by incorporating two additional variables, namely, objective knowledge and benefit perceptions, the planned risk information seeking model (PRISM) was extended in the context of the WtE incineration project. A total of 1726 respondents were interviewed in person. Results from the structural equation modeling indicated that the extended PRISM achieved excellent fits, suggesting the generalizability of the model in these contexts. Objective knowledge and subjective knowledge have a positive and direct impact on perceived knowledge insufficiency. The impact of benefit and risk perceptions on worry is the opposite, with risk perceptions having a greater direct effect on worry. However, compared with risk perceptions, the direct effect that benefit perceptions have on perceived knowledge insufficiency is stronger. Additionally, perceived knowledge insufficiency partially or completely mediates the relationships between all antecedents, except perceived seeking control, and information-seeking intention. This study suggests that risk managers carefully consider how to establish powerful communication tactics to shape these antecedents when introducing a "not in my backyard" project to the local community.
{"title":"Seeking information about waste-to-energy incineration projects: The role of objective knowledge and benefit perceptions in an extended PRISM.","authors":"Jing Zeng, Hongyu Duan, Zhonglin Zhou, Jingyan Song","doi":"10.1111/risa.14282","DOIUrl":"10.1111/risa.14282","url":null,"abstract":"<p><p>Although waste-to-energy (WtE) incineration projects have boosted the economic effectiveness of the waste management system, locals frequently view them with suspicion, opposition, or even outright rejection because of potential environmental and health risks. In this study, by incorporating two additional variables, namely, objective knowledge and benefit perceptions, the planned risk information seeking model (PRISM) was extended in the context of the WtE incineration project. A total of 1726 respondents were interviewed in person. Results from the structural equation modeling indicated that the extended PRISM achieved excellent fits, suggesting the generalizability of the model in these contexts. Objective knowledge and subjective knowledge have a positive and direct impact on perceived knowledge insufficiency. The impact of benefit and risk perceptions on worry is the opposite, with risk perceptions having a greater direct effect on worry. However, compared with risk perceptions, the direct effect that benefit perceptions have on perceived knowledge insufficiency is stronger. Additionally, perceived knowledge insufficiency partially or completely mediates the relationships between all antecedents, except perceived seeking control, and information-seeking intention. This study suggests that risk managers carefully consider how to establish powerful communication tactics to shape these antecedents when introducing a \"not in my backyard\" project to the local community.</p>","PeriodicalId":21472,"journal":{"name":"Risk Analysis","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139736060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-01Epub Date: 2024-02-14DOI: 10.1111/risa.14280
Michael T French, Gulcin Gumus
Although they comprise a relatively small subset of all traffic deaths, hit-and-run (HR) fatalities are both contemptible and preventable. We analyze longitudinal data from 1982 to 2008 to examine the effects of blood alcohol concentration (BAC) laws on HR traffic fatalities. Our results suggest that lower BAC limits may have an unintended consequence of increasing HR fatalities, whereas a similar effect is absent for non-HR fatalities. Specifically, we find that the adoption of a 0.08 BAC limit is associated to an 8.3% increase in HR fatalities. This unintended effect is more pronounced in urban areas and during weekends, which are typical settings for HR incidents.
{"title":"Hit-and-run or hit-and-stay? Unintended effects of a stricter BAC limit.","authors":"Michael T French, Gulcin Gumus","doi":"10.1111/risa.14280","DOIUrl":"10.1111/risa.14280","url":null,"abstract":"<p><p>Although they comprise a relatively small subset of all traffic deaths, hit-and-run (HR) fatalities are both contemptible and preventable. We analyze longitudinal data from 1982 to 2008 to examine the effects of blood alcohol concentration (BAC) laws on HR traffic fatalities. Our results suggest that lower BAC limits may have an unintended consequence of increasing HR fatalities, whereas a similar effect is absent for non-HR fatalities. Specifically, we find that the adoption of a 0.08 BAC limit is associated to an 8.3% increase in HR fatalities. This unintended effect is more pronounced in urban areas and during weekends, which are typical settings for HR incidents.</p>","PeriodicalId":21472,"journal":{"name":"Risk Analysis","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139736059","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-01Epub Date: 2024-06-23DOI: 10.1111/risa.14350
Michael R Greenberg, Karen W Lowrie
{"title":"John D. Graham: Making sense of risk.","authors":"Michael R Greenberg, Karen W Lowrie","doi":"10.1111/risa.14350","DOIUrl":"10.1111/risa.14350","url":null,"abstract":"","PeriodicalId":21472,"journal":{"name":"Risk Analysis","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141459054","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Limited access to food stores is often linked to higher health risks and lower community resilience. Socially vulnerable populations experience persistent disparities in equitable food store access. However, little research has been done to examine how people's access to food stores is affected by natural disasters. Previous studies mainly focus on examining potential access using the travel distance to the nearest food store, which often falls short of capturing the actual access of people. Therefore, to fill this gap, this paper incorporates human mobility patterns into the measure of actual access, leveraging large-scale mobile phone data. Specifically, we propose a novel enhanced two-step floating catchment area method with travel preferences (E2SFCA-TP) to measure accessibility, which extends the traditional E2SFCA model by integrating actual human mobility behaviors. We then analyze people's actual access to grocery and convenience stores across both space and time under the devastating winter storm Uri in Harris County, Texas. Our results highlight the value of using human mobility patterns to better reflect people's actual access behaviors. The proposed E2SFCA-TP measure is more capable of capturing mobility variations in people's access, compared with the traditional E2SFCA measure. This paper provides insights into food store access across space and time, which could aid decision making in resource allocation to enhance accessibility and mitigate the risk of food insecurity in underserved areas.
{"title":"An integrated approach to analyze equitable access to food stores under disasters from human mobility patterns.","authors":"Zhiyuan Wei, Sayanti Mukherjee","doi":"10.1111/risa.16873","DOIUrl":"https://doi.org/10.1111/risa.16873","url":null,"abstract":"<p><p>Limited access to food stores is often linked to higher health risks and lower community resilience. Socially vulnerable populations experience persistent disparities in equitable food store access. However, little research has been done to examine how people's access to food stores is affected by natural disasters. Previous studies mainly focus on examining potential access using the travel distance to the nearest food store, which often falls short of capturing the actual access of people. Therefore, to fill this gap, this paper incorporates human mobility patterns into the measure of actual access, leveraging large-scale mobile phone data. Specifically, we propose a novel enhanced two-step floating catchment area method with travel preferences (E2SFCA-TP) to measure accessibility, which extends the traditional E2SFCA model by integrating actual human mobility behaviors. We then analyze people's actual access to grocery and convenience stores across both space and time under the devastating winter storm Uri in Harris County, Texas. Our results highlight the value of using human mobility patterns to better reflect people's actual access behaviors. The proposed E2SFCA-TP measure is more capable of capturing mobility variations in people's access, compared with the traditional E2SFCA measure. This paper provides insights into food store access across space and time, which could aid decision making in resource allocation to enhance accessibility and mitigate the risk of food insecurity in underserved areas.</p>","PeriodicalId":21472,"journal":{"name":"Risk Analysis","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141793382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We constructed a rapid infection risk assessment model for contacts of COVID-19. The improved Wells-Riley model was used to estimate the probability of infection for contacts of COVID-19 in the same place and evaluate their risk grades. We used COVID-19 outbreaks that were documented to validate the accuracy of the model. We analyzed the relationship between controllable factors and infection probability and constructed common scenarios to analyze the infection risk of contacts in different scenarios. The model showed the robustness of the fitting (mean relative error = 5.89%, mean absolute error = 2.03%, root mean squared error = 2.03%, R2 = 0.991). We found that improving ventilation from poorly ventilated to naturally ventilated and wearing masks can reduce the probability of infection by about two times. Contacts in places of light activity, loud talking or singing, and heavy exercise, oral breathing (e.g., gyms, KTV, choirs) were at higher risk of infection. The model constructed in this study can quickly and accurately assess the infection risk grades of COVID-19 contacts. Simply opening doors and windows for ventilation can significantly reduce the risk of infection in certain places. The places of light activity, loud talking or singing, and heavy exercise, oral breathing, should pay more attention to prevent and control transmission of the epidemic.
{"title":"Risk assessment of infection of COVID-19 contacts based on scenario simulation.","authors":"Wei-Wen Zhang, Yan-Ran Huang, Yu-Yuan Wang, Ze-Xi Lu, Jia-Lin Sun, Ming-Xia Jing","doi":"10.1111/risa.15103","DOIUrl":"https://doi.org/10.1111/risa.15103","url":null,"abstract":"<p><p>We constructed a rapid infection risk assessment model for contacts of COVID-19. The improved Wells-Riley model was used to estimate the probability of infection for contacts of COVID-19 in the same place and evaluate their risk grades. We used COVID-19 outbreaks that were documented to validate the accuracy of the model. We analyzed the relationship between controllable factors and infection probability and constructed common scenarios to analyze the infection risk of contacts in different scenarios. The model showed the robustness of the fitting (mean relative error = 5.89%, mean absolute error = 2.03%, root mean squared error = 2.03%, R<sup>2</sup> = 0.991). We found that improving ventilation from poorly ventilated to naturally ventilated and wearing masks can reduce the probability of infection by about two times. Contacts in places of light activity, loud talking or singing, and heavy exercise, oral breathing (e.g., gyms, KTV, choirs) were at higher risk of infection. The model constructed in this study can quickly and accurately assess the infection risk grades of COVID-19 contacts. Simply opening doors and windows for ventilation can significantly reduce the risk of infection in certain places. The places of light activity, loud talking or singing, and heavy exercise, oral breathing, should pay more attention to prevent and control transmission of the epidemic.</p>","PeriodicalId":21472,"journal":{"name":"Risk Analysis","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141793383","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Risk science is the most updated and justified knowledge-in the form of concepts, principles, approaches, methods, and models-for understanding, assessing, characterizing, communicating, and handling risk, with applications. It is also about the practice that gives us this knowledge. It is commonly stated that risk science is politically neutral. This perspective article discusses this assertion by scrutinizing the relationship between risk science and politics. In particular, it looks into the position of The Society for Risk Analysis (SRA) on this matter. Using some current risk problems as illustrating examples, including COVID-19 and climate change risk, the article argues that defending the political neutrality aim is paramount, especially with today's increasingly divisive political landscape.
{"title":"Risk science and politics: What is and should be the relationship?","authors":"Terje Aven","doi":"10.1111/risa.16558","DOIUrl":"https://doi.org/10.1111/risa.16558","url":null,"abstract":"<p><p>Risk science is the most updated and justified knowledge-in the form of concepts, principles, approaches, methods, and models-for understanding, assessing, characterizing, communicating, and handling risk, with applications. It is also about the practice that gives us this knowledge. It is commonly stated that risk science is politically neutral. This perspective article discusses this assertion by scrutinizing the relationship between risk science and politics. In particular, it looks into the position of The Society for Risk Analysis (SRA) on this matter. Using some current risk problems as illustrating examples, including COVID-19 and climate change risk, the article argues that defending the political neutrality aim is paramount, especially with today's increasingly divisive political landscape.</p>","PeriodicalId":21472,"journal":{"name":"Risk Analysis","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141793297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Digital twins have become a popular and widely used tool for assessing risk and resilience, particularly as they have increased in the fidelity and accuracy of their representation of real-world systems. Although digital twins provide the ability to experiment on and assess risks to and from a system without damaging the real-world system, they pose potentially significant security risks. For example, if a digital twin of a power system has sufficient accuracy to allow loss of electrical power service due to a natural hazard to be estimated at the address level with a high degree of accuracy, what prevents someone wishing to lead to disruption at this same building from using the model to solve the inverse problem to determine which parts of the power system should be attacked to maximize the likelihood of loss of service to the target facility? This perspective article discusses the benefits and risks of digital twins and argues that more attention needs to be paid to the risks posed by digital twins.
{"title":"Digital twins as a security risk?","authors":"Seth Guikema, Roger Flage","doi":"10.1111/risa.15749","DOIUrl":"https://doi.org/10.1111/risa.15749","url":null,"abstract":"<p><p>Digital twins have become a popular and widely used tool for assessing risk and resilience, particularly as they have increased in the fidelity and accuracy of their representation of real-world systems. Although digital twins provide the ability to experiment on and assess risks to and from a system without damaging the real-world system, they pose potentially significant security risks. For example, if a digital twin of a power system has sufficient accuracy to allow loss of electrical power service due to a natural hazard to be estimated at the address level with a high degree of accuracy, what prevents someone wishing to lead to disruption at this same building from using the model to solve the inverse problem to determine which parts of the power system should be attacked to maximize the likelihood of loss of service to the target facility? This perspective article discusses the benefits and risks of digital twins and argues that more attention needs to be paid to the risks posed by digital twins.</p>","PeriodicalId":21472,"journal":{"name":"Risk Analysis","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-07-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141789012","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
As extreme weather events like floods and storms continue to increase, it is crucial to examine the degree to which various disaster preparedness and mitigation investments can lower these risks. In this research, we empirically examine the effects of multiple federal disaster aid programs on reducing subsequent flood- and storm-related damages across US coastal states. Our analysis distinguishes aid programs and their funded projects targeting different emergency management functions, including preparedness, nonstructural and structural mitigation, emergency response and protective measures, and rehabilitation of public infrastructure. We construct panel data of more than 1800 US counties over the years 2000-2019 and estimate a fixed-effects model with time-varying county-level socioeconomic and demographic characteristics. We find that disaster aid generally helps mitigate property damages, although this loss-reduction effect varies by program. Among all aid programs, the Emergency Management Performance Grant results in the largest reduction of future flood damages. The Public Assistance grants supporting emergency work are also found to exert a strong effect on risk reduction. We also find that the impacts of disaster aid are higher in coastal counties. Our study is one of the first few examining the resilience implication of disaster aid in coastal counties, and our results underscore the importance of investing in capacity building, contingency planning, and consistency in maintenance.
{"title":"Estimating the loss-reduction effects of disaster preparedness and mitigation: An empirical study of US coastal states.","authors":"Qing Miao, Meri Davlasheridze","doi":"10.1111/risa.16111","DOIUrl":"10.1111/risa.16111","url":null,"abstract":"<p><p>As extreme weather events like floods and storms continue to increase, it is crucial to examine the degree to which various disaster preparedness and mitigation investments can lower these risks. In this research, we empirically examine the effects of multiple federal disaster aid programs on reducing subsequent flood- and storm-related damages across US coastal states. Our analysis distinguishes aid programs and their funded projects targeting different emergency management functions, including preparedness, nonstructural and structural mitigation, emergency response and protective measures, and rehabilitation of public infrastructure. We construct panel data of more than 1800 US counties over the years 2000-2019 and estimate a fixed-effects model with time-varying county-level socioeconomic and demographic characteristics. We find that disaster aid generally helps mitigate property damages, although this loss-reduction effect varies by program. Among all aid programs, the Emergency Management Performance Grant results in the largest reduction of future flood damages. The Public Assistance grants supporting emergency work are also found to exert a strong effect on risk reduction. We also find that the impacts of disaster aid are higher in coastal counties. Our study is one of the first few examining the resilience implication of disaster aid in coastal counties, and our results underscore the importance of investing in capacity building, contingency planning, and consistency in maintenance.</p>","PeriodicalId":21472,"journal":{"name":"Risk Analysis","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141767218","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Advances in machine learning (ML) have led to applications in safety-critical domains, including security, defense, and healthcare. These ML models are confronted with dynamically changing and actively hostile conditions characteristic of real-world applications, requiring systems incorporating ML to be reliable and resilient. Many studies propose techniques to improve the robustness of ML algorithms. However, fewer consider quantitative techniques to assess changes in the reliability and resilience of these systems over time. To address this gap, this study demonstrates how to collect relevant data during the training and testing of ML suitable for the application of software reliability, with and without covariates, and resilience models and the subsequent interpretation of these analyses. The proposed approach promotes quantitative risk assessment of ML technologies, providing the ability to track and predict degradation and improvement in the ML model performance and assisting ML and system engineers with an objective approach to compare the relative effectiveness of alternative training and testing methods. The approach is illustrated in the context of an image recognition model, which is subjected to two generative adversarial attacks and then iteratively retrained to improve the system's performance. Our results indicate that software reliability models incorporating covariates characterized the misclassification discovery process more accurately than models without covariates. Moreover, the resilience model based on multiple linear regression incorporating interactions between covariates tracks and predicts degradation and recovery of performance best. Thus, software reliability and resilience models offer rigorous quantitative assurance methods for ML-enabled systems and processes.
机器学习(ML)技术的进步已在安全关键领域(包括安全、国防和医疗保健)得到应用。这些 ML 模型面临着真实世界应用中动态变化和积极敌对的条件,要求包含 ML 的系统具有可靠性和弹性。许多研究提出了提高 ML 算法鲁棒性的技术。然而,考虑采用定量技术来评估这些系统的可靠性和复原力随时间推移而发生的变化的研究较少。为了弥补这一不足,本研究展示了如何在适合应用软件可靠性(有协变量和无协变量)和弹性模型的 ML 训练和测试过程中收集相关数据,以及随后对这些分析的解释。所提出的方法促进了对 ML 技术的定量风险评估,提供了跟踪和预测 ML 模型性能下降和提高的能力,并协助 ML 和系统工程师采用客观的方法来比较其他训练和测试方法的相对有效性。该方法以图像识别模型为例进行说明,该模型受到两种生成性对抗攻击,然后进行迭代再训练以提高系统性能。我们的研究结果表明,与不包含协变量的模型相比,包含协变量的软件可靠性模型能更准确地描述误分类发现过程。此外,基于多元线性回归的弹性模型结合了协变量之间的交互作用,能最好地跟踪和预测性能的下降和恢复。因此,软件可靠性和弹性模型为支持 ML 的系统和流程提供了严格的定量保证方法。
{"title":"Quantitative assessment of machine learning reliability and resilience.","authors":"Zakaria Faddi, Karen da Mata, Priscila Silva, Vidhyashree Nagaraju, Susmita Ghosh, Gokhan Kul, Lance Fiondella","doi":"10.1111/risa.14666","DOIUrl":"https://doi.org/10.1111/risa.14666","url":null,"abstract":"<p><p>Advances in machine learning (ML) have led to applications in safety-critical domains, including security, defense, and healthcare. These ML models are confronted with dynamically changing and actively hostile conditions characteristic of real-world applications, requiring systems incorporating ML to be reliable and resilient. Many studies propose techniques to improve the robustness of ML algorithms. However, fewer consider quantitative techniques to assess changes in the reliability and resilience of these systems over time. To address this gap, this study demonstrates how to collect relevant data during the training and testing of ML suitable for the application of software reliability, with and without covariates, and resilience models and the subsequent interpretation of these analyses. The proposed approach promotes quantitative risk assessment of ML technologies, providing the ability to track and predict degradation and improvement in the ML model performance and assisting ML and system engineers with an objective approach to compare the relative effectiveness of alternative training and testing methods. The approach is illustrated in the context of an image recognition model, which is subjected to two generative adversarial attacks and then iteratively retrained to improve the system's performance. Our results indicate that software reliability models incorporating covariates characterized the misclassification discovery process more accurately than models without covariates. Moreover, the resilience model based on multiple linear regression incorporating interactions between covariates tracks and predicts degradation and recovery of performance best. Thus, software reliability and resilience models offer rigorous quantitative assurance methods for ML-enabled systems and processes.</p>","PeriodicalId":21472,"journal":{"name":"Risk Analysis","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2024-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141752625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}