Pub Date : 2024-01-02DOI: 10.1186/s42467-023-00015-y
Florian Heidecker, Maarten Bieshaar, Bernhard Sick
Applications using machine learning (ML), such as highly autonomous driving, depend highly on the performance of the ML model. The data amount and quality used for model training and validation are crucial. If the model cannot detect and interpret a new, rare, or perhaps dangerous situation, often referred to as a corner case, we will likely blame the data for not being good enough or too small in number. However, the implemented ML model and its associated architecture also influence the behavior. Therefore, the occurrence of prediction errors resulting from the ML model itself is not surprising. This work addresses a corner case definition from an ML model’s perspective to determine which aspects must be considered. To achieve this goal, we present an overview of properties for corner cases that are beneficial for the description, explanation, reproduction, or synthetic generation of corner cases. To define ML corner cases, we review different considerations in the literature and summarize them in a general description and mathematical formulation, whereby the expected relevance-weighted loss is the key to distinguishing corner cases from common data. Moreover, we show how to operationalize the corner case characteristics to determine the value of a corner case. To conclude, we present the extended taxonomy for ML corner cases by adding the input, model, and deployment levels, considering the influence of the corner case properties.
使用机器学习（ML）的应用，如高度自动驾驶，在很大程度上取决于 ML 模型的性能。用于模型训练和验证的数据量和质量至关重要。如果模型无法检测和解释一种新的、罕见的或可能存在危险的情况（通常被称为 "角落情况"），我们很可能会责怪数据不够好或数量太少。然而，实施的 ML 模型及其相关架构也会影响行为。因此，出现由 ML 模型本身导致的预测错误并不奇怪。这项工作从 ML 模型的角度出发，对角情况进行定义，以确定必须考虑哪些方面。为了实现这一目标，我们概述了有利于描述、解释、再现或合成生成角案例的角案例属性。为了定义 ML 角案例，我们回顾了文献中的不同考虑因素，并将其总结为一般描述和数学表述，其中预期相关性加权损失是将角案例与普通数据区分开来的关键。此外，我们还展示了如何将角案特征操作化，以确定角案的价值。最后，考虑到角案例特性的影响，我们通过添加输入、模型和部署级别，提出了 ML 角案例扩展分类法。
{"title":"Corner cases in machine learning processes","authors":"Florian Heidecker, Maarten Bieshaar, Bernhard Sick","doi":"10.1186/s42467-023-00015-y","DOIUrl":"https://doi.org/10.1186/s42467-023-00015-y","url":null,"abstract":"Applications using machine learning (ML), such as highly autonomous driving, depend highly on the performance of the ML model. The data amount and quality used for model training and validation are crucial. If the model cannot detect and interpret a new, rare, or perhaps dangerous situation, often referred to as a corner case, we will likely blame the data for not being good enough or too small in number. However, the implemented ML model and its associated architecture also influence the behavior. Therefore, the occurrence of prediction errors resulting from the ML model itself is not surprising. This work addresses a corner case definition from an ML model’s perspective to determine which aspects must be considered. To achieve this goal, we present an overview of properties for corner cases that are beneficial for the description, explanation, reproduction, or synthetic generation of corner cases. To define ML corner cases, we review different considerations in the literature and summarize them in a general description and mathematical formulation, whereby the expected relevance-weighted loss is the key to distinguishing corner cases from common data. Moreover, we show how to operationalize the corner case characteristics to determine the value of a corner case. To conclude, we present the extended taxonomy for ML corner cases by adding the input, model, and deployment levels, considering the influence of the corner case properties.","PeriodicalId":510896,"journal":{"name":"AI Perspectives","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139080325","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：