{"title":"ARFIS:用于重尾分布回归的自适应稳健模型","authors":"","doi":"10.1016/j.ins.2024.121344","DOIUrl":null,"url":null,"abstract":"<div><p>As heavy-tailed distributions are ubiquitous in many real applications, robust regression has been extensively applied in machine learning and exhibits the superiority in deal with heavy-tailed distribution. Current existing robust methods for regression are based on independence assumption of features and ignore interaction between them, which may lead to poor generalization due to most datasets unsatisfying the assumption. Actually, interaction features formed by the composition of two or more features are particularly helpful to obtain the high-order information in many applications. In this paper, we propose a novel adaptive robust feature interaction selection model for regression with heavy-tailed distributions, termed Adaptive Robust Feature Interaction Selection (ARFIS). Firstly, we consider pairwise feature interaction by augmenting a feature vector with product of features for regression with heavy tailed distribution. Secondly, we propose feature interaction selection models based on quantile loss with different regularizers to learn parameters. The consistency of the proposed model ARFIS is theoretically proven, and an efficient algorithm is presented for solving proposed model. Finally, experimental results on simulation data, UCI datasets and a real-world dataset validate good accuracy, interpretability and robustness of our proposed models.</p></div>","PeriodicalId":51063,"journal":{"name":"Information Sciences","volume":null,"pages":null},"PeriodicalIF":8.1000,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"ARFIS: An adaptive robust model for regression with heavy-tailed distribution\",\"authors\":\"\",\"doi\":\"10.1016/j.ins.2024.121344\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>As heavy-tailed distributions are ubiquitous in many real applications, robust regression has been extensively applied in machine learning and exhibits the superiority in deal with heavy-tailed distribution. Current existing robust methods for regression are based on independence assumption of features and ignore interaction between them, which may lead to poor generalization due to most datasets unsatisfying the assumption. Actually, interaction features formed by the composition of two or more features are particularly helpful to obtain the high-order information in many applications. In this paper, we propose a novel adaptive robust feature interaction selection model for regression with heavy-tailed distributions, termed Adaptive Robust Feature Interaction Selection (ARFIS). Firstly, we consider pairwise feature interaction by augmenting a feature vector with product of features for regression with heavy tailed distribution. Secondly, we propose feature interaction selection models based on quantile loss with different regularizers to learn parameters. The consistency of the proposed model ARFIS is theoretically proven, and an efficient algorithm is presented for solving proposed model. Finally, experimental results on simulation data, UCI datasets and a real-world dataset validate good accuracy, interpretability and robustness of our proposed models.</p></div>\",\"PeriodicalId\":51063,\"journal\":{\"name\":\"Information Sciences\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":8.1000,\"publicationDate\":\"2024-08-20\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Information Sciences\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0020025524012581\",\"RegionNum\":1,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"N/A\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information Sciences","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0020025524012581","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"N/A","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
ARFIS: An adaptive robust model for regression with heavy-tailed distribution
As heavy-tailed distributions are ubiquitous in many real applications, robust regression has been extensively applied in machine learning and exhibits the superiority in deal with heavy-tailed distribution. Current existing robust methods for regression are based on independence assumption of features and ignore interaction between them, which may lead to poor generalization due to most datasets unsatisfying the assumption. Actually, interaction features formed by the composition of two or more features are particularly helpful to obtain the high-order information in many applications. In this paper, we propose a novel adaptive robust feature interaction selection model for regression with heavy-tailed distributions, termed Adaptive Robust Feature Interaction Selection (ARFIS). Firstly, we consider pairwise feature interaction by augmenting a feature vector with product of features for regression with heavy tailed distribution. Secondly, we propose feature interaction selection models based on quantile loss with different regularizers to learn parameters. The consistency of the proposed model ARFIS is theoretically proven, and an efficient algorithm is presented for solving proposed model. Finally, experimental results on simulation data, UCI datasets and a real-world dataset validate good accuracy, interpretability and robustness of our proposed models.
期刊介绍:
Informatics and Computer Science Intelligent Systems Applications is an esteemed international journal that focuses on publishing original and creative research findings in the field of information sciences. We also feature a limited number of timely tutorial and surveying contributions.
Our journal aims to cater to a diverse audience, including researchers, developers, managers, strategic planners, graduate students, and anyone interested in staying up-to-date with cutting-edge research in information science, knowledge engineering, and intelligent systems. While readers are expected to share a common interest in information science, they come from varying backgrounds such as engineering, mathematics, statistics, physics, computer science, cell biology, molecular biology, management science, cognitive science, neurobiology, behavioral sciences, and biochemistry.