COMBSS: best subset selection via continuous optimization

IF 1.6 2区数学 Q2 COMPUTER SCIENCE, THEORY & METHODS Statistics and Computing Pub Date : 2024-02-12 DOI:10.1007/s11222-024-10387-8

{"title":"COMBSS: best subset selection via continuous optimization","authors":"","doi":"10.1007/s11222-024-10387-8","DOIUrl":null,"url":null,"abstract":"<h3>Abstract</h3> <p>The problem of best subset selection in linear regression is considered with the aim to find a fixed size subset of features that best fits the response. This is particularly challenging when the total available number of features is very large compared to the number of data samples. Existing optimal methods for solving this problem tend to be slow while fast methods tend to have low accuracy. Ideally, new methods perform best subset selection faster than existing optimal methods but with comparable accuracy, or, being more accurate than methods of comparable computational speed. Here, we propose a novel continuous optimization method that identifies a subset solution path, a small set of models of varying size, that consists of candidates for the single best subset of features, that is optimal in a specific sense in linear regression. Our method turns out to be fast, making the best subset selection possible when the number of features is well in excess of thousands. Because of the outstanding overall performance, framing the best subset selection challenge as a continuous optimization problem opens new research directions for feature extraction for a large variety of regression models. </p>","PeriodicalId":22058,"journal":{"name":"Statistics and Computing","volume":"16 1","pages":""},"PeriodicalIF":1.6000,"publicationDate":"2024-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Statistics and Computing","FirstCategoryId":"100","ListUrlMain":"https://doi.org/10.1007/s11222-024-10387-8","RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, THEORY & METHODS","Score":null,"Total":0}

引用次数: 0

Abstract

The problem of best subset selection in linear regression is considered with the aim to find a fixed size subset of features that best fits the response. This is particularly challenging when the total available number of features is very large compared to the number of data samples. Existing optimal methods for solving this problem tend to be slow while fast methods tend to have low accuracy. Ideally, new methods perform best subset selection faster than existing optimal methods but with comparable accuracy, or, being more accurate than methods of comparable computational speed. Here, we propose a novel continuous optimization method that identifies a subset solution path, a small set of models of varying size, that consists of candidates for the single best subset of features, that is optimal in a specific sense in linear regression. Our method turns out to be fast, making the best subset selection possible when the number of features is well in excess of thousands. Because of the outstanding overall performance, framing the best subset selection challenge as a continuous optimization problem opens new research directions for feature extraction for a large variety of regression models.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

COMBSS：通过连续优化选择最佳子集

摘要在线性回归中考虑最佳子集选择问题的目的是找到最适合响应的固定大小的特征子集。当可用的特征总数与数据样本数相比非常大时，这个问题尤其具有挑战性。解决这一问题的现有最优方法往往速度较慢，而快速方法往往准确率较低。理想的情况是，新方法比现有的最优方法更快地完成最佳子集选择，但准确度相当，或者比计算速度相当的方法更准确。在这里，我们提出了一种新颖的连续优化方法，该方法能确定子集求解路径，即大小不一的小模型集，该模型集由候选的单一最佳特征子集组成，在线性回归的特定意义上是最优的。我们的方法证明是快速的，当特征数量远远超过数千个时，最佳子集选择是可能的。由于我们的方法具有出色的整体性能，因此将最佳子集选择挑战作为一个连续优化问题，为各种回归模型的特征提取开辟了新的研究方向。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Statistics and Computing 数学-计算机：理论方法

CiteScore

3.20

自引率

4.50%

发文量

审稿时长

6-12 weeks

期刊介绍： Statistics and Computing is a bi-monthly refereed journal which publishes papers covering the range of the interface between the statistical and computing sciences. In particular, it addresses the use of statistical concepts in computing science, for example in machine learning, computer vision and data analytics, as well as the use of computers in data modelling, prediction and analysis. Specific topics which are covered include: techniques for evaluating analytically intractable problems such as bootstrap resampling, Markov chain Monte Carlo, sequential Monte Carlo, approximate Bayesian computation, search and optimization methods, stochastic simulation and Monte Carlo, graphics, computer environments, statistical approaches to software errors, information retrieval, machine learning, statistics of databases and database technology, huge data sets and big data analytics, computer algebra, graphical models, image processing, tomography, inverse problems and uncertainty quantification. In addition, the journal contains original research reports, authoritative review papers, discussed papers, and occasional special issues on particular topics or carrying proceedings of relevant conferences. Statistics and Computing also publishes book review and software review sections.

期刊最新文献

funBIalign: a hierachical algorithm for functional motif discovery based on mean squared residue scores. Accelerated failure time models with error-prone response and nonlinear covariates Sequential model identification with reversible jump ensemble data assimilation method Hidden Markov models for multivariate panel data Shrinkage for extreme partial least-squares