Optimization driven data mining and credit scoring

Abstract

An optimization tree approach to the mining of very extensive and complex databases for performance optimizing opportunities is described. This methodology is based on a combination of three innovations: a data management system designed explicitly for data intensive computing; a distributed algorithm for growing classification and regression trees (CART); and a tree based stochastic programming paradigm for the selection of control attributes to optimize a specified objective function. This methodology provides a general technique for optimization in financial applications that is scalable as the number of objects in the database and as the number of attributes per object grow. This scalability allows for a complete data driven analysis of large scale data sets, without the need to restrict attention to sparsely sampled data sets that limits previous methods.