R2 versus r2

Abstract

Abstract Cost estimating relationships (CERs) with multiplicative-error assumptions are commonly used in cost analysis. Consequently, we need to apply appropriate statistical measures to evaluate a CER's quality when developing multiplicative error CERs such as minimum-unbiased-percentage error (MUPE) and minimum-percentage error under zero-percentage bias (ZMPE) CERs. Generalized R-squared (GRSQ, also denoted by the symbol r2) is commonly used for measuring the quality of a nonlinear CER. GRSQ is defined as the square of Pearson's correlation coefficient between the actual observations and CER-predicted values (see Young 1992). Many statistical analysts believe GRSQ is an appropriate analog to measure the proportion of variation explained by a nonlinear CER (see Nguyen and Lozzi, 1994), including MUPE and ZMPE CERs; some even use it to measure the appropriateness of shape of a CER. Adjusted R2 in unit space is a frequently used alternative measure for CER quality. This statistic translates the sum of squares due to error (SSE) from the absolute scale to the relative scale. This metric is used to measure how well the CER-predicted costs match the actual data set, adjusting for the number of estimated coefficients used in the model. There have been academic concerns over the years about the relevance of using Adjusted R2 and Pearson's r2. For example, some insist that Adjusted R2, calculated by the traditional formula, has no value as a metric except for ordinary least squares (OLS); others argue that Pearson's r2 does not measure how well the estimate matches database actuals for nonlinear CERs. This article discusses these concerns and examines the properties of these statistics, along with pros and cons of using each for CER development. In addition, this article proposes 1) a modified Adjusted R2 for evaluating MUPE and ZMPE CERs and 2) a modified GRSQ to account for degrees of freedom (DF).