Regression and Effect Size

Back in the July–August issue of 2020, I discussed regression and R values. In this issue, the article titled, “Predictors of Job Satisfaction for People with Visual Impairments,” allows me to continue this discussion. In this article, authors Steverson and Crudden used multiple linear regression to identify predictors of job satisfaction. They also reported effect sizes for their analyses. Previously in this column, I have discussed the importance of reporting effect sizes for different statistical tests (e.g., January– February 2016 and July–August 2019). Although the concept of effect size is the same across all statistical tests (a measure of the size of the effect or difference being studied), how it is calculated is different for each statistical test. In the July–August 2020 Statistical Sidebar, I noted that the statistic η (read as eta squared) can be used as a measure of effect size in regression analyses. Other measures of effect size for regression include R (for the magnitude of the effect of the entire model), f 2 (for the magnitude of the effect of the entire regression model or individual predictors), and rpart (for the magnitude of the effect of individual predictors). The rpart statistic is called the squared semipartial correlation and is the measure used in the article under discussion in this issue. The R statistic, sometimes called the coefficient of determination, is where the main measures of effect size for regression all begin. Many researchers report the R value because it lends itself well to the interpretation of how much variability in the dependent variable is explained by the regression model. However, the use of the f 2 or rpart statistics allows researchers to focus on individual predictor variables. This operation is especially useful in multiple linear regressions where there are several predictor variables in the regression model. For a simple linear regression, where there is one predictor variable and one outcome variable, the R statistic is just the square of the correlation coefficient between the two variables. For more complicated regressions, R is calculated as 1-RSS/TSS where RSS is the sum of squared residuals and TSS is the total sum of squares. Without getting too deep into the details, these two measures relate to how far each datapoint in a dataset lies off of the regression line that minimizes the overall distance from the regression line to all the data points. The R statistic ranges from 0 to 1 where 0.01 is considered a small effect, 0.09 is a medium effect, and 0.25 is a large effect. The f 2 statistic is calculated based on the R statistic through the equation f =Rinc/ 1-Rinc where R 2 inc is the change in the overall R for a regression model when a given predictor variable is added to a group of other predictor variables. A rule of thumb is that for the f 2 statistic, 0.02 is a small effect, 0.15 is a medium effect, and 0.35 is a large effect. The rpart statistic is also calculated by running two regression models and comparing the R statistics for the two models. By adding one predictor variable to the second regression model, the difference in the R statistics for the two models gives the rpart statistic for that predictor variable. In the article under discussion in this issue, values for rpart are given for 21 different variables and the values range from< 0.001 to 0.028. In Table 4, only the largest 5 rpart measures are statistically significant (p < .05). Statistical Sidebar