Examining the Carnegie Classification Methodology for Research Universities

Abstract

ABSTRACT University ranking is a popular yet controversial endeavor. Most rankings are based on both public data, such as student test scores and retention rates, and proprietary data, such as school reputation as perceived by high school counselors and academic peers. The weights applied to these characteristics to compute the rankings are often determined in a subjective fashion. Of significant importance in the academic field, the Carnegie Classification was developed by the Carnegie Foundation for the Advancement of Teaching. It has been updated approximately every 5 years since 1973, most recently in February 2016. Based on bivariate scores, Carnegie assigns one of three classes (R1/R2/R3) to doctorate-granting universities according to their level of research activity. The Carnegie methodology uses only publicly available data and determines weights via principal component analysis. In this article, we review Carnegie’s stated goals and the extent to which their methodology achieves those goals. In particular, we examine Carnegie’s separation of aggregate and per capita (per tenured/tenure-track faculty member) variables and its use of two separate principal component analyses on each; the resulting bivariate scores are very highly correlated. We propose and evaluate two alternatives and provide a graphical tool for evaluating and comparing the three scenarios.