The Business Model.

Abstract

LIKE THE MAKERS OF HOT DOGS, PSYCHOMETRICIANS, economists, and other testing experts know too well what goes into the creation of achievement tests. Their intimate knowledge of the technical difficulties involved in measuring student achievement makes a number of these testing experts some of the most vocal (and persuasive) opponents of testing. But the flaws in techniques like value-added assessment do not automatically lead to the conclusion that those techniques shouldn't be used to hold educators accountable. Testing may be imperfect, but the alternative--the old system, which allowed us to know very little about the performance of educators--is far, far worse. To be sure, many of the technical criticisms of value-added testing are correct. It's true that there is more random error in measuring gains in test scores than in measuring the level of test scores. It's true that there is some uncertainty as to whether gains in one area of the test scale are equal to gains at another point in the test scale. And it's true that factors besides the quality of the schools can influence the gains that students achieve. But, on balance, these downsides hardly outweigh the benefits to be reaped from being able to measure and reward productivity in education. Consider what is likely to continue to happen in education without high-stakes value-added assessment. Unless productivity is measured, however imperfectly, it is not possible to reward teachers, administrators, and schools that contribute most to student learning. If we do nor reward productivity, we are unlikely to encourage it. If we do not encourage it, we should not expect more of it. In fact, this is precisely what has been happening in U.S. education during the past few decades. Between 1961 and 2000, spending on education tripled after adjusting for inflation, from $2,360 to $7,086 per pupil. During that time, student performance, as measured by scores on the National Assessment of Educational Progress (NAEP) and high-school graduation rates, has remained basically unchanged. Whenever spending triples without any significant improvement in outcomes, there is a serious productivity crisis. Yet U.S. public schools just keep chugging along, resisting serious attempts at reform. Meanwhile, private firms in the United States have been able to achieve steady gains in productivity because the discipline of competition has forced them to adopt systems for measuring and rewarding productivity. Firms that fail to measure and reward productivity lose out to their competitors who do. Moreover, the systems that private companies use to measure and reward productivity are far from flawless. In fact, the challenge of measuring productivity in the private sector is often as great as or greater than in education. Imagine a soft-drink company that wishes to measure and reward the productivity of its sales force. The company might determine bonuses (and even decisions on layoffs) based on its salespeople's success at increasing soda sales in their sales area. Like measuring gains in test scores, measuring increases in soda sales is fraught with potential error. Changes in soda sales could be influenced by a variety of factors other than the sales acumen of an employee. Unusually cold weather in an area, a local economic downturn, or exceptional promotional efforts by competitors could all suppress the soda sales of even a very good salesperson. If data on sales are collected using survey techniques, there is also the possibility of random error attributable to the survey method, just as resting has random error. Moreover, if we are comparing sales increases across geographic areas, it is unclear whether it rakes more skill to sell soda in an area where the market is already saturated than in an area that initially consumes less soda. In short, many of the same technical flaws that critics find in value-added testing also exist in the measurement of increases in soda sales. …