Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement

Mickey Muldoon

Jesse Rothstein
Princeton University and National Bureau of Economic Research
February 2010 (anticipated)

There's a very serious and scholarly--and to the lay person, nearly
unintelligible--exchange happening between academic economists these
days on the topic of Value-Added Models (VAM), which rate teacher
performance based on student test score gains, rather than snapshots of
achievement. In theory, the idea works as follows: Randomly assign
students to classrooms such that their average test scores are
comparable, and then at the end of the year, give a higher VAM grade to
the teachers whose students' test results rise the most. The problem is
that, in reality, students are not (and should not be) randomly assigned
to teachers--and statistically compensating for this fact turns out to
be enormously tricky. As proof, Rothstein breaks down three real-life
examples of VAM and applies them to a much larger student sample
(approximately 90,000 pupils) than that for which they are typically
used. By doing so, he shows that statistical problems that could be
hidden in acceptable margins of error in a small sample size are
actually larger--and problematic--trends when applied to many more
students. While his peers evaluate and reformulate their models based on
these findings, it is both reassuring and frightening that the topic
has entered the arcana of high economics: reassuring, because there is
honest and rigorous debate happening on behalf of better performance
measures for our schools, and frightening, because someday someone's
going to have to translate these models into terms that teachers and
principals can digest. One thing's for sure: Rothstein's cautious
recommendation to both include observational data as well as VAM scores
in overall teacher evaluation should be taken seriously. Read it here.