Gathering Feedback for Teaching: Combining High-Quality Observations with Student Surveys and Achievement Gains

The first
set of preliminary findings
from the Gates-funded Measures of Effective
Teaching (MET) project generated much conversation—and some
criticism
. This latest report, also preliminary, is not much different.
(Remember that this $45 million project seeks to ferret out, or design, an
optimal teacher-evaluation system through the analysis of student test scores,
surveys, and thousands of hours of classroom observations.) While the first
iteration compared student scores with survey responses, this one analyzes the
predictive strength of five frameworks for classroom observations (think D.C.’s
IMPACT program
for an idea of what they look like). The study finds that,
while each method is positively correlated to pupil achievement (on both state
tests and independent tests), the reliability of observations pales in comparison
to value-added measures (VAM): The reliability of VAM is about double that of a
single observation—from any of the tested measurement systems. Predictive
abilities increase significantly when VAM and student-survey data are combined
with classroom observations—leading the authors to recommend use of multiple
measures when evaluating teachers. In response, Jay
Greene
has again sounded the battle cry. And perhaps rightly so. While the
report’s analyses clearly show that value-added data is the single strongest
predictive factor for student achievement and that adding observations almost
negligibly improves reliability, nowhere do the authors caution policymakers
about the potentially high cost and low yield that come with that addition. In
other words, it is cheaper and almost as reliable to rely entirely on
value-added data. (Gates does remind that classroom observations can help
buttress strong teacher-improvement programs, though—in a way that VAM can’t.)This
report deserves a close look. And expect the debate to be rekindled later this
year when the study’s final report is due to be released.

Gathering Feedback for Teaching: Combining
High-Quality Observations with Student Surveys and Achievement Gains
(Seattle, WA:
Bill and Melinda Gates Foundation, January 2012).

More By Author

Related Articles