The Accountability Illusion

Take away all the jargon, emotion, envy, confusion, and embarrassment
and much of the No Child Left Behind debate comes down to this: Which
schools are good, which are bad, and does NCLB do a decent job of
telling the difference?

The short answer, provided by a major new study from Fordham and the Kingsbury Center at the Northwest Evaluation Association, is no, not by a mile.

The analysis is complex and the report is long but its premise is
simple: Take a set of real schools, pretend that we can drag them across
the map and drop them down in various states, and see how many would
make "Adequate Yearly Progress" (AYP) in each place. If the U.S. had
something akin to a shared notion of what it means to be a good or bad
school, we wouldn't see a whole lot of variation.

Yet we found nearly the opposite. In a few of the 28 states we studied (e.g, Wisconsin, Arizona), almost all of the elementary schools in our sample made AYP, while in other jurisdictions (e.g., Massachusetts, Nevada), almost none
did. Putting it bluntly, most of the schools in our sample would be
considered failures in some states but perfectly okay, even
praiseworthy, in others. These are the same exact schools, mind you.
Same students. Same teachers. Same achievement. What's
different--sometimes drastically different--are the arcane AYP rules
that vary from state to state.

Such variation surely existed before NCLB. Does it matter that it exists now?

Yes, for three reasons. First, it surely demoralizes educators
(and probably students, parents, taxpayers, etc.) to know that their
very own schools, deemed "in need of improvement" under NCLB, would be
considered acceptable, even praiseworthy, if located elsewhere. Such
capriciousness not only breeds cynicism toward NCLB; it undermines the
idea of accountability itself--and certainly retards efforts to
revitalize horrific schools and boost low-performing pupils.

Second, what drives the state-to-state variation in AYP results isn't
principled differences about what it means to be a good school.
Instead, obscure, deep-in-the-bureaucracy decisions around "cut scores,"
"annual measurable objectives," "minimum n sizes," and "confidence
intervals" create these discrepant and discordant outcomes. We'd mind
less if the variation were based on things that truly matter: whether
schools are judged on their progress over time, for example, rather than
each year's absolute performance; whether they're helping all students
make gains or just those below a fixed bar; whether determinations hinge
solely on reading and math or include other core subjects like science
and history; and so forth. Those would be legitimate reasons for
discrepancies, issues worth arguing over. But that's not what we're
seeing here. Rather, we witness state education departments going
through subtle machinations to create outcomes that they judge sensible,
or at least politically saleable. 

Third, NCLB created the trappings of a national accountability
system. Variation around school ratings was okay when states also set
the penalties (if any) for schools that didn't make the grade. But now
every state operates under a federal mandate to offer "public school
choice" and "supplemental services" (tutoring) to children stuck in
"failing schools," and under further mandates to overhaul
change-resistant schools. They're told in which year to intervene in
which way. This cascade of sanctions is the same everywhere in America.
The man in the street surely believes that it's a uniform accountability
system. Yet we see that it's not. All those sanctions and
interventions, uniform as they are, are triggered by AYP systems that
couldn't be more different. At best, this is a disconnect. At worst,
it's chaos.

What to do? Some imply that NCLB might be "repealed." Not
likely. Another option would be to nationalize and standardize
everything. Perhaps that's not as unthinkable as it might once have
been, given the astounding growth of federal authority these past few
months. We could move toward a national school system. Arne Duncan would
determine each year which of America's 100,000 public schools makes the
grade and what to do about those that don't.

But that isn't what we'd recommend. That approach would push Uncle
Sam ever deeper into the impossible task of running schools and turning
around those that fail.

We picture a very different approach to NCLB 2.0. Uncle Sam should
create incentives for states to sign on to common standards (such as
those now under development by the National Governors Association and
other groups) and tests. Ensure that the tests are rigorous and
comprehensive. Publish their results for every school in the country,
with data sliced every which way--by race, income, disability status,
progress over time, etc. And then stop.

That's right, stop. That's the end of Uncle Sam's role under NCLB 2.0.

Return to a regimen where each state then decides what to deduce from
those test data and what to do about schools whose results don't
satisfy the state's definition of success. Civil rights groups and
others (e.g. Education Trust)
that don't like the states' judgments can create their own school
ratings, using the same uniform national data, accessible and
transparent to all. So could private organizations such as
GreatSchools.net. We could reopen the debate about what it means to be a
good or bad school. And then it would be up to states and communities
to do something about schools that aren't making the grade.

Such "Reform Realism"
foregoes the utopian rhetoric of the NCLB era. It admits that Uncle Sam
cannot actually ensure that every American child will get a world-class
education. But what this strategy would do is ensure greater
transparency around student results--something our new study shows is
hard to come by--based on assessments that are rigorous, credible and
uniform throughout the land. And it would reinforce the idea that states
remain responsible for K-12 education and must make decisions that
their own citizens will accept. Best of all, it would end the
gamesmanship that has characterized the federal-state relationship since
Congress passed NCLB.

"Study: Academic standards vary across states," by Libby Quaid, Associated Press, February 19, 2009

More By Author