The Accountability Illusion

Michael J. Petrilli / February 19, 2009

Take away all the jargon, emotion, envy, confusion, and embarrassment and much of the No Child Left Behind debate comes down to this: Which schools are good, which are bad, and does NCLB do a decent job of telling the difference?

The short answer, provided by a major new study from Fordham and the Kingsbury Center at the Northwest Evaluation Association, is no, not by a mile.

The analysis is complex and the report is long but its premise is simple: Take a set of real schools, pretend that we can drag them across the map and drop them down in various states, and see how many would make "Adequate Yearly Progress" (AYP) in each place. If the U.S. had something akin to a shared notion of what it means to be a good or bad school, we wouldn't see a whole lot of variation.

Yet we found nearly the opposite. In a few of the 28 states we studied (e.g, Wisconsin, Arizona), almost all of the elementary schools in our sample made AYP, while in other jurisdictions (e.g., Massachusetts, Nevada), almost none did. Putting it bluntly, most of the schools in our sample would be considered failures in some states but perfectly okay, even praiseworthy, in others. These are the same exact schools, mind you. Same students. Same teachers. Same achievement. What's different--sometimes drastically different--are the arcane AYP rules that vary from state to state.

Such variation surely existed before NCLB.

» Continued

The Accountability Illusion

A dandy from Randi

February 19, 2009

Gadfly tends to give AFT President Randi Weingarten a (deserved) hard time, so when she does something praiseworthy his antennae perk up. So it was this week, when she took a strong stand for national standards in the Washington Post. "[E]very child attending U.S. public schools should be taught to high standards, regardless of where he or she lives," she opined. We agree. But how to get from here to there? An idea: Start with the big cities. Let them secede from their respective state accountability systems and band together under a national umbrella. Big cities already tend to be hot spots for reform and many of their superintendents already favor national standards. Furthermore, Randi's AFT tends to be the union in their systems. With all these players on board, these metropolises could pave the way. So, Randi, what say you to that?

"The Case for National Standards," by Randi Weingarten, Washington Post, February 16, 2009

» Continued

A dandy from Randi

Aversion to conversion

February 19, 2009

Just last week, we learned that Michael "Noah" Bloomberg would pack his ark with four charter-converted Brooklyn Catholic schools. Unfortunately, though not unexpectedly, the seas for this journey are already proving stormy. "There are real concerns about whether this is parochial school education by another name," yelps Donna Lieberman of NY's Civil Liberties Union. Concerns over creaming, lingering religious influence, and curricular issues (specifically teaching sex ed, which is banned in Catholic schools but required by the city in public ones) abound. Even more problematic is a pesky state law banning the conversion of private schools (including those of parochial persuasion) to charter schools. Mayor Bloomberg will need to take his case to Albany to see the law changed. If we could cease the hand-wringing for one moment, we'd encourage New Yorkers to look south to D.C., which fretted its way through many of these problems when it converted 7 Catholic schools to charter last year. The legal challenges are certainly real, but let's give Bloomberg a chance to right the ship and take his case--and D.C.'s precedent--upstate before drowning the plan altogether.

"Hurdles for a Plan to Turn Catholic Classrooms Into Charter Schools," by Javier C. Hernandez, New York Times, February 15, 2009

» Continued

Aversion to conversion

But did he score proficient?

February 19, 2009

At last, veritable proof that test prep pays real-life dividends. Seventeen-year old Geoffrey Stanford approached the Kansas state test just like his teachers told him to: "Every sentence. Every word. Slow down." Unfortunately, the test-makers would do well to heed such advice, for young Geoffrey's fastidiousness uncovered a gaffe on their part: "emission" (of greenhouse gases) was spelled "omission." His reward: 100 Facebook friend requests, emails and texts from strangers, and a cable news appearance. Has this young American everyman--linebacker, IB student, aspiring mechanical engineer--become the Joe the Plumber of the assessment world? Time shall tell. As to the state's error, DOE spokeswoman Karla Denny explained: "I think it's one of those things where the people writing the test were so close to it, they probably just read over it." Gadzooks, does the phrase "a fresh pair of eyes" mean nothing on the prairie?

"Error on state test slips past everyone--except East High student," by Suzanne Perez Tobias, The Wichita Eagle, February 12, 2009

"Blizzard of buzz for student who found error on test," by Suzanne Perez Tobias, The Wichita Eagle, February 14, 2009

» Continued

But did he score proficient?

A-level tan

February 19, 2009

The Brits have taken voc ed to new lows. In an effort to encourage teens to sign up for courses more suited to their abilities, education officials have expanded the number of courses that count toward "league tables," i.e. how schools are evaluated and compared. But it's not just shop and home ec that made it onto the expanded list; now pottery, cake decorating, flower arranging, and fake tanning make the cut, too. A diploma in "tanning treatments," for which students are taught how to operate sun beds and apply a fake tan without streaks, is now worth 45 points--the same as an "A" in an advanced math course. That's like creating Advanced Placement basket weaving and counting the scads of 5s scored on its exam towards the school's overall performance. But more than inflated school attainment measures is the worry that students will neglect traditional qualifications--core subjects--in favor of these easy "A" classes. Luckily, exam watchdog Ofqual has brought the matter to light. Enjoy the fake rays while they last, young Brits; tomorrow, perhaps, you'll again have to head to the Costa del Sol to obtain that grade-A tan.

"Now tanning courses are 'equal' to maths A-level," by Laura Clark, Daily Mail, February 17, 2009

"Courses in tanning 'worth the same as A-level maths' in school league tables," Telegraph.co.uk, February 17, 2009

» Continued

A-level tan

Closing the Expectations Gap: Fourth Annual 50-State Progress Report on the Alignment of High School Policies with the Demands of College and Careers

February 19, 2009

Achieve, Inc.
American Diploma Project Network
February 2009

While many in the education reform community focus on closing the achievement gap, this compendium marks Achieve's fourth annual summary of states' efforts to close the expectations gap--the gulf between high school exit requirements, and colleges' and employers' entrance expectations. This report gauges whether states possess five main things: standards, graduation requirements, assessments, P-20 longitudinal data systems, and accountability measures. Since the report is annual, we can look at previous editions (2008, 2007, and 2006) to judge actual progress. For instance, in 2008, 13 states anticipated aligning their standards with college readiness requirements by 2009--yet only 4 actually did so. Still, since the 2005 National Education Summit on High Schools there has been a sea change in states aligning their K-12 academic expectations with those of colleges and employers. Four years ago, just 2 states had college- and career-ready graduation requirements; today, 20 (plus D.C.) have them. Three years ago, only three states had P-20 longitudinal data systems; now 12 do. The report also has a nifty state-by-state chart detailing in which categories states excelled and which they fell short. Texas, for example, has the most effective policies in place, while Georgia, Tennessee, and Washington have taken significant steps in that direction since 2006. To see for yourself, click here.

» Continued

Closing the Expectations Gap: Fourth Annual 50-State Progress Report on the Alignment of High School Policies with the Demands of College and Careers

Impact of For-Profit and Nonprofit Management on Student Achievement:

February 19, 2009

Paul E. Peterson and Matthew M. Chingos
John F. Kennedy School of Government at Harvard University
Spring 2009

When the RAND Corporation released its lukewarm evaluation of Philadelphia's school management experiment in February 2007, Paul Peterson responded with his own study. Now two years later, Peterson is joined by Matthew Chingos in a report that builds on those initial findings and compares student achievement in reading and math at for-profit, nonprofit, and district-managed schools (whose achievement remains below the district median) from 2001-2008. The new study yields similar findings: Philadelphia's for-profit providers are beating the competition by miles when it comes to student achievement. (For those who may not recall the background, seven years ago, the Philadelphia School Reform Commission (SRC) arranged for 30 of the city's lowest performing schools to be taken over by for-profits and 16 by nonprofits.) RAND found that only district-managed schools made significant progress. Peterson and Chingos say otherwise. Accordingly to their analysis, not only do students learn "substantially more in reading and math if they attended a school under for-profit rather than one under nonprofit management," but they learned almost 60 percent more each year of the six years studied in a for-profit school than one under district management. The district, however, has been backing away from the whole outsourcing approach. Perhaps they should reconsider? You can find the Peterson-Chingos study here.

» Continued

Impact of For-Profit and Nonprofit Management on Student Achievement:

An Evaluation of Teachers Trained Through Different Routes to Certification: Final Report

Amber M. Winkler, Ph.D. / February 19, 2009

Jill Constantine, Daniel Player, Tim Silva, Kristin Hallgren, Mary Grider, and John Deke
Mathematica Policy Research
Institute of Education Sciences
February 2009

What are the effects of different routes to teacher certification on student achievement? And what aspects of certification programs (e.g. amount and type of coursework) are associated with teacher effectiveness? These are the key questions posed in this study. Looking at 2,600 students in 20 districts and 63 schools across 7 states, what makes this analysis different is its strong research design: students were randomly assigned within each school to either the class of an alternatively certified teacher (AC) or a traditionally certified teacher (TC). Interestingly, the researchers also excluded the most selective AC providers like Teach For America since the vast majority of AC and TC programs aren't selective and don't produce most of our teachers anyway. It found that there were no statistically significant differences in student performance between AC and TC teachers (all were novices) nor was there evidence that the content or amount of coursework is correlated with teacher effectiveness. Very important, though the definitions and terminology make it a bit confusing. TC teachers were simply defined as those who had completed their training while AC teachers were those who had not yet completed their training. So the majority of the AC providers--like the TC providers--were education schools and not truly "alternative" as it's commonly understood. Thus, this isn't exactly a study of AC vs.

» Continued

An Evaluation of Teachers Trained Through Different Routes to Certification: Final Report