Not really. But Rick Hess and Jay Greene do see problems with Florida’s Amendment 9, which teachers’ unions and their allies are trying to keep off November’s ballot. The two policy analysts don’t like the amendment’s conflation of vouchers and the “65-percent solution.” We’ll ask Rick to tell us more on tomorrow’s Education Gadfly Show Podcast.

There will be vouchers in Louisiana.

Congress looks set to grant D.C.’s voucher program a one-year reprieve. (You have to scroll down a bit to see the story.)

The Wall Street Journal’s Bill McGurn (and lots of others) wonders: What will Obama do on school choice? Now we know (via The Corner):

TAPPER: You talked about the need to change the status quo in education today.

OBAMA: Right.

TAPPER: But one of the ways that proponents of school choice say that the best way to change the status quo is to give parents, inner-city parents a choice. Why not?

OBAMA: Well, the problem is, is that, you know, although it might benefit some kids at the top, what you’re going to do is leave a lot of kids at the bottom. We don’t have enough slots for every child to go into a parochial school or a private school. And what you would see is a huge drain of resources out of the public schools.

So what I’ve said is let’s foster competition within the public school system. Let’s make sure that charter schools are up and running. Let’s make sure that kids who are in failing schools, in local school districts, have an option to go to schools that are doing well.

But what I don’t want to do is to see a diminished commitment to the public schools to the point where all we have are the hardest-to-teach kids with the least involved parents with the most disabilities in the public schools. That’s going to make things worse, and we’re going to lose the commitment to public schools that I think have been so important to building this country.

TAPPER: So it would help some kids, but overall it would be bad for the system?

OBAMA: I think it would be overall bad for most kids. 

The more things change, the more they stay the same.

Mike opened the door for my response to the Washington Opportunity Scholarship Program external evaluation, and I’ve just completed a fairly quick read of it. First, in the spirit of full disclosure, I’ll note that my former employer, Westat, was the prime contractor for the evaluation. Though I never personally worked with the Westat staff who conducted the evaluation, I do know their reputations for quality work. This is not the only reason, of course, that I found the evaluation to be of high-quality, but it’s worth mentioning. Disclosure aside, I have a couple takeaways from the evaluation.

First, the impact findings for the program are simply not that compelling (sorry Mike), and even the subgroup analyses—which do provide a ray of hope—are presented with important caveats. The design comprised a randomized controlled trial where eligible applicants were randomly assigned to receive or not receive the scholarship. By all accounts, the sample was drawn appropriately and is of sufficient size (n=2,308 which is, we’re told, larger than impact samples in previous, similar evaluations); furthermore, the analyses appear thoughtfully and meticulously conducted.

So, while I have few qualms with the evaluation design itself, I do think something that occurred naturally within the impact sample—namely, lots of student mobility—is worth keeping in mind. Over the course of two years in the treatment group, only 4 percent remained in the same school they were in when they applied to the program; 71 percent switched schools once, and 25 percent switched schools twice. Among the control group, 22 percent remained in the same school they were in when they applied to the program; 57 percent switched schools once; and 21 percent switched schools twice. That’s a majority of kids (even more so in the treatment group) not attending any one participating school for very long. The authors report that “both groups experienced higher rates of school mobility than the typical annual rate for urban students (22 to 28 percent).” It’s not surprising, then, to see unimpressive findings in an evaluation that covers such a short duration (2 years) and examines achievement data from students who are extremely transient (not to mention that students were tested on Saturdays!).

Second, I’m struck by the number of times that the phrase “adjustments for multiple comparisons suggest that this finding may be a false discovery” (or similar nomenclature) appears in the report. Researchers concern themselves with multiple comparisons because they are in a position of simultaneously evaluating multiple questions and hypotheses. Simply put, when you consider the results of multiple, separate statistical tests together, there is more room for error. The issue has gotten more attention of late, in part because of this recent report from IES which presents methods for dealing with the multiple comparisons problem. Like most people involved with education, I’m interested in the best research possible given the time and resources available to conduct it. Many statisticians believe that ignoring the multiplicity problem leads to misinterpretation of findings, so these researchers covered their bases.

But with all of those “false discovery” caveats in the report, I found myself harkening back to Judith Gueron’s comments in this book. Ms. Gueron (of Manpower Demonstration Research Corporation or MDRC) writes:

Finally, rigor has its drawbacks. Peter Rossi once formulated several laws about policy research, one of which was: the better the study, the smaller the likely impact. High quality policy research must continuously compete with the claims of greater success based on weaker evidence.

Ahh, so true. Sooner or later, we must come to terms with the fact that the bar we set for rigor may unintentionally and preemptively knock out of the running a program that may, in fact, make some improvement in American education.  Mind you, I’m not calling for a return to the age of education anecdote equals research. Here’s Gueron again on a lesson she learned about running successful social experiments:

You do not need dramatic results to have an impact on policy. Many people have said that the 1988 welfare reform law, the Family Support Act, was based and passed on the strength of research—and the research was about modest changes. When we have reliable results, it usually suggests that social programs (at least the relatively modest ones tested in this country) are not panaceas but that they nonetheless can make improvements. One of the lessons I draw from our experience is that modest changes have often been enough to make a program cost-effective and can also be enough to persuade policymakers to act. However, while this was true in the mid 1980’s, it was certainly not true in the mid 1990’s. In the last round of federal welfare reform, modest improvements were often cast as failures.

The question is: Will the OSP ultimately pass the “modest improvement” test? At two years—a time period that’s too short to capture impacts that may evolve over time—we don’t know. What I do know is that parents believe the OSP is making improvements, that improvement for certain groups of students may exist, and that school choice in and of itself may prove a laudable goal even without raise-the-roof achievement gains. Also, as an educational community, we’d be wise to continue the dialogue around the financial, political, methodological, and common-sensical (I think that’s a word) tradeoffs involved in rigorous research.

Quick and the Ed writes about the recently released study of D.C. vouchers’ effectiveness:

Those who’d like to end the program can point out that the results were, all-in-all, underwhelming, but supporters of the voucher program can point to the positive results among certain subgroups. Most notably, students from the first cohort who used the voucher scored significantly higher in reading - supporters might use this to convince lawmakers to hold out for another year or two in order to see if the effects continue for subsequent cohorts. But will a few positive results be enough to save D.C. vouchers?

In fact, it is possible to support D.C. vouchers without referencing the above-mentioned study at all. Private school choice, in itself, has plenty of powerful arguments to support it, especially when it occurs in a district such as Washington, D.C., where the public schools are less than satisfactory and filled with poor and minority children who haven’t the means to leave them. The stat-happy crowd scoffs, but about this statistical evaluation (in which, of course, both sides will find nuggets of support for their positions) I think: Who cares? Certainly parents whose children are enrolled in the schools don’t—for one reason or another, rational or not, they like their new private schools. Perhaps they’re safer, perhaps they’re in better neighborhoods, perhaps they have nicer smelling hallways. But does it matter? The real question should be: Is the small number of D.C. voucher students worse off educationally than they were before? If not, why shut down this infinitesimally tiny program?

Update: Here’s the Washington Post editors on the voucher study in question. Here’s the National Review editors.

Update II: More evidence for why we shouldn’t really care what the D.C. voucher study finds.

Wall Street Journal columnist Bill McGurn wonders where Obama will come out on the question of D.C. vouchers for poor kids.

Update: Eleanor Holmes Norton defends in the Washington Post her anti-voucher stance.

That’s what the headlines should say about this recently released study on Washington’s federally-funded school scholarship program, though they probably won’t. That’s because, as the Institute for Educational Sciences (IES) summary states, the study “found no significant differences in student achievement between those who were offered scholarships to attend a participating private school and those who were eligible for, but were not offered (as assigned by a lottery) a scholarship.” But wait, there’s good news:

However, being offered a scholarship may have improved reading test scores among three subgroups of relatively more advantaged students: those who had not attended a School in Need of Improvement (SINI) school when they applied to the program, those who had relatively higher pre-program academic performance, and those who applied in the first year of program implementation.

Here’s what you need to know that the media (not to mention anti-voucher groups) will neglect to tell you: in tiny programs such as D.C.’s, it’s really hard to find “effects” because the sample size is so limited. Participants have to do dramatically better than the control group in order for researchers to detect a statistically significant difference. So the fact that impacts were found for three subgroups (whose members make up 88 percent of the program’s participants, according to the Department of Education’s press release*) is pretty darn impressive.

But hey, Fordham now has its own fully-credentialed Research Director, so I’ll let Amber take it from here.

* This is the kind of context (or spin, if you like) that would have been helpful when IES released its infamous Reading First study. What’s ironic is that Margaret Spellings’s press office did a great job on today’s report release and was totally MIA on Reading First—even though Spellings herself is lukewarm about vouchers and fanatic about phonics. Maybe, on Reading First, Spellings has something to hide.

Last week the Wall Street Journal editors defended D.C.’s voucher program after the Washington Post reported that its days could be numbered. They made the decent point that “The $7,500 voucher is a bargain for taxpayers because it costs the public schools about 50% more, or $13,000 a year, to educate a child in the public schools.” It would have been an excellent point, though, had they known and let it be known that in reality the district spends closer to $24,000 per pupil.

The Florida Teachers Union and friends sued the state on Friday to remove pro-voucher proposals from the November ballot, including a provision designed to restore, you guessed it, the Opportunity Scholarship Program, which was ruled unconstitutional in 2006 (D.C.’s version of the program may face a similar fate). The conspiracy theorists hold that the amendments are “part of a well-organized, well-financed campaign to outsource public schools” (yawn). Others believe that the issue needs to be decided by the voters as opposed to the union (or the courts)—a novel idea indeed.

John J. Miller, who wrote a segment of Fordham’s recent Catholic schools report, has a nice piece in the most recent National Review that traces the beginnings of school choice—charters and vouchers—in Washington, D.C. (Right now, it’s available only to subscribers, but once the NR brass makes it public, I’ll be sure to repost the link.) It helps clarify at least one thing about the city’s Opportunity Scholarship Program, which Eleanor Holmes Norton and her Congressional colleagues are planning to kill: The burden of explanation rests with them. That is, OSP supporters include a wide range of people: Conservative Republicans; liberal Democrats; Washington, D.C.’s mayor and schools chancellor; Marion Barry; private school administrators; parents whose students are enrolled through the program. If Norton and Congressional Democrats choose to stick their finger in the eyes of such a truly diverse and widespread crowd, they 1) will need to justify their actions with some convincing arguments (which have heretofore hid), and they should 2) be ready to receive some serious backlash. The battle is over a specific policy that involves only D.C., but it’s going to make national news... and it’s unlikely that our presidential contenders can be silent about it. (If McCain wants some easy education points [points he's mostly lacking], he might want to jump in on the right side of this fight.)

“In a major legislative success for Gov. Bobby Jindal, the Louisiana Senate voted 25-12 Wednesday for a bill that would let up to 1,500 low- to middle-income students in New Orleans attend private schools at taxpayer expense.”

Article here.

The Wall Street Journal editors defend D.C.’s voucher program.

As the D.C. voucher program comes under attack, where is Democratic Senator Dianne Feinstein, whose initial support for the program was instrumental in its birth? Apparently too busy working on a much more pressing public-private partnership issue: the management of the Senate cafeterias.

As you can see, we’re not exactly doing cartwheels over here upon hearing what Eleanor Holmes Norton had to say about the D.C. Opportunity Scholarship Program. She’s apparently concerned about “protecting the children.” There was not one mention in the Washington Post article, however, about basing future funding decisions on the evidence regarding impacts of the program. Choice supporters (like ourselves) would surely like it if the rigorous external evaluation of the program pointed to significant and large positive impacts for children participating in the program, but alas, it’s simply not that cut and dry.

The first year impact evaluation (released last June), in fact, measures differences occurring just 7 months after the start of the students’ first year in the program. Not surprisingly, researchers found no statistically significant impacts, positive or negative, on student reading or math achievement for year one. They did, however, find that the program had substantial positive impact on parents’ views of school safety (i.e., parents in the treatment group perceived their child’s school to be less dangerous than parents in the control group) and on parents’ overall satisfaction with their child’s school. These findings echo what we have learned in other studies; that is, that parents want choices for their children and that they care about a wider variety of outcomes (e.g., school safety) than the outcomes preferred by other education stakeholders (e.g, student achievement). The executive summary of the evaluation closes with this:

The findings here are based on information collected only a year after students applied to the program and may not reflect the consistent impacts of the OSP [Opportunity Scholarship Program] over a longer period of time.... The first year, results, therefore provide an early look at student experiences in what was a transitional year for most of them. Future reports will examine impacts 2 and 3 years after application to the program, when any short-term effects of students’ transition to new schools may have dissipated.

Unfortunately, the political shenanigans surrounding this program may draw the curtain on it before the program has time to gain traction and potentially demonstrate results for longer than a 7 measly months—an admittedly in-flux time period for transitioning students.

It’s unclear exactly what Eleanor Holmes Norton, who represents Washington, D.C., in Congress, so dislikes about the Opportunity Scholarship Program that provides some 2,000 low-income students in the District an opportunity to receive their educations from private providers. She told the Washington Post, “We have to protect the children, who are the truly innocent victims here.” But victims require victimizers, and Holmes Norton neglects to be specific. Are “the children” victims of too much educational choice? Are they victims of their parents’ desire that they receive stronger educations? From whom or what, exactly, does Holmes Norton purport to protect D.C.’s low-income kids? Ah, but she’s not really protecting kids, is she. Holmes Norton is protecting a public-school system that has itself victimized countless numbers of Washington, D.C., children. She should’ve told the Post: “I intend to destroy any challenge to the Washington, D.C., public-school monopoly, and the desires of low-income children and their parents, my constituents, be damned.”

Sadly, the D.C. Opportunity Scholarship Program seems to be on its last legs. Non-voting D.C. delegate Eleanor Holmes Norton, who has lobbied hard to kill the program, said, “We have to protect the children, who are the truly innocent victims here.”

Victims? Of a cruel plot to give students trapped in failing schools a real shot at a decent education? How can anyone swallow this?

Mike and I can disagree all day, during normal business hours, about the level of transparency we should demand from voucher schools. But in the NOLA case, the issue is “contentious” and might stall the $10 million proposal (although the city’s Catholic schools will accept standardized testing—they’re desperate).

It’s entirely appropriate that the Louisiana Senate would require schools participating in a possible New Orleans voucher program to “administer state tests to voucher students.” That’s hardly out of line for other voucher programs. Milwaukee’s requires schools to test all students (not just those in the program); D.C.’s requires all voucher recipients to participate in an annual evaluation—i.e., testing. (Though, regrettably, the D.C. results aren’t broken out by school.)

These are public dollars. The more transparency, the better.

Worth keeping an eye on this, especially because “the Senate plan would require schools to administer state tests to voucher students.”

Parental choice advocates might think that Sol Stern’s critique of school vouchers has harmed the cause, but surely Marion Barry’s embrace of the DC school choice program in today’s Washington Post is much, much worse for the movement. (Though they don’t seem to know it.)

So reports Ron Matus in the St. Petersburg Times.

The New York Times offers a piece today about the progress of providing good public education in New Orleans.

And this bill, backed by Gov. Bobby Jindal and fighting its way through the state legislature, is promising.

The Washington Post editors turn in a nice defense of the D.C. Opportunity Scholarship Program today. As they point out, it will be tough to get Congress to approve the $18 million set aside for the program, especially considering the fierce opposition of D.C. Delegate Eleanor Holmes Norton.

But the good news, frankly, is that Mayor Adrian Fenty, a Democrat, is going to the Capitol to defend the program—i.e., to defend vouchers. For an issue propelled primarily by the fuel of party affiliation, it’s extremely heartening to see a political leader have the guts to say, “Hey, this idea has enormous potential for turning around a district in shambles, and I’m going to stand up for it, simple as that.”

(Cory Booker, the Democratic mayor of Newark and a man of seemingly boundless integrity and conviction, has also publicly supported vouchers. Let’s hope the trend continues.)

Here in D.C., the politics of education reform seem tame compared to what our Fordham team in Ohio faces, a point made clear in this Columbus Education Association interview with Governor Ted Strickland. In outlining his “6 point plan” on education, Strickland continues the attack on charter schools that began during his campaign, calling them “destructive to our students and wasteful of our tax dollars,”* repeating his previous calls for “a moratorium on the creation of new charter schools,” and strongly hinting that if only he had a Democratic legislature he could truly kill the state’s charter (and voucher) program.

He’d also like to turn back the clock on accountability, arguing that “testing and assessment ought to be diagnostic,” and “teachers must have the freedom to teach without the fear of standardized test results communicating that you’re a bad teacher.”

Of course he’s genuflecting before the unions, so much so that this quote—which apparently addressed how teachers have influenced his life—seems like a comic Freudian slip about their role in his administration: “Teachers have incredible power and monumental influence. What’s most important... is that (teachers) need to be respected by the government.”

And what about the students, Governor?

We hope Democrats outside Ohio (e.g., Eduwonk) notice that he’s giving the party a bad name in education.

*Correction: The CEA wrote to tell us that their interview had erroneously attributed the Governor’s “destructive and wasteful” quote to charters rather than to vouchers. They have since corrected the interview.