New policy shop, Innovation Ohio, producing faulty, partisan research

A new policy and research organization has opened shop in
the Buckeye State. Innovation Ohio (IO) bills itself as a “nonpartisan
organization dedicated to promoting public policy that moves Ohio ahead without
leaving some of its people behind.” The organization is led by former deputy
chief of staff to Governor Strickland, Janetta King. Though most of its work
thus far has been published anonymously, IO relies on Stephen Dyer, a former
Democratic state representative for at least some K-12 education policy
research. Dyer pushed Strickland’s ill-fated evidence based model of school
funding through the House two years ago, and, in part as a reward for his
effort, lost his House seat in November.

As state and local leaders wrestle with the Ohio’s daunting
budget deficit and languishing economy, and as educators struggle to improve
their schools, advocacy groups like IO are a welcome addition to the state’s
policy debate. Ohioans only stand to benefit when more policy partisans are
publicly debating important issues. In this regard, IO proved valuable last
week by pressing the Kasich administration to be more forthright and
transparent about funding amounts local school districts will receive under its
budget proposal.

Aside from that instance, however, the reports and analyses
produced by IO – at least those relating to K-12 education policy – have been
less than stellar. They’ve billed themselves as serious researchers and garnered
ample media attention (see here and here
for examples) because they are a simple foil to the new governor and General
Assembly. But upon analysis, IO’s work appears to be hastily produced,
including egregious errors or inaccurate conclusions from the data.

Take two examples, IO’s debut report, Ohio
Teachers and Collective Bargaining: An Analysis
and last week’s Analysis: Kasich Budget Transfers $567
Million More For Public Schools to Private and Failing Charter Schools.

Ohio
Teachers and Collective Bargaining: An Analysis.

We previously shared the flaws in IO’s collective bargaining
report.

IO, relying on two
years’ worth of data showing average wages for teachers in Ohio, finds an average drop in annual wages of 3.8
percent. Without exploring the cause behind this (which could easily be that
the makeup of Ohio’s teacher pool changed, specifically that it got “younger”
and therefore cheaper). A number of school districts offered early retirement
buy-outs last year. Regardless, IO announced that the drop in annual wages was
evidence of dramatic cuts to wages and evidence of the “extraordinary financial
sacrifices” that teachers have faced.

Further, the report
mistakes correlation for causation. Taking a basic trend seen in two years’ of
data (average wage drop) it then claims that collective bargaining – or its
absence – is the cause of those
observed trends. Using several frightening if-then statements, it attempts to
make the case that if Ohio eliminates collective bargaining for teachers it
will lead to salary increases, spending volatility, and uncertainty for
taxpayers.

Rather than relying
on actual state or national achievement data (NAEP) to evidence claims that
removing collective bargaining will harm student achievement, IO used Education Week’s annual state rankings,
which thematically vary year-to-year and incorporate all kinds of non K-12
variables like “adult outcomes.” That the report doesn’t rely on achievement
data to make claims about student achievement is simply odd.

Finally, IO uses one Race to the Top reviewer’s flattering
comment about Ohio’s application as evidence that the state’s current teacher
evaluation and accountability systems are sufficient. Using a single anecdote
as proof (and it took nearly a full page of the eight-page report) is using a
criminally low threshold for evidence.

Analysis:
Kasich Budget Transfers $567 Million More For Public Schools to Private and
Failing Charter Schools.

IO’s recent charter
school funding report is no better. It claims that charter schools and Ohio’s
voucher program (EdChoice) are consuming an inordinate amount of state revenue
at the expense of financially-strapped traditional public schools. It puts
forth an astounding (and inaccurate) assessment that “it costs [Ohio], on
average, more than twice as much to educate a student in a charter school than
a traditional public school.”

It’s common knowledge
that charters in Ohio make do with
fewer public dollars than their district counterparts.
Research by disinterested experts concurs; see the aptly named Ball State
University study Charter School Funding: Inequity
Persists
. Based on the findings in that report, a child living in the Cleveland Metropolitan School District (CMSD)
arrives with a backpack of cash worth about $13,016 (using 2006-7 figures),
representing the total taxpayer contribution of that child’s education. That
backpack actually contains two wallets. One represents federal and state
dollars ($8,190); and the second contains local resources, including receipts
from property, local sales tax and other local tax revenues ($4,085). If
that child then leaves CMSD for a local charter school, the backpack that
accompanies her move is considerably lighter,
because the local money stays with the district. The child takes only the state
and federal dollars into the charter school, totaling about $8,200. Further,
charter schools receive no taxpayer money for school facilities.

In similar fashion to its
collective bargaining report, IO’s lambasting of charter schools draws from a
lot of assumptions and leaps of logic, with few or no accompanying footnotes,
facts, or figures. For example, it “projects that by FY 13, the Kasich budget
would transfer an additional $567.7 million to private and failing charter
schools” but doesn’t explain where that figure comes from. IO puts forward the
claim that “charter schools have experienced a 1285
percent increase in funding since 2001 compared with a 25 percent increase for
traditional school districts.”

This arresting
statement is accompanied by a similarly spectacular graph in the report. This,
too, is explained, if vaguely and briefly, in a footnote, which states that
“traditional public schools have seen almost zero increases” since FY 2004, the
first year after the Ohio Supreme Court ruled that the state legislature was
making a sufficient, good-faith effort to adequately fund the state’s schools.
A more logical explanation behind the surge in charter funding is that charter
enrollment grew dramatically over the decade and the 1285 percent figure
represents a commensurate increase in the number of students in charters
(instead of the notion that, as the report insinuates, the same cluster of charter schools received that much more money over
the decade).

The report lacks
precision and accuracy in several important areas. On countless occasions, it
sets up a false dichotomy in the state’s K-12 budget between traditional
district and charter schools, failing to remind readers that charter schools
are still public schools serving public school students and participating
fully in the state’s accountability system. It assumes that growth in the
charter sector will continue at the exact same rate as over the last decade
(and uses this to inform financial “projections”), but doesn’t explain how
researchers arrived at that assumption. It lumps charters into a broad
category, claiming that they are “failing,” “less accountable,” and
“unchecked.”

Lest IO appear to be
behind on its own research, it should realize that Ohio has the single most
rigorous charter accountability system in the nation, and actually forces
closure of chronically underperforming schools in a manner that other states
are trying to replicate. Further, not all charters in Ohio are failing and some
are in fact leading their communities in student achievement. In Cleveland and
Dayton, the majority of the top-performing schools are charters. Finally,
contrasting charter schools – which are centered in Ohio’s urban areas and
therefore serve a fundamentally different population of kids, to district
schools statewide (serving all income groups) is a really shoddy comparison by
any standard of research.

 Innovation Ohio is a welcome addition to the expanding body
of public policy research shops and think tanks in Columbus. It has the
potential to expand the marketplace of ideas in the Buckeye State and make the
conversation less divisive or at least more interesting. But until IO focuses
more on the issues and data and less on its preconceived outcomes, it’s
difficult to take the organization or its analyses seriously.

More By Author