Dumbed-down and becoming more so?

Here we are, somewhat dubious, but still enthused that Maryland reported record gains in proficiency scores this year, when we learn that Maryland neglected to mention they made their test easier. By making the test time 1 hour shorter (which, they claim only prevented the students from getting fatigued but did not decrease the difficulty—isn’t part of what makes tests difficult the time limit?), some questions were necessarily cut. These were not just any questions, however, these were the psychometrically approved questions.

Maryland had taken an off-the-shelf test, created by psychometric experts at Harcourt or similar, in 2002 and combined it with in-house questions. They argued that the off-the-shelf test lent validity to the assessment because it had been tested on millions of students across the nation, while the in-house questions were more relevant and aligned with the curriculum. In a cryptic Baltimore Sun article, we learn that Maryland elected to drop the vetted off-the-shelf questions for the following four reasons:

Although students had to answer about 40 questions on the standardized portion of the test, Maryland officials did not count most of them. Instead, they elected to count the questions that focused on material they cared about and those that reflected the state curriculum.

But Maryland students were not informed that some questions did not count and might have gotten bogged down on questions that covered unfamiliar material. In addition, teachers who looked at the tests when they were given each year saw material they had not focused on in class and might have been confused about what to teach the next year, according to Leslie Wilson, who heads the state’s assessment office.

Dropping the standardized portion also meant that students did not have to be given two sets of directions for the two parts of the examination.

Let me see if I understand this correctly. Not only are the properly vetted questions on material that is not covered by the state curriculum, but they’re not counted either. Instead, the questions that have not been vetted are counted because Maryland officials (who are, of course, experts in the art of test question design) just decided one day that that was the “material they cared about”? Teachers, of course, would be confused by the fact that the curriculum they had been given wasn’t aligned with the test so instead of, oh I don’t know, fixing the curriculum, we’ll just eliminate those pesky questions. And while we’re at it, let’s blame a minor procedural question because obviously students can’t listen to directions, even though that is, ostensibly, what they do all day, every day in school. Come now, Maryland. You can’t possibly think that you’d be able to defend your “historic” rise in test results when you made the test easier, can you?

But don’t lose all hope, because Fordham was there to shed light on this abominable double dealing in the form of Amber “The Axe” Winkler and the Baltimore Sun took notice:

Maryland’s test also has been questioned for other reasons. When compared with state tests around the nation, Maryland’s ranked 26th in difficulty, according to a report by the Fordham Institute, a nonprofit think tank in Washington.

“If already the proficiency bars are lower than half the nation, what can we really make of this?” asked Amber Winkler, Fordham’s research director.”

Bam! Answer that one for us, Maryland.

“1 in 4 California students—and 1 in 3 in Los Angeles—quit school,” reports the Los Angeles Times.

This made me go hunting for other ways students in California can graduate from high school or receive a diploma equivalent. Interestingly, California has a number of options. Students can take their GED as early as age 17 (but only within 60 days of their 18th birthday), or they can take the California High School Proficiency Examination (CHSPE). The CHSPE can be taken as early as 16 or after completing sophomore year (regardless of age!) with parental permission to stop attending school. While the GED tests reading, writing, math, science, and social studies, the CHSPE only tests reading, writing, and math. The CHSPE also differs from the California High School Exit Exam (CAHSEE), which all students take for the first time in 10th grade and is required for graduation. Passing the CAHSEE is not equivalent to a diploma and tests in English language arts and math. Students have at least six opportunities to pass the CAHSEE—one in grade 10, two in grade 11, and three in grade 12. If a student continuously fails the CAHSEE, the state is required to provide extra assistance until he or she can pass.

Between all these alternative options and the number of dropouts, one wonders if anyone is actually graduating anymore. I commend the fine state of California for biting the bullet and publishing these numbers, troubling as they are, but I hope they won’t spur more alternative education programs like the 2-hour-a-week (non) school the LA Board of Ed announced last week. Giving students options is great, but there is a line. California needs to be careful not to cross it.

“N.J. raises bar for pupil test scores”:

New Jersey made it harder yesterday for public school students to prove their proficiency on state exams—a change that could cause more schools to run afoul of the federal No Child Left Behind Act....

... with the changes, passing rates are likely to drop in a majority of tests, markedly in some cases, [Education Commissioner Lucille Davy] said.

In sixth grade, for example, state estimates show the language arts passing rate would have risen from 76 percent to 80 percent this year using the old cut scores, but instead will drop to 54 percent.

In an age when most states are wimping out on standards, one wonders how on earth the Board of Education mustered the will to do this. On the other hand, one wonders what, exactly, this means:

To provide districts some short-term protection against the predicted drop in passage rates, state officials plan to reduce the proficiency requirements considered by No Child—but set by the state.

The Washington Post reports that Maryland has shown huge gains in test scores, particularly among disadvantaged students, though the usual doubts about dumbing-down abound. Fordham’s own study of state test cut-scores suggests such skepticism is warranted.

Fun factoid of the day: Neither the ACT nor the College Board/ETS (giver of the SAT) tells colleges or universities why they cancel student scores. Joe Shmoe faints during a test? Joe Shmoe has his pal Freddy take the test for him? All the same in the testing companies’ eyes. They’ll cancel the score and let the student take the test again. And this is how they might explain it (as ACT recently did to UCLA):

The ACT cancels scores for a variety of reasons, including illness of the examinee, mis-timing of the test, disturbances or irregularity at the testing site.... It is the ACT policy to treat the ACT’s reasoning for canceling a specific score as confidential.

Even in cases where cheating is suspected, as described in today’s Los Angeles Times, the testing company investigates students directly—but doesn’t tell the high school or college that Joe has run quite the scam.

It seems to me that, although sponsored by external organizations, college entrance exams are inextricably linked to high schools and universities alike. Their value has come into question as some institutions no longer require the scores, but for many students the SAT/ACT remains a significant part of the portfolio. And many colleges still treat it as such. So, if students choose to tinker with their score, is a statement about a student misbehavior a natural extension of a score report?

Fun factoid #2: Apparently this policy has been alive and thriving for over 25 years. Now that’s impressive.

More on yesterday’s announcement that D.C. test scores are up. A Washington Post article today says that some principals are attributing their schools’ successes to Michelle Rhee. Super, great, on and on.  It was this line that really caught my eye:

At Bell Multicultural High School in Northwest, Principal Maria Tukeva introduced a Saturday “Quiz Bowl” in which students competed on sample tests for prizes such as iPods and movie tickets.

iPods? iPods?!?!?!?! Now, perhaps they were the little guys that retail for around $50 and I shouldn’t have had a heart attack after reading that sentence. Or not. Is a fancy gadget really an appropriate reward for correct test scores? What’s next, laptops? High-definition plasmas (50 inches or more thankyouverymuch)? It’s a slippery slope to motivate students this way. We all want the principals to be creative, but throwing a sexy reward at them isn’t the way to go.

Not to mention that in Washington, DC, we’ve had a little problem with iPod theft for some time now.

Test scores are up.

From the Los Angeles Times: “California mandates testing every eighth-grader in algebra—ready or not“

Florida Governor Charlie Crist signed this week a bill that lessens the emphasis of the state’s high-stakes test, the FCAT. The House minority leader, Dan Gelber, a Democrat, and Patricia Levesque, who directs former Governor Jeb Bush’s Foundation for Florida’s Future, both supported the bill—a rather odd pairing, to be sure. Here’s more about the changes.

The forthcoming debate between Sol Stern and Chris Cerf, over at Eduwonk, should be must-see blogging.

This news item out of the U.K. truly confounds.

Does anyone out there believe that the dramatic test-score increases coming out of the Empire State are legitimate? Sol Stern, for one, highly-knowledgeable on all educational goings on in New York, is with the naysayers. He points out in a piece on the City Journal website that

almost none of the dramatic improvements in the state tests show up in the most recent tests administered by the federal National Assessment of Educational Progress (NAEP), otherwise known as the “nation’s report card.” NAEP scores in fourth- and eighth-grade reading and eighth-grade math in New York State remained flat from 2005 to 2007.

Many critics have jumped on this embarassing comparison already. But Stern also illuminates this dubious idea of “rigorous peer review,” which state schools chief Richard Mills has used to try to deflect the inevitable charges of test-rigging:

One of the slides in his PowerPoint presentation was titled ENSURING THESE RESULTS ARE ACCURATE and claimed that “New York’s testing system passed rigorous peer review by [the] U.S. Dep’t of Education.” But this “rigorous peer review,” which all 50 states now undergo under the No Child Left Behind Act (NCLB), is less impressive than it sounds. I was told by a federal education department official that the review covers only the general process used by the states in establishing a reliable system of standards and assessment. It does not constitute a federal seal of approval for the accuracy of any state’s particular tests.

That’s not really a surprise, considering how forgiving the feds have been on pretty much all of NCLB’s most important requirements. What is suprising is that most everyone still believes this unwieldy law can be dramatically improved during the next go-round and that, somehow, politicians at all levels will lose the motivation to game high-stakes tests.

Mayor Bloomberg will announce today that test scores are way up in New York City. But no one, it seems, thinks the gains are legitimate.

I hope the College Board catches the flack it deserves for its decision to (starting in 2010) show colleges only the SAT scores that the students who earn them choose to reveal—i.e., if Johnny takes the test 10 times, Johnny gets to show State U. only his best score. Currently, university admissions officers see both how many times a student took the SAT and how he scored on each attempt.

The College Board’s purported reason for the rules shift: Taking the pressure off test-takers. The real reason: Making more money. Unfortunately for poor kids, they won’t be able to pay to take the SAT multiple times. Suburban kids—the ones who already shell out thousands for private prep classes, and who go to schools where guidance counselors map out every step of the test-taking plan—will. You better believe that starting in 2010, it’s going to be assumed that middle-class high schoolers take the SAT every single time its offered.  

Furthermore, the College Board is watering down the SAT’s integrity. I know, I know—the organization tells us that an individual’s scores will not significantly increase after multiple retakes. Of course, common sense tells us that claim is baloney and will be proven as such come 2011. But assuming that it is true, doesn’t it render irrelevant the College Board’s ostensible concern about lowering test-taking pressure? (If a kid knows he won’t score better on his tenth attempt than on his first, why would the option of multiple retakes without consequences lessen his anxiety?)  

This change is just so wrong on so many levels. It’s bad for poor kids and should offend those who care about standards and merit in college admissions.

Naomi Schaefer Riley takes it to the college-entrance-tests-are-biased crowd—especially those within it who profit from the very tests they decry.

On the front page of today’s Washington Post is a feel-good story about Ocean City Elementary, a Maryland school in which 100 percent of the students passed the state’s math and reading tests. I don’t want to rain on the school’s parade (or give the Post reporter, Dan de Vise, a hard time for finding an excuse to mix business with pleasure (hmm... this school is at the beach... the article appeared just after Memorial Day Weekend...)) but isn’t it worth pointing out (again) that when everyone can meet a standard, it means it’s not really a “standard”? Perhaps this is a sign that Maryland should raise the passing scores on its tests? Can you imagine a front-page Washington Post story reporting that an entire high school student body got a perfect score on the SAT? Surely someone would question whether standards had slipped.

Still, I’m sure we’re only days away from hearing Secretary of Education Margaret Spellings declare that “this school proves that 100 percent proficiency is an achievable goal.” And with low enough standards, yes it is.

Calexico (a U.S. border town) is kicking out of its schools Mexican students, who bring down test scores.

From The Tallahassee Democrat: “According to the Florida Department of Education, more students statewide are writing at or above grade level.” (The results are here.)

It’s great that Florida continues to concentrate on improving its students’ writing skills, but can FCAT writing scores really be an accurate depiction of Sunshine State youngsters’ sentence-crafting abilities, especially when the data are compared one year to the next? The larger question: Is it possible to accurately assess writing in a statewide, high-stakes test?

Linda Shaw wrote an interesting piece in last week’s Seattle Times. Apparently, civil disobedience against the Washington Assessment of Student Learning (WASL) is alive and well—at least, as embodied in Carl Chew, a 60-year-old science teacher who refuses to give the test to his sixth-graders at Eckstein Middle School.

Mr. Chew, a former artist who has been teaching for eight years, is opposed to high-stakes standardized testing. He claims he is taking a stand against WASL and No Child Left Behind in general.

“I did it because I think it’s bad for kids,” he said.

For his actions (or non-actions), Mr. Chew has been placed on leave for two weeks without pay. The WASL is given each year to students in grades 3-8 and grade 10. It covers math, reading, writing, and science. It is used to measure whether the schools in Washington state are meeting the goals established in NCLB.

Whatever one thinks of NCLB or the WASL—and I am the first to admit there are problems with both of them—Mr. Chew’s supposed “civil disobedience” is not the way to fix them. In fact, it is a recipe for educational chaos and anarchy. WASL is a state-mandated exam. By refusing to give the test, Mr. Chew failed to fulfill his duties as a teacher. If he doesn’t like the WASL, he can complain to his union, write an Op-Ed piece, call his local political representative, or advocate for its overhaul or termination at a public forum—in other words, he can utilize all of the tools available to a citizen in a democratic polity without violating his professional obligations.

Eckstein is not Selma or Dandi; students are not being beaten with police clubs or thrown into British jails. Hopefully, Mr. Chew will come to his senses and realize that his teaching responsibilities—including having students pass the WASL on their way to getting a high school diploma—trump his political feelings. If not, then perhaps he should return to being an artist, where real world considerations don’t matter as much.

That Miami-Dade is considering convening a task force to investigate the testing mania that has reportedly caused some students to be hospitalized illustrates how little trust district officials often place in their principals. School Board member Solomon Stinson so noted. According to the Miami Herald, “he warned against micromanaging teachers and principals, who have a better grasp on student needs.”

When I first started reading this Slate piece by Alexander Russo (”Chicago School Days: Obama’s lackluster record on education“), I felt my head spinning. Not only would I have to reassess my admittedly optimistic views of Barack Obama, I’d also have to concede that Russo might (in this case, at least) know what he’s talking about.

Then I finished the article and reclaimed my equilibrium. As it turns out, I wouldn’t have to change my mind on either front.

Here’s the rub. Russo dives into an important dispute from a decade ago that is little known to national audiences: whether Chicago’s local school councils—a.k.a., “mini school boards”—should have the power to fire school principals, as a 1988 law allows. At issue was an ugly history of minority-dominated boards firing white principals for little reason other than their race. By the mid 1990s, Paul Vallas, Chicago’s then-superintendent, wanted to strip the boards of this authority because he was tired of good principals getting thrown under the bus. As a state senator, Obama shadow-boxed around the issue, Russo claims, and then eventually sided with the local boards once the issue was resolved in their favor (not surprising for a former community organizer).

I don’t know whether Russo captured that part of the story accurately or not, but his analysis for what this could mean for NCLB is preposterous:

Based on Obama’s actions in Chicago in 1999, it’s hard to imagine him taking charge of the continuing debate over whether and how No Child Left Behind should be renewed. Forced to take a side, Obama’s record suggests that, ultimately, he would be sympathetic to local autonomy. But there’s not much evidence to show that he would be able to help mend deep and abiding schisms between testing hawks and local-control advocates.

But local autonomy, Chicago-style, is school autonomy. It’s about “power to the people”—at a very, very local level, more akin to charter schools than typical school boards. Outside of Chicago, “local control” is about district autonomy—and is promoted mostly by Republicans and status quo types who don’t want the feds pushing them around. The “testing hawks” are mostly liberal groups like Education Trust that want to use federal power to bust the school district oligarchy to give power to the people—poor and minority parents. So to argue that Obama would side with the pro-school board, generally conservative “local control” types over the liberal testing hawks acting on behalf of minority communities is... well, it’s typical Russo.

It’s nice to feel back to normal again.

Kevin Carey (a.k.a. Mr. Quick) is less enamored with Jim Ryan’s suggestions for fixing NCLB than I was (see here). Carey complains that Ryan “fails to notice that his proposed solutions completely contradict one another” because, on the one hand, he wants to eliminate annual testing, while on the other hand, he wants to include measures of student progress from year to year—which requires annual testing.

This is, to use a bloggy cliche, a classic “and a pony” policy agenda. NCLB can be improved, no doubt, but the people who wrote it weren’t morons; there are some very real and difficult tradeoffs to contend with in formulating accountability policy, and one of them is the tension between the costs and burdens of assessment and the need for comprehensive information.

Fair point, Kevin. But you have to admit... his proposal for national testing—no matter how frequent—is downright brilliant.