Can Philanthropy Fix Our Schools? Appraising Walter Annenberg's $500 Million Gift to Public Education
April 1, 2000
Case Studies:
New York City by Raymond Domanico
Philadelphia by Carol Innerst
Chicago by Alexander Russo
Afterword by Chester E. Finn, Jr. and Marci Kanstoroom
Contents
- Foreword
- Introduction: An Unprecedented Challenge, by Raymond Domanico
- A Small Footprint on the Nation's Largest School System, by Raymond Domanico
- Grant Brings High Hopes, Modest Gains to Philadelphia School Reform Effort, by Carol Innerst
- From Frontline Leader to Rearguard Action: The Chicago Annenberg Challenge, by Alexander Russo
- Afterword: Lessons from the Annenberg Challenge, by Chester E. Finn, Jr. and Marci Kanstoroom
- Bibliography
From Frontline Leader to Rearguard Action: The Chicago Annenberg Challenge, by Alexander Russo
From Frontline Leader to Rearguard Action: The Chicago Annenberg Challenge
Alexander Russo
When three of Chicago's most prominent education reform leaders met for lunch at a Thai restaurant six years ago to discuss the just-announced $500 million Annenberg Challenge, their main goal was to figure out how to ensure that any Annenberg money awarded to Chicago "didn't go down the drain," said William Ayers, a professor of education at the University of Illinois in Chicago. Ayers, who was at that lunch table in late 1993, helped write the successful Chicago grant application.
More than six years later, the time has come to ask whether that goal is on its way towards being met. Nearly all of the $49.2 million Chicago grant has been awarded. Over sixty "networks" have been funded. These networks, consisting of an external partner of some sort -- a reform group, higher education institution, or community advocacy organization -- work with at least three schools each on issues of school size, teacher isolation, and professional development. Thus far, 223 schools have participated in these Annenberg networks. In fact, roughly 40 percent of the students in Chicago schools have been reached, according to the Challenge.
What makes analyzing the impact of the Chicago Challenge especially difficult is that, during roughly the same period, the reputation of the Chicago public schools has been transformed from one of the worst to one of the most improved major school systems in the nation. Long-standing financial problems have been sorted out. Innovative programs have been piloted. Student achievement is on the rise. So the question is not whether things have changed in Chicago, but, rather, how the Annenberg Challenge contributed to these changes. Has it affected either the system or the individual schools with which it has worked? Has it helped improve student achievement? How successful were Professor Ayers and his colleagues in making sure that the Annenberg money was well spent? What footprint will the Chicago Challenge leave behind?
What is clear is that -- despite an impressive range of activities -- it is extremely difficult to attribute any specific achievements or progress to the work of the Chicago Challenge. Yes, the Challenge should receive some credit for any success at improving these schools or supporting the progress of their students. Many of the Challenge schools have shown significant improvements. And some observers give the Challenge credit for participating in the overall improvement of the schools. But no one seems to know for certain the extent to which these improvements might be due to Annenberg activities or to other factors.
There are several reasons for this. Many of the networks predated the Challenge. Each of them took its own approach to improving education. Many participating schools were already involved with their external partner before Annenberg came on the scene. Moreover, most of these schools were involved in several different reform efforts along with the Annenberg effort. When the Challenge grants expire in 2001, finding concrete legacies of the $50 million investment may become even more difficult. Doubtless, proponents of the Challenge will continue to argue that its role in raising test scores and improving teaching was significant. And some of the networks will continue to operate. But unless new research provides conclusive new findings, even its strongest advocates will struggle to point to any large-scale improvements attributable to the Chicago Challenge itself.
After presenting a brief chronology of events, the following pages offer several possible interpretations of what the Chicago Challenge accomplished and why. One view is that the Challenge was based on a design that gave it little influence or interaction with the schools it was trying to help. Another widespread idea is that it lacked programmatic focus and pursued too broad a set of initiatives with too many different partners. A third perspective is that it limited its own success by operating without strong links to an increasingly powerful school district. A final, more sympathetic, view is that the Chicago Challenge has kept the flame alive for decentralized, community-based school reform -- even as the system was moving in a very different direction -- and has contributed significantly to improvements through hard work in some of the most disadvantaged schools in the city.
Few assertions about the Chicago Challenge pass without disagreement. Chicago is a city where education is taken very seriously, and policy decisions, perceived press biases, and credit for progress all generate heated debate. The full analysis of whether it succeeded is extremely complex. Moreover, the Chicago Challenge still has almost two years to go before closing its doors. Two more evaluations, expected in 2000 and 2002, could shed new light on the impact of the Challenge and its effects.
Note, too, that this case study is not a comprehensive evaluation. It contains information from a series of interviews and school visits conducted in late 1999 and early 2000, as well as a review of all available documents. Most significant among these are a 1998 "baseline" study of the origins of the Challenge and a March 1999 Consortium on Chicago School Research report authored by Dorothy Shipps and others that explains many of the thoughts behind the actions of the Chicago Challenge during its first three years. Additional reports covering more recent history and student achievement data have not yet been released. In addition, some key members of the Chicago school-reform community, including the staff of the Chicago Challenge itself, refused interview requests and/or declined to provide materials for this case study. Their potentially valuable insights are not included here.
What Was the Chicago Annenberg Challenge?
When Joan Crisler arrived as the new principal of Dixon Elementary School on the city's near-West side, she brought with her a strong managerial style and inherited a demoralized and fragmented faculty. Under a grant from the Chicago Challenge, a Comer-based network in Chicago called Youth Guidance provided Dixon Elementary with a liaison to the network, funds for staff-development time and materials, a coordinated social-service plan to help students most in need, and professional working groups where principals from around the city could share ideas about school leadership. Today, Principal Crisler has delegated authority, unified her staff, and addressed student needs. Test scores at Dixon have risen steeply.
By most measures, Chicago schools were deeply in crisis when the Annenberg Challenge was announced in late 1993. The nation's third-largest school district, Chicago has over 430,000 students attending almost six hundred public schools, a pupil population that is 80 percent minority and 84 percent low income, and an annual budget of just over $3 billion. Chicago had been labeled one of the worst school systems in the country. Labor disputes, lack of clear policy direction, administrative instability, and a $415 million budget overrun hobbled the system. A 1988 law that gave individual school councils control over discretionary spending and hiring principals had opened up the system but was under constant attack and had yet to show significant gains in student achievement. The best that could be said about Chicago schools at the time was that there was plenty of room for improvement.
Despite its dysfunctional school system, Chicago did have a number of advantages when it came to applying for Annenberg funding. "Chicago was unique, or at least uncommon, in its ability to make use of that money," said Bill McKersie, currently a program officer at the Cleveland Foundation. "From 1987 on, you had an increasingly highly organized nonprofit sector out there worrying about school reform." McKersie estimates that between $9 and $11 million in philanthropic contributions to Chicago schools were being made each year even before the Annenberg funding arrived. Chicago also has a dizzying variety of school-reform groups -- over one hundred by one count -- along with a well-established set of foundations active in the education field, including the John D. & Catherine T. MacArthur Foundation and the Joyce Foundation. In addition, according to McKersie, Chicago had already developed a grassroots network and a strong research capacity. The city has a long tradition of grassroots organizing and activism dating back to Saul Alinsky, and Alinsky's spirit and organizing methods are still alive in many of the community groups that eventually became involved in this project. "In the early evaluation meetings," said McKersie, "Chicago was a good head and shoulders ahead of the other cities because of the work that had been going on." As it turned out, however, these organizations carried with them disadvantages as well as benefits for the Chicago Challenge.
Besides being awash in reform groups and philanthropy, Chicago had been through a major school-reform effort just five years earlier. Long blocked from meaningful participation in the city's schools, a broad coalition of advocacy groups and community organizations had banded together in 1988 to win legislative passage of a reform measure intended to open the school system up to parents and the community. The dominant element of the 1988 statute was creation of elected local councils at every school. These councils had broad discretionary authority, including the authority to hire and fire principals, and led to the turnaround of many schools that had lagged behind. In addition, the 1988 reform freed up hundreds of millions in discretionary state and local funds, which were shifted to schools and put under the control of these local councils. Many of the councils used these discretionary dollars to pay for supplemental services that local and national reform groups provided -- a pattern that eventually became the basic structure of the Chicago Challenge. But here, too, the city's history of activism and reform may have hindered the Chicago Challenge as much as it helped.
From Informal Working Group to Independent Foundation
Out of that initial Thai lunch came a working group of educators and community leaders who collaborated in 1994 to fashion Chicago's successful Annenberg grant application. With over seventy members, "the working group was a selection of people drawn from the school-reform community," said Anne Hallett, executive director of the Cross-City Campaign for Urban School Reform, who was closely involved in the creation of the grant application. "It was a pretty big and fluid group." In fact, this process brought together many of the same groups that had worked together during the late 1980s in a 125-member entity known as the Citywide Coalition.
The working group met over the course of several months, developing a framework for the grant application. These efforts were aided by the fact that both Hallett and Warren Chapman, an education reformer now at the Joyce Foundation, had been involved with creation of the national Annenberg Challenge. At some point in the process, according to Hallett, a subset of ten to twelve members was elected from the wider group to craft the grant proposal. In the end, the proposal developed by the working group beat out competing proposals from several other entities, including the Chicago Public Schools and the mayor's office.
Having secured Annenberg funding for Chicago, the working group would soon evolve into a more formal organization, albeit with strong ties to the groups that wrote the grant proposal. Initially run out of shared space in the offices of the Cross-City Campaign and administered through an existing philanthropic organization called the Donors Forum, the Chicago Challenge soon became its own new foundation with status as an independent fiscal agent. By late 1995, Ken Rolling had been named executive director, a board of directors had been established, and the first round of grants had been awarded. Rolling lacked experience in education but came from the foundation world and was well-versed in community organizing. The board, which was intended to set policy, raise matching funds, and hire an executive director, included prominent educators and business leaders. A second entity, the newly-created Chicago School Reform Collaborative, was also established. Its twenty-plus members were elected from the group of educators and advocates who had helped shape the grant proposal. Initially, at least, this offshoot of the working group functioned as the operations arm of the Chicago Challenge. However, this situation created procedural and ethical concerns and in time the Collaborative was transformed into an advisory body.
The Chicago Challenge would give grants to "networks," each consisting of at least three schools and one external partner -- usually a school-reform group, community organization, or university-based center. In addition to the issue of school size, which was the primary focus of the Annenberg Challenge in New York City, the Chicago Challenge established two other areas of interest: time and isolation. Another variation was that schools would not be allowed to participate in more than one network, but external partners could -- and did -- take part in more than one network.
Getting the Word -- and the Grants -- Out: The First Funding Cycles
To help get the word out about how the Chicago Challenge would operate, a series of community meetings was held around the city in 1995. Then, with a possible $49.2 million in the pipeline, assurances that existing public and private funds could cover the matching requirements, and a rudimentary review process established, the Challenge began to give out money. In the first funding cycle held during 1995, 177 letters of intent were submitted. In the end, twenty-five proposals were funded that year (including both implementation grants and planning grants) and the full amount of $3 million was awarded -- mostly to networks based around community organizations or institutions of higher education.
This initial wave was the largest group of grants ever awarded by the Challenge. Roughly $2 million in larger implementation grants was awarded to thirteen organizations during that first cycle, which had originally been limited to no more than ten. At $100,000 to $200,000 annually, most of these grants were awarded to school reform or higher education groups that were given funds to continue and extend previously-established partnerships. During that same funding cycle, twelve smaller planning grants were also awarded, averaging $25,000 each. Other applications from business, cultural, and labor organizations did not fare as well in comparison to higher education institutions and community organizations. Until 1997, for example, business coalitions had just a 1 percent approval rate, in comparison with over 40 percent for education-reform groups and higher-education institutions.
Sheriff Vallas Rides Into Town
Just as the Challenge was getting off the ground, massive changes were beginning to unfold in the way Chicago's schools were run. While local school councils had invigorated some schools and led to important changes in how they were run, it was clear to many that the 1988 reform had not engendered the widespread improvements in student achievement that the public demanded. Some studies showed that local councils could be extremely effective, yet districtwide scores remained low. Long-standing fiscal and administrative malfunctions persisted.
As a result, there was another massive wave of reform, this one involving one of the first mayoral takeovers of a major school district. In 1995, the Illinois legislature turned control of the Chicago schools over to Mayor Richard Daley's office and established new accountability measures for failing schools. Led by new chief executive officer Paul Vallas, Mayor Daley's former budget director, the 1995 reform ushered in a number of changes. Gery Chico, the mayor's former chief of staff, was named president of the new, mayorally-appointed Reform Board of Trustees. After years of trouble, the district's financial woes quickly began to be sorted out and a blizzard of new initiatives followed. Labor contracts were signed without strikes. High-stakes testing was implemented, forcing tens of thousands of students to attend summer school in order to pass from grade to grade. Failing schools were put on probation or reconstituted. Uniform academic standards were established.
Despite the positive press and the much-needed administrative reforms implemented by the Vallas team, however, not everyone was happy. Conflicts between Vallas and the 1988 reform groups erupted within the first year. In addition to being outraged that Vallas took credit for test-score increases published shortly after his arrival, many community groups were suspicious that Vallas would try to water down the 1988 reforms and tie the hands of local school councils. Many of the reform groups also opposed the increased use of standardized test and systemwide mandates. The Vallas reforms took attention away from long-standing efforts to help improve Chicago schools, and the fact that Vallas received so much positive press in such a short time just added salt to the wound.
Settling in to Work: Grant Making from 1995 to 2000
The implementation networks funded by the Chicago Challenge during 1995-97 were extremely diverse in terms of both the numbers of schools they were working with -- from as few as three to as many as twelve -- and their programmatic approaches. Demographically, participating schools were by and large representative of the district as a whole. High schools, middle schools, elementary schools, and schools-within-schools were all included. Networks such as the Chicago Middle Grades Network focused on curriculum and instruction. Others, such as the Center for School Improvement at the University of Chicago, had strong leadership-development elements. At least two networks were focused on developing small schools. And about half emphasized parent and community involvement.
Through its network grants, the Challenge supported both nationally-known initiatives such as the Comer Model, Success For All, and the Coalition of Essential Schools, as well as locally-developed reforms. In most cases, these efforts were already underway before the grant was received, and the Challenge grants have served to strengthen and intensify them rather than to establish new initiatives or even extend existing approaches to a larger number of schools or students.
The pace of grant making slowed markedly after 1995 and 1996, at least partly in response to the reality that nearly two thirds of the Challenge's resources were by then committed to the original group of grantees. Twenty-five implementation grants had been awarded during the first three years. By 1997, ten of the original thirteen implementation networks had won continuation grants, and several other organizations that had received planning grants in 1995 won the larger implementation grants. Concerns about the quality of the networks were another factor slowing the flow of new grants, along with the poor quality of the applications coming in. "After the first cut, the proposals were not as rich and substantial as we had expected," wrote Ken Rolling of the slowdown that began in 1996. Additional concerns about the workings of the networks and negative perceptions of the Chicago Challenge surfaced within the first two years.
This is not to say that the Challenge ceased to give out money. Despite resource limitations and other factors, the number of networks continued to grow. From 1995 through 1999, the Chicago Challenge supported over sixty networks, reached an estimated 223 schools, and produced a series of research reports and studies. In this way, the Chicago Challenge effort has reached almost 40 percent of the city's public schools, public school students, and classroom teachers during this period, according to Annenberg materials -- though it is not clear how many students, schools, and teachers received direct support or benefit.
Anecdotes but No Data: Limited Evidence of Impact
Anecdotally, there is a strong sense of progress and achievement among those closely involved with the Challenge. "There are more and more schools improving the quality of education" as a result of the Chicago Challenge, said Peter Martinez, a senior program officer at the John D. & Catherine T. MacArthur Foundation, who has worked closely with the Challenge. "There are more and more good staff development programs, as opposed to half-baked efforts. Overall, there's more movement in this system now than there has ever been."
Others, such as William Ayers of the University of Illinois, paint a similarly positive picture. Ayers said the Chicago Challenge has done an "astonishingly good job" in several key areas. For example, it has "raised for public debate systemwide the issues of school size, professionalizing teaching, and the relationships between communities and their schools." Ayers also believes that the Annenberg Challenge has demonstrated the power of networks to create a sense of community among schools grappling with similar issues.
The sense that the Chicago Challenge has had positive effects is also palpable among teachers, administrators, and program coordinators who have been involved with it. "The need for outside organizations is clear," stated Vivian Loseth, who runs Youth Guidance, the Comer network supported by an Annenberg grant. And the 1998 and 1999 Consortium reports reveal that many of those who received Annenberg support found their participation to be useful.
Beyond testimonials from those associated with the Challenge, however, it becomes difficult to find conclusive indications of the program's impact. Outside of anecdotal examples, few of the networks contacted were able to distinguish clearly what specific role Annenberg funds had played in their effectiveness, and none of the networks contacted could supply research that attributes student-achievement gains to Annenberg funding. "What Annenberg does is to award money to networks to deepen what they are already doing," explained network leader Sara Spurlark, who is also co-director of the Center for School Improvement at the University of Chicago. "Our efforts predated Annenberg, and we did not expand because of them." Other network heads echoed these sentiments, stating that Annenberg support allowed them to enhance their activities but was so closely integrated into what they had already been doing that they could not distinguish its precise effect.
Therein lies the problem. While few connected with them doubt the value of the programs supported by the Chicago Challenge, their impact is not yet established. This lack of hard evaluation data on the effectiveness of the Challenge is a source of widespread frustration in a city where test scores have increasingly become the coin of the realm. "We don't have a lot to tell you," admitted University of Illinois professor Mark Smylie, who is principal investigator for the Chicago Annenberg Challenge Study being conducted by the Consortium on Chicago School Research at the University of Chicago. The Challenge is "a difficult thing to evaluate," he explained. "None of these Challenges reflects a tightly designed programmatic initiative that renders itself useful to traditional evaluation." The Chicago Challenge and the networks seem well aware of this situation. "This is one of their big gaps," said Jack Mitchell, who oversees an Annenberg-funded network using the Coalition of Essential Schools model. "They recognize it now but one of the biggest voids is the ability to connect the conceptual framework with the gains in student achievement."
An upcoming study will provide the first real look at how the work of the Chicago Challenge may have affected student outcomes. According to Smylie, it will show how well Annenberg schools have done in comparison to similar schools not participating in Annenberg networks. It will highlight differences in impact among the different types of Annenberg-funded networks in Chicago. And it will examine how sensitive student achievement is to variations in implementation of the networks in each school.
Given the enormous number of changes that have taken place over the last five years and the erosion of the Challenge's prominence, however, it is unlikely that even the most optimistic results from the upcoming study will be broadly accepted as watertight evidence of success.
Why is there such a limited sense of Annenberg's effect on the now-improving Chicago schools? With so many supporters around the city, why is it so difficult to point to concrete examples of the Challenge's impact on student achievement or school effectiveness? Perhaps future studies will be able to answer these questions with hard data. It is clear already, however, that several key events and decisions significantly affected the success of the Annenberg Challenge in Chicago.
The Cult of the "External Partner": Effects of Working Outside the System
Of all the factors that may have affected the success of the Chicago Challenge, both its basic design and its institutional "location" outside of schools, existing reform organizations, and district efforts appear to have played an enormous role.
For the Chicago Challenge and many other reform efforts, success rests in large part on balancing the need to be an accepted part of school, community, and district without becoming so enmeshed in the system that they lose their independent voice. It certainly makes sense that the Chicago Challenge would want to start with a clean slate, independent from both the school system and the existing reform groups in Chicago.
Signs abound that the Challenge made strenuous efforts to ensure that it did not become more a part of extant systems than it thought was advantageous. Not only was it independent from the district, it was administered by an entirely new entity. (In other Annenberg sites such as New York City, the Challenge operated through existing reform groups.) The Chicago Challenge was different. It also gave out funds to a large number of networks, rather than providing services or working directly with a smaller number of schools. In many ways, the Chicago Challenge ended up functioning as its own medium-sized foundation.
Support for the concept of intermediaries and networks working alongside school systems runs strong in Annenberg literature. "The business of improving schools requires intense, ongoing facilitation, and one cannot expect this help to come from within," states the 1999 Annenberg Challenge report. "Schools cannot achieve whole-school change alone," wrote two Chicago Annenberg program officers in a recent publication. The decision to direct funds to networks rather than straight to schools or districts has always appeared self-interested to some observers, but this approach to reform had been floating around for a number of years and was already operating among many Chicago schools. In fact, the notion of external networks is at the core of the whole Annenberg endeavor, which was built on the idea that serious district-level change could not be effected from inside the system.
This approach is praised by some local reformers, such as Warren Chapman of the Joyce Foundation, as an innovation in grant giving that may prove superior to aiding individual schools. "We've funded individual schools since the beginning of philanthropy in education," said Chapman. "But this approach is relatively new. Do you evaluate a five-year-old child and say this child is not doing well and throw him or her away?" Some second Chapman's praise of the network approach as solid and appealing. "I like it as a structure," said Paul Reville, who is director of the Pew Forum and a board member of the Public Education Fund. "I think the notion of a third party playing a catalytic role is a healthy one," he said. "Injecting funds into the system, there is a great danger that it will be put into normal operating systems."
What is not clear is how much independence the Challenge actually achieved -- or if it benefited from the independence it sought. Working outside the established school reform community in Chicago may have slowed the initial work of the Chicago Challenge. By several accounts, the Challenge was slow out of the starting blocks, which may have contributed to its lack of momentum. Office space had to be found, new staff hired, procedures crafted. Observers report that, at least for the first two grant cycles, the Challenge was often out of sync with school schedules, budgets, and planning timelines.
It also took some time for the newly-formed Challenge staff, some of whom lacked extensive experience in grant making, to develop necessary expertise. This included becoming familiar with schools and the real-world capacities of the many organizations that sought funding. "They got smarter as they went," said Peter Martinez. "While at the beginning they took at face value that the partnerships were formed and the commitments were there, they got smarter after the first round." Others reported that networks funded in the early rounds of the Chicago Challenge sometimes included schools that were too troubled to benefit from participating, requiring a change of focus in subsequent rounds.
Of course, none of these start-up problems was insurmountable, and many were addressed within a reasonable amount of time. Yet the Chicago Challenge remained out of direct contact with schools. As described in the Consortium's 1998 baseline report, the Challenge's relationship with participating schools was indirect and attenuated. It was the networks, rather than the Chicago Challenge, that recruited schools to participate -- and not even all the networks could claim to have the full attention of their partner schools. According to early reports and close observers, there was little contact between the Challenge and individual schools. Almost all of its work was done through these intermediaries.
In this respect, the Chicago Challenge may have been asking too much of its networks, which may have had independence from the school system and some valuable expertise but lacked much clout in the schools. The network design may also have contributed to the lack of recognition, status, and influence experienced by the Challenge in Chicago. The March 1999 report on the program's first three years states that "The Challenge does not yet have a secure basis for legitimacy" among the education and business leaders who had been surveyed. In late 1997, only about half of the city leaders were familiar enough with the Chicago Challenge even to speculate on what its main activities had been. John Ayers, executive director of Leadership for Quality Education (and brother of Bill Ayers), is quick to point out that the tough work being done by the Challenge and its networks was least likely to get noticed. "They are by definition less likely to gain positive press because they do the hard work at schools," he said.
Ironically, for all its attempts to appear independent, the Chicago Challenge remained closely associated with the 1988 reform groups. To some extent this is due to the relatively small changes that it was willing to make. In fact, the percentage of grants given to school-reform and community-activist groups increased over time, according to the 1999 Consortium report. A full 25 percent of the implementation grants went to these groups in 1995, according to the report. This figure exceeded 50 percent in 1996, when eight of the fourteen grants awarded went to these groups. An additional eight grants were awarded to community organizations and reform groups in 1997. Adjustments were made, yes, but no major effort to break from the reform groups or reconsider the network strategy. In terms of appearances, it didn't help matters much that the Challenge twice shared office space with reform organizations.
Sixty-One Networks and 223 Schools: Uncontrolled Growth or Planned Diffusion?
The list of over sixty networks funded by the Chicago Challenge during the past six years takes up more than a page, including efforts as broad and diverse as the Best Practices Network, the Center for International Technology, the Chicago Comer School Network, the Chicago Middle Grades Network, the Flower Cluster, the Beverly/Morgan Park International Baccalaureate Middle Years Program, the Network for Experiential and Adventure Learning, and the Woodlawn Schools/Community Network.
While pluralism was clearly envisioned in the Annenberg Challenge, it is less certain whether the Chicago Challenge was ever intended to reach over 40 percent of the students and schools in the city. This is especially true considering that many of the networks were focused on time-consuming and complex staff-development initiatives.
Accounts vary as to whether such a large number of networks was envisioned from the start. The Joyce Foundation's Chapman, who helped write the grant application, asserts that the number of networks arose organically out of the work that needed to be done. "The money did not drive the work," said Chapman. "It was the ideas behind what we might do." Still, while the original application stated that no more than ten implementation grants would be given in the first year, that number was exceeded almost immediately. One possible explanation is that the somewhat amorphous nature of the "size, time, and isolation" trinity and the dominant role of the working group contributed to the sheer number of awards given.
It is not known what, if any, pressure was exerted on the Challenge to fund members of the working group, but it is easy to imagine that the group exercised influence on expanding the number of grants awarded to different networks. For perhaps the first year, the working group -- now called the Collaborative -- was highly involved in key activities and decisions. For example, the original sessions outlining how the application process would work were conducted by the Collaborative, according to Bill McKersie. McKersie wrote his 1996 dissertation on the education reform work of several Chicago-area foundations (and at one point was under consideration as the executive director of the Challenge). "The Collaborative was very much running the show," said McKersie of the three community meetings he attended in 1995. "They were way out in front of the staff," he said.
In fact, the working group was responsible for reviewing the first round of applications from organizations in Chicago that wished to be funded. According to Hallett, the group members did not formally approve grants. "There was no intention for the Collaborative to be the grant-making entity," said Anne Hallett. "They simply made recommendations, and in any case we were already staffed [with separate Chicago Challenge employees] when the first grants were made." However, others suggest that the working group/Collaborative remained influential throughout the formative period of the Challenge. In addition, Bill McKersie points out that many of those in control of the process were neophytes. "A set of people who didn't have a lot of experience in grant making got control of the process," said McKersie.
Another element may have been the push to get the first wave of grants out the door as quickly as possible. "Why the rush to move $3 million in 1995?" asked McKersie, who believes that the hurried process of allocating the initial grants hindered the overall effectiveness of the Chicago Challenge by locking it into a set of fixed costs before its processes and ideas were fully formed. "The Chicago Challenge stepped back a couple of decades in the understanding of how private money can spark deep educational change."
This concern is echoed in the March 1999 Consortium report, which notes that the program lost flexibility and was hindered by its 1995 grant-making decisions. The report indicates that the number of grants was early on known to be a problem. "The Challenge's strategic flexibility remains constrained by its early decisions and a rapidly changing environment," states the report. "Grant decisions made in 1995 have strongly influenced the Challenge's direction and obligated much of its available resources."
Intentional or not, the benefits of this approach are doubtful. The decision to support so many different networks almost certainly created administrative, operational, and evaluative problems. With a staff of only nine, the Chicago Challenge could not effectively oversee the difficult and complex work it funded. The networks and schools were spread all across the city, creating logistical obstacles such as increased travel time.
Yet despite the difficulties incurred by funding so many networks -- as well as concerns about the awards that had already been given -- the Chicago Challenge appears to have been unwilling to cut anyone off or limit the number of implementation grants it would give. While not as critical as McKersie, others in the foundation world, such as Paul Reville, agree that controlling quality and retaining flexibility in grants awarded is essential. "The foundations have traditionally built in that wiggle room," said Reville, who cited several cases where a foundation had changed direction or even curtailed efforts when adequate progress wasn't being made. Yet the Chicago Challenge proved reluctant to act in this way even as late as 1997.
The Challenge did change at least some things. As it sought to hone its grant-making operations and refine its strategic role, it delved ever more deeply into the substantive work its grantees were undertaking. In 1997, Challenge executive director Ken Rolling expressed his concerns that networks and school staff needed extra help turning their ideas and good intentions into effective reform efforts. "Just because you build it, they will not come," he wrote.
First, the Challenge developed a more articulated grant evaluation mechanism. Then additional support activities and technical assistance grantees become part of the Challenge's work. Starting in 1997, for example, Rolling tried to help ensure that existing grantees had enough support to do high-quality work, as well as to widen the pool of funded networks to include business and other organizations that had been less successful in winning funds. The Challenge seems to have determined that its role consisted of mailing out grants to as many well-conceived initiatives as it could and helping the networks to implement them as effectively as possible. Given the large number of networks and schools involved, this was no small task.
Despite these improvements, the shotgun approach to grant making resulted in small amounts of additional resources for each school participating in the Challenge. "By the time you divide it among all those schools, it isn't as if any school is getting a lot of money," said Spurlark, who directs a Challenge network. The modest awards made winning the trust and cooperation of individual school leaders a tremendous challenge. Lack of funds also limited the amount of staff-release time and materials that could be provided to support network activities. With only $30,000 to $50,000 per school, networks reported that they struggled to convince principals and local school councils of the merits of their approach. (In comparison, grants under a new federal comprehensive school-reform program start at $50,000 and range upwards of $100,000 per school.)
To be truly effective, the network also had to convince school leaders to deploy their discretionary resources in complementary ways. "You have to be able to establish a central relationship with a principal so that he or she sees you as more than just another project in the school," said MacArthur's Martinez. "You have to become the main consultant. Without that, you're not able to help them look at how the total resources of the school are being used in a way that either aligns with or works against what you're trying to do." Annenberg documents suggest that it was particularly difficult to persuade school leaders to drop preexisting reform initiatives even when they were working at cross-purposes. In some schools, as many as twenty different outside reform efforts are being conducted, Annenberg included. As few as 20 percent of teachers were active participants in some Annenberg schools. About half of the Annenberg schools reported that most of the time available to spend on school improvement was spent on non-Annenberg efforts. Lack of time and resources was also a major impediment to network participation.
The large number of networks and the decision to support existing efforts rather than new ones also curbed the Challenge's influence on each partner to whom it awarded funds. Many of these organizations receive funding from multiple sources. Youth Guidance, the network that supported schools involved in the Comer process, had received $1.5 million for similar efforts from 1990 to 1993. Fifty million dollars in grants to sixty networks translates into less than $1 million apiece, spread over five years. Two hundred thousand a year may have been a massive influx for some, but too little in many cases to give Annenberg a strong role in ensuring the quality or priority of the networks' activities. This situation was compounded by the practice of letting external partners operate more than one Annenberg network at a time. The Chicago Teachers' Center, for example, is listed as the external partner for several different networks, as is National Louis University. As a result, the Chicago Challenge may have been limited in its ability to refocus networks' efforts once grants were awarded.
"What I think they forced themselves into from the get-go was that you just go to too many sites," said McKersie. "Instead of saying, ‘Let's get ten networks going and get some policy changes made,' they just made the aggressive run on getting as many local efforts out there as they could." While a strong defender of the Annenberg effort, the MacArthur Foundation's Martinez agrees to some extent that the large number of networks and schools may have led to what he calls "narrowly-constructed initiatives."
This is not to say that the efforts of the networks or the Challenge had no impact or can't be justified. Indeed, as ill-considered as the large number of networks may seem, this approach may have been somewhat effective. After all, 80 percent of the principals surveyed in late 1996 and in 1997 reported that the Challenge was either central or very central to the work of their schools. The majority reported that participation provided useful resources, and many of the networks working with them have plans to continue after the Annenberg funds are gone. The Chicago Challenge was designed around the ideas of decentralization, school-based reform, and bottom-up change. Any hopes of systemwide revitalization along the lines favored by the Challenge and its supporters may have evaporated fairly quickly. So it makes sense from that perspective that it would award a large number of grants rather than fund fewer but more centralized initiatives. And these efforts then became the "sustaining force" for school-based reform.
Restructuring vs. Accountability: Effects of the 1995 Mayoral Takeover
Seeing an unfamiliar face in his office suite on a cold afternoon in October 1999, Paul Vallas introduced himself and asked if there was anything he could do. Told that the purpose of the visit was to learn about the Annenberg Challenge, Vallas made a face and said, "Oh, them. Well at least they're a little more practical than most of the other groups." Without further comment, he walked away.
Perhaps more than any other factor, the passage of the 1995 reform and the arrival of Paul Vallas had a tremendous effect on the Chicago Challenge. Neither the school-reform groups in Chicago nor the new Challenge staff had imagined all these changes when conceiving of the Chicago Challenge the year before. And no other Challenge site has ended up so enmeshed in a very public war between the district and the school-reform community.
The nearly-simultaneous arrival of the Annenberg grant and the 1995 legislation seems to have surprised just about everybody. "It was strange timing for everyone," said Chapman. By several accounts, very little publicity was given to the possible mayoral takeover of the city schools before it was announced in 1995 and implemented in 1996. "My recollection is that the 1995 mayoral control piece was subterranean," said McKersie. While he had long desired it, a direct takeover was not prominently featured in Mayor Daley's education platform at the time. "It was not even a glimmer in anyone's eye," recalled Anne Hallett.
Under these conditions, developers of the Challenge had no way of knowing that their ideas, and their change model, would be so at odds with the new administration. The approach favored by the Annenberg networks was clearly ill-matched with the views of the new district leadership. The Chicago Challenge was designed for a highly decentralized school system with weak central leadership, not a strong new CEO with broad authority. The Chicago Challenge initiatives were meant to support profound but hard-to-measure changes in school design and operation that would presumably help to improve learning, while the Vallas team was focused on test-based standards and accountability. While the Chicago Challenge plan was conceived to some extent as a safety net for at-risk schools, the Vallas team took a "tough love" approach, most notably by reconstituting schools and ending social promotion of pupils.
The mayor and other city leaders attended the award ceremony in 1995 where the Annenberg funds were presented to the city, but things appear to have deteriorated quickly thereafter. Relations between the Chicago Challenge and the Vallas administration seem to have worsened for a long time, especially as Vallas grew more active in instructional and academic issues. "Since 1995, a lot of the reform groups have refused to play with Vallas," observed John Ayers. In fact, Chicago Public Schools officials report only infrequent contact with Annenberg staff or network leaders. No one in either the school system or the Mayor's office was assigned to work with Annenberg, and many staff members contacted were only minimally familiar with the Challenge. The fact that the Chicago Challenge joined several other foundations and reform groups in a media effort to highlight the successes of the 1988 reform law -- in contrast to the very different thrust of the 1995 reforms -- was a clear indication of just how far apart the two were. (It should also be recalled that City Hall -- and the previous school-system administration -- had been two unsuccessful applicants to Annenberg in 1994.)
Little is known about what efforts, if any, were made to work within the policy context being developed by the Vallas administration. What evidence is available suggests that these efforts were infrequent and ineffective. As Shipps notes in her 1999 report, partnership with the district was a "neglected" objective of the original plan, perhaps because of conflicting views within the Challenge. Even in the more recent grants, only a few of the Annenberg-funded networks were even focused on curriculum and instruction issues that related directly to the accountability measures instituted by Vallas.
The lack of coherence between Annenberg efforts and district strategies did not go unnoticed by the networks and schools involved. Surveys showed that participating schools perceived conflicts between the two, especially when the schools wanted to make scheduling, budget, or staffing changes that required district approval. Network leaders were on the front lines to see this dynamic at work. "Annenberg said early on that they would advocate for the kinds of systemwide changes that would accommodate the changes needed for reform," said Jack Mitchell, who runs an Annenberg network built around the Coalition of Essential Schools model. "I don't know how much success they have had." In the end, it appears that the individual schools were faced with either curtailing their involvement or negotiating their own compromises with district administrators.
It is one thing to observe that the Chicago Annenberg Challenge effort was ill matched initially with the goals, priorities, and top-down approach of the new team running the city schools. But the fact that so few relationships were built between the Challenge and the Vallas team over time suggests that the Challenge was also unable or unwilling to work closely with the district. This lack of cooperation is important because the relationship between the Chicago Challenge and the Vallas administration might have been mutually beneficial, however unlikely a match they were. Other Annenberg sites developed effective -- or at least benign -- relationships with central school system authorities. Other Annenberg sites also survived changes in governance and district leadership.
In fact, the 1995 reform presented tremendous opportunities to Chicago's fledgling Annenberg Challenge effort. In immediate terms, the tough accountability measures in the new school-reform act gave the Annenberg networks greater potential leverage with their schools. Under the 1995 reform, schools could be put on probation and then reconstituted if they continued to lag. Tens of thousands of students were forced to go to summer school in order to make it to the next grade level. Forty-three Annenberg schools were on the initial list of 109 schools on probation. "Now all of a sudden, there are new consequences," explained Martinez of the situation that over one hundred low-performing schools faced, "and a principal says, ‘I know I need help.'"
Few knew the ins and outs of individual schools throughout the city as well as the reform groups working with the Annenberg Challenge. Especially since the 1995 reforms left the local school councils largely intact, the Challenge members could have functioned as a valuable liaison between the central administration and the schools. The 1988 law gave principals the flexibility and autonomy to participate in a network, and resources that could be used for related purposes.
Yet none of this happened. Why is that? It is not surprising that the contradictions between network activities and new district priorities created conflicts. The 1995 reform was a political accountability strategy, while the 1988 reform -- and the Challenge that was in so many ways tailored to it -- was built around the idea of expert-driven school redesign. To that extent, the 1995 reform was fundamentally at odds with the deeply-held beliefs of those who created the Chicago Challenge. Still, it seems hard to imagine that those involved with the Chicago Challenge, with all their experience in past reform efforts and political infighting, weren't aware of the benefits of pragmatic cooperation or the dangers of inflexibility.
Perhaps the Chicago Challenge simply had no room to maneuver. "They threw in with the grassroots reform community where the energy was in the early 1990s," said John Ayers of Leadership for Quality Education. "They bet on that horse. And when the energy and the activism kind of changed towards Vallas, they were already so identified with [the reform groups] that it was hard to change." Several observers report that internal efforts to create more linkage with the Vallas initiatives were repeatedly rebuffed by the Challenge board, the Collaborative, or both. "Ken tried to make political peace and to make accommodations with Chico and Vallas," said Ayers.
Wondering about a potential Vallas-Annenberg coalition is nothing more than Monday-morning quarterbacking, according to at least one of the grant's authors. "The context is different today," said William Ayers, who points out that there was no strong district leadership when the Chicago Annenberg Challenge was conceived. "If we were to do it all over today, we would work on creating a more robust inside-outside partnership with the district."
It may be true that the momentum behind the 1995 reforms was not immediately apparent. But it is hard to deny that the Chicago Challenge had opportunities to revamp its approach during the following years.
Not that adjusting to this massive change or working with the highly controlling Vallas would have been an easy feat. The Chicago Challenge was perhaps the only site to face such a fundamental transformation of governance of the schools with which it was engaged. Few if any of the other Annenberg sites were confronted with district-level initiatives that were so difficult to integrate with established Annenberg priorities. And perhaps no other site encountered as much hostility towards outside reform efforts as came from Vallas, who was quoted in 1997 as saying, "I don't know what I have to do to satiate some of these groups. I don't have time to meet with them and wax nostalgic about the old days of school reform. I have a job to do."
However, Vallas was not the only one who had come to see the reform groups as anachronistic in their approach. "The agitator is not there just to stir things up, but to clean," said James Deanes, who once led one of Chicago's reform organizations and now works in the central administration.
Another view is that, by becoming an independent foundation, the Chicago Challenge simply took the appropriate middle road between being overidentified with the reform groups and becoming an extension of the Vallas administration. After all, the Challenge was funded to promote change and raise the profile of approaches that were not widespread in Chicago -- not to support district policies.
Dorothy Shipps details this reasoning in her 1999 study, which describes organizational and tactical changes made by the Challenge during its first three years. While the conventional wisdom is that the reform groups and the Vallas administration were polar opposites feuding like the Hatfields and McCoys, her analysis suggests that the Chicago Challenge was attempting to span a much broader and more diverse spectrum of ideologies.
Unfortunately, the organization of the Chicago Challenge into an independent foundation seems to have done little to establish a distinct identity or role apart from the community organizations and reform groups -- largely because these same groups remained the chief beneficiaries of Annenberg funding. And it is certainly arguable that some degree of cooperation with Vallas could have been undertaken without any significant erosion of integrity or independence.
To some degree, the lack of conclusive data from the Challenge is also a result of deep ambivalence about another element of the Vallas reign: standardized accountability measures. "Accountability systems neither encourage nor help schools to adopt reflective methods for continuous improvement," stated the national Challenge's 1999 midterm report. Statements like these reflect a deep-seated ambivalence about the intensifying nationwide focus on student achievement. Misgivings about measuring schools and students through tests and other standardized measures remains strong. "You can't come into a situation and say, ‘We're only going to do what's easy to measure,'" says a defiant Martinez. "In the long run, it won't make any impact on the school."
And so what may have prevented any real alteration of course by the Annenberg Challenge in Chicago was the strength of its belief in the previous decentralized approach to reform, and its hostility towards standardized measures of the Vallas sort. Well into 1997, the Chicago Challenge continued to fund proposals from the same pool of original applicants. While almost a third of the grantees identified conflicts between their mission and the Vallas initiatives as a problem, only moderate efforts to refocus the existing Challenge grantees on more achievement-oriented results seem to have been made.
In this respect, the Chicago Challenge mirrored the spirit of the national Challenge. In fact, there has long been a strong anticentralization, antidistrict policy sentiment in Annenberg Challenge literature, suggesting that Challenge staff saw themselves as effective outsiders who should not succumb to the dictates and priorities of district officials.
Yet the Chicago Challenge seems to have gone farther, espousing not only a school- and intermediary-based strategy to change, but also a different objective: school reform through local action. The leaders of the Chicago Challenge disagreed not only with the Vallas approach -- top-down, no-nonsense -- but also with his priority on raising test scores no matter what. And it was this pair of obstacles that may have kept them from even wanting to join forces with him and the mayor.
A Legacy from the Chicago Challenge? Winding Down and Changing Tactics
"It's a constant tension between the inside and the outside," says John Ayers, executive director of Leadership for Quality Education. "The outside needs to be critics, watchdogs, complainers pushing for deeper change -- and yet we also have to work with the district when they're doing good things."
In the end, the Challenge's function as foil or counterweight to the Vallas initiatives may prove to have been its most significant impact on Chicago schools during the past five years. The idea that the Challenge played an important complementary role is expressed by many, including Warren Chapman, who said, "I'm not sure that the education of children should be seen as a competition between two ideologies. However, it was probably the combination of both [Annenberg and Vallas] happening that led to the positive things. Take one away, and I'm not sure what you have." As the grant period winds down, the short- and long-term impacts of Annenberg-funded activities remain unclear. While well aware of problems and missteps that may have occurred in the past, those closely involved with the Challenge remain strongly supportive of it. "Annenberg has been very effective in Chicago and nationally," insists Hallett, "in strengthening and deepening the need for a strong investment in good education at schools." She contends that one of the most important benefits of the Chicago Challenge has been the support for the external partners. "The broad school-reform community that works outside of the schools is extremely important in Chicago," said Hallett. "They are a sustaining force."
Even those who are critical of certain aspects of the Challenge acknowledge that its impact could turn out to be beneficial. "The real question," said McKersie, " is how many of those sixty networks are going to be sustained. If out of the sixty you've got forty that last, that's significant." While acknowledging the faults of the Challenge, this "see what sticks" argument suggests that attention should focus on the most successful schools and networks, however few there may be. Given that many of the networks existed long before Annenberg arrived, it may be that many of the Annenberg-funded networks in Chicago will remain in operation -- although not necessarily because of Annenberg.
Rather than just letting its work peter out, the Chicago Challenge has undertaken two new initiatives to institutionalize its efforts. Some time during the 1999-2000 school year, the Chicago Challenge plans to give dissemination grants to several "breakthrough" schools that were recommended by various networks around the city. These schools are those that the Challenge considers to be model programs that could serve as examples for similar efforts in additional schools into the future.
At the same time, the Chicago Annenberg Challenge has also provided $2 million in seed money to help start a new education fund. Called the Chicago Public Education Fund, this districtwide initiative will focus on developing effective school leadership and recruiting and retaining qualified teachers -- an area of education reform that is clearly defined but that has not been strongly identified with any particular segment of the school reform community in Chicago.
While the exact motivation behind these particular efforts is not known, it is interesting to note that these concluding acts by the Chicago Challenge run somewhat counter to much of its effort over the previous five years. In giving money directly to breakthrough schools rather than networks, the Challenge appears to be reconsidering its fealty to the "external partner" concept that had dominated its work. And in helping start a local education fund, the Challenge is, in effect, assisting district initiatives, not school-based efforts. Giving money to the district is something that the Challenge had long resisted, even in the face of a direct request from Mayor Daley in 1997. While Annenberg literature describes this new fund as an idea being promoted by the Chicago Challenge, district officials depict it as an independent entity with no obligations to continue Annenberg initiatives.
Perhaps in its final acts the Chicago Challenge is facing some of the practical and political realities in a new way. If these concluding efforts do represent a change of