| |
               |
Setting the Bar: How High?
ike most rural towns along the Mississippi Delta, Hollandale has two sets of schools. There are the private schools, attended by all but 47 of the town's white students and anyone else willing to pay the tuition, and the public schools, for everyone else.
Most children in the public schools are poor and black. More than half are being raised by a single parent. Hollandale has no working library, no movie theater, no museum--just a dusty main street, a few nearby fish-processing plants, and acres of rice, wheat, and cotton fields. For all those reasons, it's no surprise that Hollandale's public schools usually get low to middling marks when Mississippi grades its school districts each year. Still, the 1,300-student district has tried hard to raise its academic standing. Administrators extended the school year by 20 days, for example, to provide extra study time for the weakest students. The district also imported a national school reform model for its elementary school, a new algebra program for its high school, and an Even Start program for the families of the town's youngest citizens. The payoff? This year, instead of going up in the state rankings, Hollandale dipped from a mid-range 3 to a bottom-hugging 2 on a 1-to-5 scale. And that, educators there say, isn't right.
"It has been demoralizing in some areas because it just has not been fair," Superintendent Howard R. Sanders says. The state uses test scores to measure a district's worth relative to other districts in the state. But it doesn't take into account the poverty and family problems that educators in towns like Hollandale say make it unrealistic to compare their students with children from more affluent areas. "Those of us in the Delta area get stomped on and jumped on by the news media quite a bit," Sanders says. "And no one's been able to recognize the hard work our teachers are doing." He is among a number of educators around Mississippi--including state Superintendent Richard L. Thompson--who would like to see the 11-year-old rating system changed. Already, a state task force has met on the issue, and more committees are at work. But as Mississippi looks to other states for models of the criteria it can use to hold schools and districts accountable, state policymakers will find a confounding array of alternatives. States can focus on average test scores of students as Mississippi does. Or they can grade schools on the percent of students who meet or exceed a minimum score, or the gains that students--or groups of students--make over time. They can adjust for factors such as poverty that conspire to keep test scores low in places like Hollandale. And they can take into account a school's dropout rate or its average daily attendance. A state can set a high cutoff, knowing that lots of schools will fail, or it can set the bar low to start with and notch it up over time.
"Where states set the bar is arbitrary--hopefully not capricious--but arbitrary," says George F. Madaus, the Boisi professor of education and public policy at Boston College. "There's no outside data that I know of that supports the meaning of those categories. The whole process is more political than psychometric." Yet determining what level of performance is good enough may be the most fundamental question states face in building accountability systems. And a lot hinges on where those academic goal posts are set. In Mississippi, as in some other states, the state can take control of districts that consistently fail to make the grade. Principals and teachers can lose their jobs. And millions of dollars in state aid for troubled schools or rewards for top performers may depend on where a school or district falls in the rankings.
To be considered successful, a district's average test scores have to meet or exceed the state-determined "annual minimum value"--a figure calculated by taking the average score for all districts and subtracting half a standard deviation. Districts are given numerical rankings, ranging from a low of 1 to a high of 5, based on how far their scores diverge from that average. Districts that score a 1 are considered troubled, a status that qualifies them for some extra support from the state and one that could eventually lead to a takeover. Of the state's 153 districts, Mississippi has assumed control of only two so far. To be recognized as high performing, districts must cross a second hurdle as well. Their students have to outscore the average for all the high-scoring districts in the state, a much smaller pool.
That system, educators in the state have long complained, is unfair to poor districts. "We can bring in a child at the zero percentile and move him up to the 45th and not get credit for it," says Sanders, the Hollandale superintendent. For better-off districts, where children often start off achieving at relatively high levels, the climb to the top is shorter. It is no coincidence that virtually all of the bottom-ranked districts in Mississippi lie along the Delta, where the public schools are overwhelmingly impoverished. The proportion of children in a district who qualify for the federal free- and reduced-price lunch program is the single biggest predictor of where a school ends up on the state's rating scale, according to a study by the Program for Research and Evaluation for Public Schools, the research arm for a consortium of 90 Mississippi districts. The research group, based at Mississippi State University, recalculated districts' rankings by determining what each district's predicted score would be if poverty enrollments were taken into account. When a district's actual scores significantly exceeded its predicted score, the researchers singled it out for "adding value" to students' learning. Twenty-seven districts, some poor and some well-off, fell into that category. This time, Hollandale was near the top of the list. "There's a world of difference between basing something on gains and using something based on achievement levels," says Helen F. Ladd, a professor of public-policy studies and economics at Duke University in Durham, N.C. "States clearly want to get all of their students up to some level," she adds. "But the danger is that you might tend to blame the school or the teacher when schools don't get there, but the real reason schools don't get there is they're serving hard-to-teach kids."
To avoid penalizing schools for problems beyond their control, several states--including Kansas, Kentucky, and North Carolina--base their school or district accountability systems on improvements that students make on tests. Kentucky's system, coming as part of the 1990 overhaul of the state's entire education system, is probably the best known. Policymakers there decided that all students should reach "proficient" levels of achievement on a range of state tests by 2012. To set some shorter-term goals along the way, the state calculated the difference between a "proficient" score of 100 and a school's baseline score and then divided by 10. To stay on target, schools must increase their test scores by that increment every two years. Schools that exceed their goals win bonuses. A school is "in decline" when students' test scores drop, and schools where scores drop five points or more must notify parents that they may transfer their children. Under Kentucky's system, low-scoring schools and schools with a high proportion of poor students still have to work harder than others to keep up, but they are not penalized unless student test scores actually drop. As much as it has served as a model, however, Kentucky's system has its drawbacks, and educators and policymakers there are re-examining it as well. "One of the problems with Kentucky is schools might start out by working with students who are near the top to get to 'advanced' or 'proficient' and leave out the bottom," says Edward G. Roeber, a former director of student-assessment programs for the Council of Chief State School Officers in Washington. Another drawback is that very high-achieving schools may not be able to show sustained improvement from year to year. As a result, schools that are among the top-ranked in the state, but that show a slight dip in test scores, may suddenly find themselves declared "in decline." Roeber, now the vice president of Advanced Systems in Measurement and Evaluation Inc., a testing firm based in Dover, N.H., also notes that schools don't always make progress in a steady, linear pattern. "More often," he says, "what you see is a rapid gain and things kind of taper off." The trick, experts say, is finding a middle ground between penalizing schools for a natural lull in progress and keeping districts that make big gains early from coasting on their reputations.
Indiana, which also uses test scores as a measuring stick, tries to account for differing demographics among its schools yet another way. It groups schools into "leagues," similar to sports divisions, based upon their size, socioeconomic characteristics, and students' average scores on cognitive tests embedded in the state assessments. Schools in that state are identified as low-performing when their scores drop significantly below the average scores for all 50 schools in their league. That way, state officials reason, schools are only measured against schools that are in many ways just like them. Tennessee's "value-added assessment" tracks the gains students make on nationally normed tests over the course of their school careers. By comparing students' results with their own past efforts, experts say, the value-added assessment puts students of different backgrounds on a level playing field and focuses on the effect of schooling alone. It also allows administrators to keep tabs on the effectiveness of districts, schools, and even individual teachers. But in trying to make a system fair and effective, educators encounter another dilemma: The more complicated a system becomes, the less confidence parents and policymakers may have in the results. "My first rule is if you can't explain it to a policymaker or a parent or a teacher in five minutes without their eyes glazing over, it's no good," Roeber says. Another drawback is that setting different standards for different schools or districts can send the dangerous message that some students, because of their race or economic circumstances, cannot be expected to reach high standards. Some states, such as Florida, have purposely eschewed ratings systems designed to focus on equity. "We're saying there are no excuses," says Andrea Willett, the director of the office of school improvement for the Florida Department of Education. In that state, every school is expected to have consistently large percentages of students pass state-developed tests in core subjects. "There is no reason that you can give external to a child that should make a difference in how you treat that child," Willett says. As a result, no fewer than 158 schools got flunking grades on the first go-around with the state's new accountability system in 1995. Willett says schools in wealthy and poor districts made the list.
But equity and excellence need not be mutually exclusive goals, some experts argue. North Carolina, for example, is experimenting with a compromise that rewards consistently high-achieving schools yet also recognizes hard-won gains of others. Under the "ABCs plan," launched statewide last year, all schools have to demonstrate that they have provided their students with "a year's worth of learning." That means schools must show improvements in test scores that are at least as large as the gains from previous years. But teachers in both schools that make larger-than-average gains and schools with high percentages of students performing above grade level are eligible for cash bonuses from the state. The biggest problem with the system so far, says Phillip J. Kirk Jr., the chairman of the North Carolina school board, has been that large numbers of schools have qualified for the bonuses. In 1997, the legislature allotted $98 million for the program, but schools qualified for $117 million in bonuses. But, Kirk adds, "it's a nice problem to have."
In Mississippi, the difficulty of finding a balance between fairness and high performance is only one of the flaws critics see in the system. Because the state reports scores at the district rather than the school level, one low-scoring school can drag down an entire district's rating. In Hollandale, for example, Sanders says the local high school is partly to blame for the district's dip in ratings. And, since a district's scores are measured against a state average that fluctuates from year to year, local superintendents often complain of having to hit a moving target. Critics outside the state also note that, in Mississippi's case, measuring scores relative to the state average isn't setting a very high benchmark. Mississippi is a perennial low scorer on the National Assessment of Educational Progress, the congressionally mandated tests in core subjects that are given to nationally representative samples of students. On the Iowa Tests, scores of the state's best-performing districts rank only in the 55th to 60th percentile nationally. "If you only scrimmage against your own team, you never really know how good you are," says Mark D. Musick, the president of the Southern Regional Education Board, a 15-state cooperative organization. Regardless of what system states use, the results tend to get attention--and, some say, results. Of the 158 "critically low-performing" schools identified in Florida in 1995, for example, all had made it off the list by the fall of 1998. But the ease with which schools and districts can remove themselves from the failing lists in some states also raises questions about what such progress really means. In many states, schools show dramatic gains on tests in less than a year. Some of those improvements may be due to better teaching--usually because most states direct at least some extra dollars and advice to the low-performing schools they have identified. "But some may be due to inappropriate activity as well," Roeber of Advanced Systems cautions. Educators may, for example, spend too much time teaching test-taking skills at the expense of regular instruction and, in extreme cases, may even try to cheat. There is little question, though, that test-based accountability systems are becoming a permanent fixture of state-level education policy. The 1994 reauthorization of the federal Title I program for disadvantaged students, in fact, requires states to keep track of the gains students make in Title I schools--a further incentive to keep or build accountability programs. "I think the public and the business community have liked this concept," says Steve Williams, the director of the state office of accreditation in the Mississippi Department of Education. "We're required, and the public expects, that we should judge our educational institutions based on how well our students learn." What Mississippi and other states have to do now is figure out where--and how high--to set the bar.
Howard R. Sanders, the superintendent of the Hollandale, Miss., schools,
says it isn't fair for his Mississippi Delta district, where most of the children are
poor, to be compared with districts in more affluent parts of the state.
|
|||||||||||||||||||||||||||||||||||||||||||||||
   |
||||||||||||||||||||||||||||||||||||||||||||||||
Vol. 18, number 17, page 21 |
