|
|
Ten Recommendations
For Reporting School Results
To the Public by
A-Plus Communications
 ore and more states and school districts are producing "accountability
reports" that are supposed to inform the public about how schools are performing. The
quality of these reports varies widely. Moreover, few parents, taxpayers, or educators
have ever seen one of these reports.
 |
Report
cards on schools represent a significant opportunity for educators to communicate directly
with their key constituents about how schools are performing. |
In conjunction with Education Week, A-Plus Communications conducted a
national research project to help make reports like these more useful. We examined
existing reports to probe what parents and taxpayers actually want to know about schools,
and to find out how they want the information presented and delivered. We worked with two
prominent public opinion research firms to find the answers.
Educators often complain that the public doesn't understand how schools really are doing
and that parents and taxpayers get a distorted view from the media. Report cards on
schools represent a significant opportunity for educators to communicate directly with
their key constituents about how schools are performing.
Research reports available on this site are presented in portable document format (PDF) which requires Adobe's Acrobat Reader. The available reports include:
For educators, policymakers, and others who want to put the
research findings to use, we offer the following recommendations: 1. Plan Ahead -- And Avoid Wishful Thinking
This may seem gratuitous coming from a communications company, but we [A-Plus Communications] believe it takes
considerable communications planning to produce effective accountability reports. Who
should get the reports? What information should go into the reports, and what should not?
What are the right indicators of school performance that the public wants to know and will
understand? What are the key messages? How will the report cards be disseminated? How will
feedback from the public be collected? Communications planning should start when the
accountability systems themselves are being designed - not at the tail end of the process,
when policymakers and education leaders are scrambling to figure out how best to
"spin" the results.
A top priority is to make sure accountability reports get into the hands of parents.
Neither sending reports home with students nor expecting parents to pick them up at
meetings at schools is enough. A much better approach is a mix of mailing them directly
home, making them easily available at a wide range of locations in the community (from
banks and hardware stores to community centers and supermarkets), presenting the
information at meetings at schools and elsewhere, and making them available on the
Internet.
Going the extra yard to make these accountability reports more widely available to more
audiences is especially important given our research finding that large majorities of
parents and taxpayers have never seen such a report. Indeed, half the educators A-Plus Communications
interviewed said they had never seen or received a school accountability report. This
tracks with our previous research and experience showing that teachers and school staff
often "feel out of the loop."
Yet it has been widely documented that parents rely heavily on teachers for credible
information about schools. At a minimum, we strongly urges administrators and policymakers
to ensure that their front-line communicators, teachers, are very familiar with the
contents of accountability reports - and are prepared to discuss them with parents and
others in the community. This might require some communications training, but this kind of
instruction tends to be the most cost-effective communications investment a district can
make.
In addition, education leaders and policymakers need to understand that what they
want to tell the public in these accountability reports is not necessarily what the
public wants to know. That's a central finding of the research.
Educators are decidedly more skeptical about the benefits of these accountability reports,
and educators perceive the schools to be doing far better than the public perceives school
performance. The public, for example, would tie financial incentives for teachers and
principals to student performance. Educators would not.
Our advice to educators and policymakers: You're better off reporting results on the
public's terms, not your own.
2. Test Scores Provide Only Part of the Picture
Many state and district accountability systems rely heavily on the results of some form of
standardized test. From the public's perspective, these systems may be putting too much
weight on test scores. While the public wants performance data, people are uncomfortable
with relying solely on tests.
Only about a third of parents and a quarter of educators thought that it makes sense to
use test scores as the main measure to hold schools accountable. And even that low level
of support decreased significantly after participants in the large community groups heard
arguments for and against relying on tests.
We suggest reporting test scores within a broader context. Include additional performance
indicators such as student graduation and promotion rates. In addition, show trend data
measuring test scores over several years, not just a single-year snapshot.
3. People Want To Know About Safety, Teacher Qualifications, and Other Measures
School safety consistently shows up as the foremost concern of parents, taxpayers, and
educators. The prototype report card
developed in the research included two "safety" indicators, which parents,
taxpayers, and educators considered useful - number of suspensions and number of violent
incidents per 100 students.
We do not believe we have yet done enough research to know exactly what the right
indicators of safety should be on these reports. And we know that the appropriate
indicators will vary by community. For example, in one of the small focus groups in
Baltimore, Md., one suburban mother said, "I don't want my child to go to a school
that has a metal detector in the front door." But another mother who lives in the
city said, "I would want my kids to be in a school with a metal detector. I'd feel a
little safer because of the way things are now."
Given the priority that parents assign to safety in particular, we suggest that educators
would be wise to do some research to ask their own community which safety indicators are
most appropriate.
Right after safety, parents and taxpayers want to know whether schools have quality
teachers. We suggest reporting information about teachers, such as average number of years
of experience, certification status, and whether they are trained to teach what they are
teaching.
4. Comparisons Count
Parents and taxpayers very strongly want to know how "their" school compares to
others within the district and the state and, to a lesser degree, the nation. They are far
more interested in these comparisons than are educators. Parents and taxpayers want to
know how their school or district is doing compared to previous performance, as well as
how they are doing compared to "the competition." We recommend that report cards
include the percentages of students in a school or district who meet academic standards.
We also recommend that these reports include normative data from standardized tests that
indicate how students in a school or district are doing compared to a national sample.
A few states and districts compare only "similar" schools, those with common
characteristics such as size and percentage of poor children. We suggest caution here.
Parents and taxpayers were not enthusiastic about this kind of measure, at least not if it
precluded comparisons to a wider universe. As one Texas father said: "A kid graduates
from school. Is he just going to be competing against kids who went to similar-sized
schools? No, he's going to be competing against everyone in the state or nation."
The research also found strong support for reporting "growth scores," with
taxpayers, parents, and educators wanting to see test scores that show how individual
students progressed from fall to spring, rather than comparisons of one class to the next
year's class. This strong preference continued even after focus group participants were
told that it would be more expensive to test the same children in fall and spring.
5. Be Cautious About the Labels You Assign to Schools
Several states have begun assigning labels such as "exemplary,"
"adequate," or "low performing" to schools based on academic
performance. Participants in our focus groups did not like these labels. If schools were
to get such an overall score, parents and taxpayers would strongly prefer assigning the
more familiar letter grades such as "A," "B," or "F" as an
alternative. Educators were strongly opposed to the use of either labels or grades.
We applaud attempts such as these to simplify information, but warn policymakers that they
need to do a better job communicating what these labels mean. Our general recommendation
is not to assume that parents and taxpayers will "get" the meaning that is
intended.
Participants in our research also raised a credibility concern about labels. For example,
in North Carolina in 1998, approximately two-thirds of elementary and middle schools were
given ratings of "exemplary." Parents and taxpayers, by margins exceeding two to
one, believed that if two-thirds of schools were reported to be "exemplary," it
would present them with a credibility problem. While they are optimistic that schools can
improve, they do not believe that such a high percentage of students, nor the schools they
attend, are performing at an exemplary level.
6. Downplay Demographic Data
Although demographic data about students (race, primary language, the percentage receiving
free- or reduced-price lunch) are a prominent staple on almost all report cards, they were
ranked lowest by all three groups (parents, taxpayers, and educators) in our research.
Many educators and researchers say that such information provides an important context for
judging student performance - that an impoverished student likely will have a harder time
reaching high standards than an upper-middle-class student, for example. But many members
of the public see this "context setting" either as an excuse for lower
performance or as an inappropriate label for schools and students. Some parents and
taxpayers expressed outrage about the inclusion of this data.
We suggest that policymakers and education leaders probe this issue within their own
communities before deciding to include student demographics on school accountability
reports. At a minimum, we would suggest not making this data a prominent part of school
report cards.
7. Make the Report Easy To Read
Don't overload people with too much data or information. We suggest an
"onion-peel" approach of providing all audiences with a brief,
easy-to-understand report, and offering more detailed information to those who want it.
Through the research, we designed a two-page prototype
school report card that was very well received.
Some people will indeed want more information. For example, parents and taxpayers very
much approved of the kinds of information presented in New York state's report card,
including sample test questions and detailed comparisons, but they were put off by its
11-page length.
Investing in graphic design pays dividends. People are more likely to read a report that's
attractive and well-organized, with clear headlines and lots of white space. This kind of
report does not have to be expensive; the prototype we produced, for example, used readily
available clip art instead of fancy graphics or photographs. People also like the use of
color, although we suggest that those on a limited budget would be better off to put money
into design and writing rather than color printing.
Also, make sure to include short narrative explanations in each section; help readers put
the data in context. The numbers and charts do not necessarily speak for themselves,
especially to non-expert readers.
8. Use Credible Messengers
It's not enough to develop a readable, understandable report. It has to be delivered by
people who are credible in the community. Identifying credible messengers should be part
of the initial communications planning. We tested the credibility of possible sources of
the reports we presented. An accountability report that came from a nonprofit watchdog
group, such as a local education fund or community foundation, would be, by far, the most
credible. Participants rated the next most credible sources - considerably lower - to be
their state education department or the local school district. People rated their local
media (TV, radio, newspapers) at the bottom of the scale of credible sources.
This doesn't mean avoiding the media as a communications resource; but be realistic about
the media's effectiveness. Bottom line: Don't rely on any single messenger or distribution
method. The most effective communications strategy utilizes multiple messengers.
9. Help People Understand How To Use the Information
Many policymakers and education leaders assume that the public will use good
accountability information to force schools to improve. We're not sure. We found a
considerable feeling of powerlessness among many parents and taxpayers.
People who have the option of actually choosing a school or moving to a different
neighborhood are likely to use good accountability information to guide that decision. On
the other hand, the research does not suggest that very many parents, armed with
accountability reports, will descend on school principals demanding changes.
We believe additional research is needed in this area, and education leaders would be wise
to do some of it within their own communities. Meanwhile, educators may want to include in
school report cards practical suggestions for how the public can use the information. For
example, Ohio is developing a prototype school accountability report that offers parents
and taxpayers a number of questions they may want to pursue, such as "What efforts
are under way to improve areas where results are not satisfactory or where the school is
not improving?
10. Use Report Cards as a Tool To Engage Your Community
We're confident these suggestions are an appropriate starting point for any community. But
we know that one size does not fit all.
We urge policymakers and educators to host conversations in their own states and
communities about what their publics want to know to hold schools accountable.
The concept of public engagement is increasingly understood as being essential to
improving public schools. Accountability reports provide an ideal opportunity for engaging
parents and taxpayers about educational issues. One of the most powerful ways that
education leaders can engage their many publics is by asking them this question: What
would you need to see to believe schools are improving? In other words, what counts?
|
