No Formal Evaluation of the Program
In their haste to improve the image of the state's public schools and make teachers accountable for the performance of their students, the SBE and DPI have failed to be accountable to the public by not evaluating the fairness and effectiveness of this high stakes testing program. Scholarly literature documents the potential negative impacts of high stakes testing programs. But, incredibly, DPI implemented the ABCs reform with no plan for its formal evaluation. A report by the National Center for Fair and Open Testing indicates that the state has not evaluated the testing program's impact on curriculum and instruction.55 When asked about the evaluation of the program, Dr. Henry Johnson, Associate Superintendent, points to improvement in scores as evidence of the ABCs' effectiveness. Similarly, he deems the assistance teams effective since none of the 15 low performing schools in 1996-97 were low performing the second year of the program.

Since improvement in scores could be due to a variety of factors, such as changes in teachers, the student body, curriculum, or resources, this evidence is inadequate proof of the effectiveness, the fairness, or the educational value of the ABCs testing program. Furthermore, state officials cannot know what negative effects the program may have if they do not systematically evaluate the program. Authorities on education and testing implore officials to consider what the unintended consequences of testing programs might be and take actions to eliminate them. A report by the National Research Council says, " Determining whether high stakes testing produces better overall educational outcomes requires that its potential benefits be weighed against its potential unintended negative consequences." The report suggests developing statistical reporting systems that will track intended effects such as higher test scores along with other impacts like changes in retention, dropout, special education referral, and teacher turnover rates.56

No Collection of Critical, School Level Data
The annual ABCs report card provides the public with test score data but with no information about other factors like dropout or teacher turnover rates. Volume Two of the ABCs Report Card includes supplemental data on each school system but not for each school. This data includes racial membership, percent of students eligible for free and reduced lunch, percent with learning disabilities, percent classified as academically gifted, average local teacher supplement, and local per pupil expenditure. Presumably, if DPI includes this data as part of the report, the agency must think it could be related to performance and accountability. Within each district, however, these variables vary widely from one school to the next.

If the school is now the unit of accountability, all data, including this supplemental information, should be available from DPI on the school level. Without school level data, DPI cannot determine whether teachers are indeed leaving low performing schools to teach at better performing schools. The agency won't know if a school's referrals to special education classes have dramatically increased since the start of the ABCs. Schools could " push" low performing students into remedial classes or out of school altogether and achieve exemplary status, and no one would ever know this is occurring or how widespread a problem it may be. The collection of school level data is critical for determining the impacts of the ABCs program. Without such data and an evaluation of the program, the ABCs cannot be deemed a success.

Fortunately, DPI has recognized the importance of the dropout rate for high school accountability. Starting this year, the dropout rate for each high school will be incorporated into the ABCs model. In addition, DPI changed the formula for calculating the dropout rate so that it is more accurate. The new guidelines for determining the dropout rate will be used in the 1999-2000 ABCs Report Card.57 This move is especially important given reports that the dropout rate among poor children is increasing.58 DPI should continue to expand its data collection to include more factors that schools should be accountable for and that influence student performance.

Information Is Inaccessible and Unintelligible
Aside from the agency's responsibility to have school-level information so it can evaluate the effects of its program, the agency also has a duty to make information available to the parents, other concerned citizens, and researchers. The American Psychological Association guide on testing standards states, " Those responsible for testing programs should provide appropriate interpretations when test score information is released to students, parents, legal representatives, teachers, or the media. The interpretations should describe in simple language what the test covers, what scores mean, common misinterpretations of test scores, and how scores will be used." 59 DPI has a duty to ensure that parents and students understand what the test scores mean and what purpose they serve. In their evaluation of the ABCs tests, researchers from the University of Alabama reported that only 44% of the students understood their scores, and the scores were useful for only 24% of the parents.60 Although some proponents of the ABCs claim that the program is simple enough for parents and the general public to understand, research shows that parents and students often don't understand what the test scores mean, much less the formulas used to develop growth standards or cut scores.

At the recent public comment session in Raleigh about the proposed promotion standards, several people spoke about the need to get more information out to parents, particularly the parents of low performing children. Speakers at the meeting generally thought that DPI had not done enough to include parents and educators in the process of developing standards and policies or even to inform them of what these standards and policies were once they had been developed. Uneducated and lower-income parents especially lack vital information about the testing program and how it might affect their children. One concerned parent urged the SBE and DPI to create an outreach program to minority and low-income communities so that churches and community centers could provide information and sample testing materials to parents and students. Parents should be able to interpret information about the school which their child attends.

Rewriting Testing Ethics to Fit the ABCs
In addition to not evaluating the ABCs by tracking potential negative impacts or making information accessible to the public, DPI fails other tests of accountability as well. For example, a committee of educators organized by the SBE rewrote the NC Testing Code of Ethics in 1996 to " more closely align it with the current state-mandated testing program and the new ABCs accountability model." 61 Table 2 lists excerpts from the 1988 Testing Code of Ethics and aspects of the ABCs that violate the 1988 Code.62

The revised 1996 code deals predominantly with the security and administration of the tests and only superficially with some of the issues explicitly prohibited in the 1988 Code. For instance, the closest the 1996 Code comes to addressing the practice of assigning students to remedial classes as a way to avoid their testing is its prohibition of " reclassifying students solely for the purpose of avoiding state testing." 63 Yet in the case of Burns High School, where the teachers routinely do just that, the media reports that the testing director " said she consulted with DPI officials and was told there was nothing improper or unethical about the practice." 64 The practice would be explicitly unethical had officials not re-defined the ethical code. With the increasing stakes of the tests for teachers and students under the ABCs program, the current Code should be more, not less, explicit about unethical uses of the tests than the 1988 version.

Unwilling to Allow Exceptions for Special Cases
In addition to rewriting the code of ethics to fit the ABCs testing program, the SBE and DPI further undermine their accountability by their unwillingness to make exceptions to the ABCs for special cases. After the results for 1996-97, nine schools appealed their growth standards. For example, Cape Hatteras Elementary in Dare County asserted that during that year, it was in the process of changing grade configurations and should be tested only on the K-5 portion. Four schools claimed that the results were not valid estimates of the school's achievement given the demographic characteristics of the student population.65 In another appeal, Manteo Elementary had a disabled student who took a computer version of the End-of-Grade test, but the computer malfunctioned and the student could not complete the exam. The student's poor score was included in the ABCs results and cost the school an exemplary rating and teacher bonuses. The school appealed to the SBE that the student's score not be included in their ABCs results.66

During the appeals procedure for the 1996-97 results, representatives from the schools were not given the opportunity to speak. They were not allowed to hear the actual deliberations since the Committee went into closed executive session. As the Principal of Olde Providence School wrote in a letter of complaint to Jay Robinson, then Chair of the SBE, " This was not a hearing. It was a justification with an audience and a rubber stamp vote. Virtually the only presenter was the individual who created the decision that we were appealing." 67 None of the appeals were granted. Similarly, three schools appealed their scores in 1997-98, and none of those appeals were granted.68 A high stakes program like the ABCs requires an open, two-sided appeals process to maintain fairness, yet state officials offer only a perfunctory, one-sided hearing.

In a recent appeals case, DPI and the SBE considered denying high school diplomas to four blind students because they could not pass a reading test in Braille. The students, who were blinded by either disease or accidents in the previous two years, have not yet mastered Braille but have done well on high school coursework by using computers that read aloud their lessons to them. Teachers from the School for the Blind say that it takes at least four years to master Braille and often takes older students even longer. DPI offered a " compromise" to the students: they can have the verbal section of the SAT read aloud to them by a computer and, if they score at least 480, they will receive a diploma. Officials with the Department of Health and Human Services counter that the state's reading exam requires only eighth grade skills to pass whereas the score of 480 on the verbal section of the SAT is close to what the average college freshman scores.69 Finally, DPI conceded that the students could have the standard reading exam read aloud by computer. Their initial unwillingness to allow this accommodation is yet another example of DPI's rigid view of accountability and how its rigidity creates unfairness.

More Money Spent on Testing
Despite examples such as these of the unfairness associated with the ABCs, state officials continue to proceed with the program and its funding without justifying this continued expense to taxpayers. Last year, the ABCs cost approximately $118 million in incentives alone. The salaries and expenses for the assistance teams cost almost $7 million. There are also costs associated with the production, administration, and analysis of the tests. As the SBE adds more tests, such as the ones now under development for English 3 and English 4, even more money will be spent on testing rather than instruction. The SBE should be accountable to the public and demonstrate that this money is funding programs that are improving the education, not just the scores, of NC's schoolchildren.

Disregarding Recommendations for Change
Another questionable decision made by state education officials concerns their disregard for the recommendations made by the Standards and Accountability Commission in 1996. This Commission was established in 1993 to develop well defined education standards that specify the skills and knowledge students should have by certain points in their academic careers. The Commission released a report in July 1996 that criticized the state's testing program as oriented too much to measurement rather than improvement of student performance. To explain their criticism, the Commission made an analogy between the state testing program and athletics:

Imagine if basketball season consisted of one game, played on the last day of the year, in which the players did not know which plays they would be asked to make. Imagine further that they would not know if their shots went in the basket until weeks later. Imagine further if instead of playing the game of basketball, teams of measurement experts invented an arcane series of drills to test with - sensible to measurement experts as valid, but unconnected to basketball playing in the minds of players and coaches. Finally, imagine a scoring system only fully understandable by the assessors, not the players and coaches. Who would improve at the game under these conditions?70

The Commission suggested that students' performance would improve if clear standards were in force daily in the classroom and reflected in instruction, assignments, testing, and grading. According to their report, assessment should involve tasks that simulate real-world use of knowledge. Using this type of assessment, the Commission proposed standards for students in grades 4, 8, 10, and 12.

State officials, however, ignored the Commission's recommendations. The Commission was disbanded in 1997.

One year later, a new Committee on Standards and Accountability was created to advise the SBE on student performance standards. This Committee, along with the SBE, developed the student accountability standards approved on April 1, 1999. Unlike the standards and assessments proposed by the original commission, the new accountability standards rely heavily on multiple-choice tests to classify students for promotion or retention - a purpose for which the tests have not even been validated.

Restoring Accountability
Given their lack of an evaluation plan for the ABCs program, their knowledge of negative, unintended consequences of the program, and their finagling of committees and recommendations, the SBE and DPI have neglected their responsibility for ensuring a quality education for all NC children. Minimally, state officials should take the following actions to restore their accountability and improve the ABCs program: