Teachers think that most education reforms are politically motivated - and that means reforms change every few years. Since reforms have come and gone every three years or so, most teachers are skeptical that the ABCs will last more than a few years. One high school teacher would wager on it: " I would bet money that two years after Governor Hunt leaves office, there will be a new reform."

Politically motivated reforms are dangerous not only because they come and go with each election cycle but also because they are often hurriedly implemented without considering the reform's effects, particularly the unintended consequences that may arise. Dating back to mid 1980s, the educational literature has documented negative and often unintended consequences of high stakes testing.11 These impacts include a decline in teacher morale, increased stress among teachers and students, and tracking of minority and poor students into remedial and special education classes. The SBE was aware of these potential impacts and discussed them at their initial meetings about the ABCs.12

In spite of such detrimental impacts on teachers and particularly at-risk children, the Board chose to proceed with a high stakes testing program. As this report will illustrate, the SBE and DPI not only implemented a program that could have deleterious effects, but also did so without a plan to monitor for possible unintended consequences. Furthermore, when teachers and others have brought negative effects of the program to their attention, state officials generally have not dealt with them.

Essentially, DPI and the SBE have not been accountable to the thousands of teachers, schoolchildren, and parents in this state. In their continuing effort " to save the public schools" and maintain their public relations campaign, state officials praise the positive aspects of the ABCs without educating the public on the program's side effects or making changes so that the program is more equitable. This report examines the impacts of the ABCs on teachers, students, and instruction.

More Stress, Lower Morale

Since teachers are at the center of the ABCs reform and could either receive more money or a dismissal based on student performance, a critical question is how this program has affected teachers. For teachers who already feel overworked, underpaid, and under-appreciated, the ABCs program has further diminished their morale. The underlying belief behind the call for accountability is that teachers have not been doing their jobs. Furthermore, with the ABCs incentive structure, the suggestion is that teachers are motivated not by a desire to help students but to receive monetary reward. Many teachers dispute these negative suppositions about the quality of instruction and the motivation of teachers. They also note that, if DPI had included more teachers in the development of the ABCs program, they would not have made such erroneous and patronizing assumptions.

The program puts enormous pressure on teachers to raise their students' test scores. Otherwise, their school may receive a low performing designation, something that embarrasses both teachers and school administrators. A PTA president at a low performing school in Edgecombe County explains the negative spotlight cast upon the low performing schools, especially in the first year of the ABCs program: " Media from around the state came here. It seemed like you couldn't turn on the TV or read the newspaper without hearing about how bad our school was." A survey of teachers by a UNC professor reveals that teachers and, consequently, their students are feeling increased stress under this testing program.13 As a teacher from Guilford County explains, " Teachers want all students to improve, but the program does not need to be so punitive. The stress level is high and the kids feel it."

Teachers Leaving Low Performing Schools
A man who taught at an Edgecombe County elementary school says that almost all of the teachers quit after the school received a low performing designation and a state assistance team. A Raleigh teacher explains that her friend who had taught for 15 years quit in disgust and embarrassment after her school received low performing status and an assistance team. Other teachers know of colleagues who have left teaching altogether because principals, who are also under pressure to obtain high ABCs results, do not want to hire teachers who once taught at low performing schools.

Equating the Worth of a Teacher to Test Performance
An administrator in the Durham County schools comments on what she thinks is the worst aspect of the ABCs: " We've tied the worth of a teacher to a test." She thinks that the state's intentions were to raise expectations and student achievement - goals that she thinks the state has accomplished. But just as it is a challenge not to equate the worth of a student to a test, she thinks it is a challenge for school administrators not to view a teacher's worth in terms of test scores. In fact, one elementary school principal reported that he uses teachers' class scores as a criterion for their annual job review. But in a 1996 evaluation of the ABCs tests, a team of researchers from the University of Alabama concluded that " Using test data as the primary source of information for evaluating teacher effectiveness would almost certainly result in inequitable decisions, degradation in instructional staff morale, and an increased probability of legal challenges." 14

One might even argue that equating a teacher's worth to a test score is an intended effect of the ABCs since the ABCs initially included the proposal that teachers in low performing schools would have to pass a competency test. The proposed test was an exam that was under legal challenge in another state for violating the civil rights of minority test-takers.15 The North Carolina Association of Educators (NCAE) sued the state and claimed that the tests were not a rational way to identify incompetent teachers.16 The teachers' organization also petitioned the General Assembly to change this policy, which lawmakers did one week before the first tests were to be administered. Now teachers must take the test only if the teacher receives an unsatisfactory evaluation from the assistance teams and if the team specifically mentions that the teacher's problem is a lack of general knowledge. Thus far, no teachers have taken the test.

One of the objections to teacher testing and the larger issue of holding teachers accountable for student performance is that teachers cannot be entirely responsible for what their students learn. For example, research has long documented the relationship of student performance to socioeconomic factors such as family income, race, and parents' education level.17 In addition, there are other issues related to test performance such as the individual student's ability to learn, the school's provision of educational resources, and even the specific circumstances of the day when the student takes the test. While all the teachers generally agree that educators should be accountable for doing their job to the best of their ability, they think it is unfair to place the burden for learning entirely on the teacher when factors outside of the teacher's control also impact performance.

As one school system administrator in Durham explains, " We should ask what's wrong with the system rather than what's wrong with the person. When kids perform poorly, there are external factors to consider beyond whether the teacher is adequate or not." These factors include the student's motivation and ability to learn along with the influence of socioeconomic variables such as race and class. The system is inherently unfair when it judges a teacher on the basis of students' test scores - with the consequences of monetary reward or potential dismissal - but does not consider the impact of external conditions on student performance. It is unfair to both teachers and students for officials to focus on teachers as the cause of poor student performance rather than addressing flaws in the educational system that affect the quality of education students receive.

Too often school reforms are politically popular but not educationally beneficial. Do increased scores on the ABCs tests mean that students are generally learning more, or are they learning very specific skills and facts that are relevant to the test? The ABCs program has definitely changed classroom instruction in both positive and negative ways. Most teachers and administrators agree that the program has gotten teachers to focus on the NC Standard Course of Study, the state-mandated curriculum. This focus has meant that educators spend more time and effort teaching the subjects and skills that the State has established as important for a quality education.

Neglecting Non-tested Subjects
Not all educators agree, however, that the ABCs program has positively affected instruction. In a survey of teachers, more than half said that they did not think the ABCs have improved education.18 Teachers question whether instruction has become too narrow and focused. Since students in grades 3 through 8 are tested in reading, writing, and math, teachers express concern that the emphasis on the " basics" has led to decreased resources for art, physical education, and science.19 At a public input session on the proposed promotion standards, a physical education teacher expressed her concern that schools were decreasing the resources they devoted to health and fitness. She, like other teachers, thinks that exposure to a variety of subjects and skills is necessary for a well-rounded education. An elementary school principal in Durham concedes that some attention has been diverted from non-tested subjects. But he thinks that the State will eventually implement tests for subjects such as social studies and physical education and then, the subjects will receive more attention.

Teaching to the Test
The expansion of the testing program to more subjects may increase the attention subjects receive, but it also has drawbacks to consider. As the number of tests increase along with the stakes of the rewards and penalties associated with them, instruction becomes increasingly oriented to test-taking. Quoted in a newspaper article, David Holdzkom, an official with the Durham schools, explains the effects of the ABCs on instruction: " ...We know teachers will teach what we test. So instead of instruction driving the testing, we now have testing driving the instruction. It perverts the relationship between the two." 20 Studies have found that the use of tests for accountability purposes can negatively affect instruction so that teachers teach to the test to the detriment of other subjects and skills which, in turn, undermines real progress in terms of students' achievement.21 A study by National Center for Fair and Open Testing found that students in states with mandatory high school graduation tests were more likely to score lower on National Assessment of Educational Progress (NAEP) tests than students from states with lower-stakes assessment systems.22

With improvement in scores on the ABCs tests, there is no way to determine how much gain is due to teaching to the test or to actual gains in knowledge. A report from Durham Public Schools concludes, " Many teachers are now teaching Ôby rote' in order to ensure success on the test, rather than teaching students in authentic, creative ways that call on students to exercise more than their recall skills." 23 A high school teacher comments, " It is not necessarily bad to teach the test if it focuses on the skills needed for success in a job or higher education, but right now the test itself still needs improvement."

Less Emphasis on Critical Thinking
Before it disbanded in 1997, the state Standards and Accountability Commission also suggested that the state's assessment system needed improvement. In a 1996 report by the Commission, the group concluded, " ...The ABCs accountability plan mandated by the legislature is necessary but alone not sufficient to improve student and teacher performance in North Carolina." 24 The Commission recommended the use of multiple assessments whereby students would apply skills to real-world situations and demonstrate competency in such areas as communication, problem solving, and teamwork. The report asserts, " A good accountability system does more than audit student performance. It is designed to improve performance." 25 For many educators, assessment would also evaluate creativity, imagination, and reasoning - skills that are not measured by multiple-choice tests.

A Lack of Diagnostic Information for Students

Given the limits of the ABCs test to assess such skills, a critical question is whether the testing program really enhances students' achievement or provides useful information about a student's academic progress. As one professor of education explains, " There are tests developed for political purposes and there are tests for educational purposes. That doesn't mean that tests for political purposes are necessarily bad, but they might not be good for educational purposes." One would hope that the ABCs tests provide teachers with diagnostic information that would improve the learning and achievement of their students. In the high school model, in particular, the end-of-course tests seem to serve no educational purpose for the individual students who take them since by the time the teachers have the test results the students have already completed the course.

As part of their evaluation of the ABCs tests, the team of educational researchers from the University of Alabama surveyed teachers and administrators about whether the tests were useful for individual student diagnosis. In terms of the end-of-course tests, teachers and administrators generally thought that the tests were not helpful in diagnosing individual student's progress and problems. Fifty-seven percent of the teachers surveyed did not find them useful.26 These findings, plus the increased practice of teaching to the test, make it questionable whether these tests are educationally desirable.

If the tests are not well suited for individual diagnosis, then what are the effects of the ABCs testing program on students and their learning environment? Like teachers, schoolchildren are also experiencing increased pressure and stress to raise their test scores. This pressure starts early in kindergarten, where standardized assessment is going to begin next year.27

In addition to increasing the pressure on students to perform, unintended consequences of the ABCs program affect certain student groups more than others. For example, some administrators and educational researchers worry that the emphasis on multiple-choice testing and memorization of facts will not challenge academically gifted students. But more evidence exists that schools are shortchanging at-risk students by placing them in remedial and special education classes so that they will be exempt from or receive modifications for the ABCs tests.

The Exclusion of Students from Testing
Some NC educators are excluding students whom they think will lower the school's composite score. A former professor of education at UNCG said that when he enrolled his son in a new school, the principal told him that his son was not welcome in his school if he was going to upset their testing outcomes. Similarly, a teacher said that when new students are seen as low performing, many teachers don't want the students in their classes. A speech pathologist who sits on the special education committee for her school comments that she has seen the number of children designated as exceptional students increase dramatically. If students receive a special education designation like learning disabled, then they will receive testing modifications such as having additional time on the test or the directions read to them.

While it may benefit needy students to be classified in a special education category, it appears that some teachers and administrators are making referrals to special education and remedial classes simply to exclude low performing children from testing. This unintended consequence of the ABCs program was not unexpected, yet DPI did not try to stave it off. Nor when confronted with evidence of this practice has the agency made substantial changes in the reform program.

This evidence includes several complaints to the U.S. Office of Civil Rights that allege that school systems have discriminated against minority and low-income students. These complaints include the placement of disproportionate numbers of these students in remedial courses, so that these students will be exempt from the ABCs testing.28 At Burns High School in Cleveland County, almost 1 in 5 ninth-graders did not take the end-of-course (EOC) test for English I.29 Most of the students who did not take the test were the school's lowest-performing students. Burns was given an " Exemplary" designation last year for improvement in EOC tests. The number of students at grade level on the English I exam went from 47.8% to 58.7%. Both Burns and Crest High School in Cleveland County endorse the practice of removing low performing students from tested courses and placing them in remedial classes until school officials believe they are ready for regular, grade-level work. DPI officials say that this practice does not violate ABC guidelines.

Similarly, in Forsyth County, some teachers are suggesting that one course that requires an EOC test, Economic, Legal, and Political Systems, is too hard for ninth-graders and should be offered later in their high school careers. The superintendent of the system indicated that the teachers want to move the course because they are concerned about ABCs test scores. The school system decided not to offer the course to 500 students this year who read below the ninth-grade level, and instead placed students in remedial math and reading courses.30 An education reform should not allow for such exclusion.

This phenomenon occurs not only in NC but also in other states with high stakes testing programs. One scholar likens the effects of high stakes accountability programs to the effects of a state's decision to rank cardiac surgeons based on their surgical outcomes. After this policy went into effect, surgeons began to turn away difficult patients because they were concerned that the increased risks associated with such patients would affect their rankings.31 To evaluate people based on numeric outcomes, such as the percentage of successful heart surgeries or the percentage of students at grade level, tends to result in the unintended consequence of excluding those patients or students who might lower that percentage.

Not all educators approve of such exclusion. The principal of Shelby High explains why his school does not remove low performing students from tested classes, " To us, it's a matter of ethics. The ABCs were set up to promote accountability. How can you be accountable for students if you don't test them?" 32 Ironically, the move toward accountability has some teachers and administrators shirking their responsibility to teach all students.

A DPI official comments on an effect of the high stakes nature of the ABCs testing program: " It creates gamesmanship. That's a feature of high stakes accountability." 33 Officials should realize that playing games with the educational outcomes of students is inexcusable. If policymakers are concerned about gamesmanship, then they need to make changes to the program to eliminate such exclusionary tactics. Poor & Minority Students at a Disadvantage
The fact that the students who are typically excluded or tracked into remedial classes are predominantly minority and poor raises the issue of how the ABCs models take socioeconomic factors into account.34 The achievement gap between students by race and social class is well documented in the education literature and is reflected in the ABCs results.35 In 1996-97, all but one of the 15 lowest performing schools that received an assistance team were attended by predominantly minority and poor students. On average, 75% of the students were minority and 80% qualified for free or reduced lunch at the lowest performing schools. The lowest performing elementary and middle schools for 1997-98 also had large percentages of minority and needy students.36

A teacher in High Point indicates that her school, which was a low performing school but did not receive an assistance team, has a large population of Hispanic students with limited English skills. She objects to the ABCs models and the rewards and penalties associated with the program because " it is not a level playing field." Another teacher comments about the ABCs program, " Its malicious impact is that it rewards teachers for moving from poor schools to affluent schools. It has always been easier to teach affluent kids, but now the State rewards teachers with more pay for doing it."

DPI claims that the ABCs program is fair because the performance of students at one school is not compared against the performance of those at another school. Instead, the program focuses on the improvement of each school from year-to-year. To claim that it is fair to judge schools based on their improvement from year to year assumes that there is no relationship between the performance level of a school's students and the level of improvement. There is, however, a direct relationship between the percentage of students who are performing at or above grade level and the school's growth or improvement from year to year. Teachers and administrators are well aware of this relationship. Otherwise, they would not try to exclude low performing students from taking the ABCs tests.

A researcher formerly at UNCG analyzed testing data from Forsyth County schools. His research demonstrates that schools with a low percentage of students performing at grade level will probably not meet their expected growth standards while schools with a high percentage of students at grade level will not only meet their expected growth but most likely will meet exemplary growth standards as well. Thus, a direct relationship between the skill level of a school's students and achieved growth exists, which means that the ABCs program for elementary and middle schools is based on the flawed assumption that growth and proficiency are unrelated measures.37 Any school that begins with a cohort of low performing students is clearly disadvantaged under this program.

Our Analysis of Testing Data
An analysis by Common Sense of ABCs testing data illustrates the impact of poverty and race on achievement. This analysis examines ABCs testing data from 1997-98 and its relationship to school funding, the percent of students in the school who receive free or reduced lunch (which is a measure of poverty), and the percent of minority students in the school. The Appendix at the end of the report discusses specifically how the analysis was done. Results show that, as the number of children who receive free or reduced lunch increases, the school's composite score decreases. Similarly as the percentage of minority students increases, the school's score decreases (see Figure 1&2).

In K-8 schools, for every 10% increase in the percentage of needy students, the school's composite score decreases 2.4 points; for every 10% increase in the percentage of minority students, the composite score decreases 1.4 points. In high schools, for every 10% increase in the percentage of needy students, the school's composite score decreases 1.5 points; for every 10% increase in the percentage of minority students, the composite score decreases 2.3 points. Since the composite score stands for the percentage of students who are performing at or above grade level, you can also say that as the percent of poor and minority students at a school increases, the percent of students performing at grade level decreases. Funding is also important for achievement, but its effect is not nearly as significant as the effects of poverty and race.

The pattern of lower scores for schools with more poor and minority students is true for both K-8 and high schools. One striking difference does emerge, however, in the analysis for the two levels of schools. In K-8 schools, poverty is the strongest predictor of score level. For high schools, race is the strongest predictor of score. This difference suggests that the education of young children is more affected by cultural limitations associated with poverty (i.e., parents with less education, nutritional deficits, less regular schedules, and a limited vocabulary). When adolescents reach high school, poverty is still important but the influence of race is greater.

The existence of a previous year of testing data for K-8 schools allows for an analysis of the change in score for each school from 1996-97 to 1997-98. This analysis indicates a relationship between improvement in score and the race and poverty of the school's students. The analysis also illustrates the common statistical phenomenon of regression to the mean - high performing schools have the largest average drop in test scores and low performing schools have the highest average gain. All schools will show year to year fluctuations that are outside the statistical model. Schools with very high scores are more likely to see drops in average scores from random changes and schools with low scores are more likely to drift upward. Basically, when you are down, there is no where to go but up, and when you are on top, there is no where to go but down.

After accounting for this regression to the mean phenomenon, poor and minority schools show smaller gains in scores than schools with lower percentages of poor and minority students (see Figure 3). For every 10% increase in the percentage of needy and minority students, the change in the school's composite score from the first to the second year decreases .8 points. These results cast doubt on one administrator's contention that the ABCs program is the " savior of at-risk kids." Schools with a high percent of poor and minority students generally score lower and show less improvement overall. Proponents may claim that scores are increasing under the ABCs, but the public needs to ask whose scores are really improving because from these analyses, poor and minority students continue to lag behind.

Holding Students Accountable
Now more needy and minority students might literally stay behind since the SBE and DPI have raised the stakes of the ABCs tests even more by using them for student promotion decisions - a move that means more poor and minority students will be retained a grade. The new student accountability standards require students to score at or above Level III on the ABCs tests in reading, writing, and math at the end of grades 3, 5, and 8. To graduate from high school, students must pass an exit exam and a computer skills test.

This use of the tests further complicates the issue of how to untangle the effects of race and social class on student performance. The literature presents a consistent relationship between retention and socioeconomic variables like race, gender, and class. The students who are most likely to be retained are poor, minority, and male.38 Using test scores to hold students accountable involves the same problems as using them to hold teachers accountable: how do you account for factors beyond the student's (or teacher's) control that also impact test performance? It is particularly questionable how officials can take such factors into account when decisions are made from the score on one standardized test.

Teachers and educational observers decry the use of a single test score as the criterion for classification and promotion decisions. A report by the National Research Council concludes, " An educational decision that will have a major impact on a test taker should not be made solely or automatically on the basis of a single test score." 39 Teachers, professors, and state officials generally agree that a test score should not be the sole criterion for major decisions regarding an individual student. But the pressure on teachers to end social promotions and on officials to improve the image of public schools combined with the centrality of the ABCs tests to accomplish these goals means that the test score will most likely - intentionally or not - become the most important criterion for promotions.

The reliance on ABCs test scores for promotion decisions is problematic for other reasons as well. A test that was developed to measure the performance of a group of students is not necessarily a valid measure to gauge an individual student's progress. The team of education experts from the University of Alabama evaluated the end-of-grade (EOG) and end-of-course (EOC) tests and found them to be valid for measuring the progress of a group of students from year to year.40 The researchers did not, however, evaluate the tests for their use as a criterion for individual student classification decisions. DPI and the SBE have a responsibility to establish the validity of such tests, and thus far, they have not provided any documentation of the tests' validity for student classification purposes.

Nor have state officials provided any evidence that social promotion is such a widespread problem that it warrants these new policies. The rhetoric around the push to end social promotions leads most people to believe that few students are held back in our public schools. Last year, however, over 59,000 students were retained in NC public schools. The non-promotion rate in NC has increased steadily since 1991 (see Figure 4).41

The Ineffectiveness of Retention
In addition, the SBE has given the public no documentation that retention is a successful method of helping students develop academically and personally. The educational research provides resounding evidence that retention does not assist students to reach higher levels of achievement. The ineffectiveness of retention as an educational intervention is perhaps the only issue in the educational literature on which researchers agree so overwhelmingly. Out of 66 articles on retention written by scholars from 1990-97, only one supports the benefits of retention.42

Educational studies indicate that retention has little effect in helping students get up to grade-level and, furthermore, has negative effects such as increasing the likelihood that a student will drop out, decreasing subsequent academic performance, and lowering the student's self-esteem. Retention is the second greatest predictor of becoming a school drop-out.43 Being retained one year almost doubles a student's likelihood of dropping out. Retention in two grades almost guarantees the student will drop out. As one scholar explains, " Incredibly, being retained has as much to do with children dropping out as does their academic achievement. It would be difficult to find another educational practice on which the evidence is so unequivocally negative." 44

DPI's own report in 1995 confirms the relationship between retention and continued poor performance. The agency's report on the characteristics of Level I (poorly performing) students indicates that for grade 3 students, 30% that are classified as Level I in reading have been retained at least one grade. That percentage doubles by grade 8 where 64% of the students classified as Level I have been retained at least one grade.45 These statistics indicate that students who have been retained continue to lag behind grade level.

In addition to being ineffectual and detrimental, retention is financially inappropriate. Economically, grade retention is a poor use of the education dollar, because it increases the cost of education (the retained child spends an additional year in the public school system) without any benefits for the vast majority of retained children. This money could be better spent on intervention and remediation strategies to identify sooner what students are at risk of failure and to help those students improve before considering retention as an option.

The new student accountability policy does create a right on the part of students to educational services such as a personalized education plan with diagnostic evaluation and intervention. Unfortunately, the SBE implemented the policy before making sure that the $100 million needed for testing, retention, and remediation would be available.46 Without adequate funds, it is doubtful that students will receive the educational resources and services they need to progress academically. Thus, the SBE has neglected its responsibility to NC schoolchildren by implementing a policy that will increase the number of students retained each year without first evaluating the effectiveness of retention as intervention strategy or even securing the funds needed for helping the students who are retained.