Comparing Different Groups of Students
In the high school model, the test score of one class of students is compared to the average performance of two previous classes. Every teacher will tell you that each class has different abilities and personalities. Is it fair to hold teachers and administrators responsible for such differences and how they impact scores?

By comparing several years of test data, one researcher demonstrates how the classification of a high school in Guilford County can change dramatically from year to year and class to class. An analysis of this data indicated that, over a period of four years, the percentage of students at the school who scored above Level III varied dramatically from year to year. In 1995, 31% scored above Level III while in 1996, only 16% scored above that level.47

Using the high school model to determine how many points the school improved from one year to the next results in a composite gain score across the EOC tests of minus 14 for 1996 and plus 6.3 in 1997. Thus, the school moves from low performing to excellent on the basis of one class.48 Such fluctuations in high school scores make it difficult to attribute changes in score to the quality of teaching rather than changes in the student body, or other random events.

DPI officials concede that the high school model is not perfect. Currently, DPI is developing a prediction model using students' scores from 8th grade EOG tests in math and reading to predict what these students will score on high school EOC tests. Thus, the agency recognizes that the current model is weak but chose to implement it anyway.

Applying Statistical Adjustments Inconsistently
In their haste to start the ABCs, state officials made other errors in their models. The results for K-8 schools in 1996-97, the first year of the ABCs, indicate that 122 schools were low performing; that number dropped to 15 in 1997-98. While some schools did improve, DPI also changed the formula so that fewer schools would be designated as low performing. Some critics charge that DPI strategically adopted a new statistical procedure in the second year to reduce the number of low performing schools and make it appear that most schools had made significant improvements.49 The agency is vulnerable to this criticism since it applies the statistical adjustment only to potentially low performing schools. If applied to all schools, the number of schools classified as exemplary would also be affected.

Similarly, after the public outcry about the proposed promotion standards, the SBE and DPI decided to adjust the cut-off scores to compensate for the standard error of measurement. This adjustment reduced the estimated number of students who would be retained from 93,000 to 53,000.50 While the SBE made a good decision to allow the use of the standard error in decisions about cut-off scores, the fact that officials did not include the adjustment in their original formulas raises doubts about their commitment to fairness. To minimize error and be fair, the statistical adjustment should have been included in the original formula to determine promotion or retention, just as it should have been included from the beginning in decisions about the classification of schools into the various ABCs categories.

Using Statewide rather than School Specific Statistics
Researchers have also questioned the agency's use of the regression to the mean constant in the ABCs formula for K-8 schools. The use of the statistical adjustment for regression to the mean results in low performing schools having to grow more than high performing schools to meet their expected levels of growth. An educational scholar claims that the use of a regression to the mean constant would be appropriate if officials used the district's or school's average regression effect rather than applying the average statewide regression effect to all schools.51

Similarly, DPI uses a state average standard deviation as the index of the distribution of scores in each school, which means that officials assume that each school in the state will have the same range of scores. Since the standard deviation is used in the formula to calculate growth or gain standards, the use of the state standard deviation disadvantages some schools. Specifically, one researcher has demonstrated that the use of the state standard deviation disadvantages schools who have narrower distributions of scores and unfairly advantages those that have a wider than average distribution.52 The use of a standard deviation specific to each school's range of scores would be more equitable and accurate.