November 2006 | Volume 64 | Number 3 NCLB: Taking Stock, Looking Forward Pages 28-31 Needed: Higher Standards for Accountability Paul E. Barton While holding schools accountable, NCLB fails to hold itself to basic evaluation standards. Four prescriptions would change that. During the last quarter century, a model for school standards and accountability has emerged in the United States that is now so locked into state and federal laws that its general shape seems here to stay. If this model is indeed here to stay, we need to keep working to get it right. The school reform movement, known in its beginning years as standards-based reform, was founded on the proposition that the education system needed to establish rigorous content standards, prepare teachers to teach that content, and align instructional materials and curriculum with those standards. A further proposition was that we could measure a school's performance in this effort by drawing on existing testing systems, such as norm-referenced tests, that in the past had been used largely to make decisions about individual students. NCLB puts teeth into the practice of using such tests to measure school accountability because it requires states to impose sanctions on schools whose students do not reach designated cut points for “proficient” performance on schedule. NCLB also requires schools to dis-aggregate test results by race, ethnicity, and other factors; and all subgroups of students must meet the standards. This system clearly looks to standardized tests to hold schools accountable. But the accountability system itself has some accountability problems. I suggest four new directions our school accountability system needs to take. Apply Basic Evaluation Standards for Using Tests for Accountability From the beginning, tests used in the school accountability system failed to meet the high standards designed into the system (Barton, 2006). The 1994 amendments to the Elementary and Secondary Education Act that required states to establish content standards and tests gave careful attention to what was needed to produce meaningful test scores in a standards-based reform system. The handbook commissioned by the U.S. Department of Education (Hansche, 1998) told states, Systems of performance standards and assessments must be created or selected and matched with the content. In an aligned system, all content standards must be accounted for in some manner .... Content standards, performance standards, and assessments must be aligned so that what is taught is tested and what is tested is taught. (p. 21) According to assessment expert W. James Popham, content standards frequently embrace a November 2006 Page 1 of 4ASCD11/27/2006http://www.ascd.org/portal/site/ascd/template.MAXIMIZE/menuitem.459dee008f99653f...wider range of content for a grade and a subject than a course can realistically cover (Popham, 2004). In this case, aligning a test to a set of content standards is not enough. In such cases, school systems need to prioritize what they plan to teach so that teaching requirements match what is tested. Otherwise, the test will not be sensitive to instruction and test scores will not reflect the results of improved instruction. Alignment was incomplete in 2001 when NCLB was passed, and it is still incomplete. The American Federation of Teachers (an early supporter of standards and tests) and the Fordham Foundation have evaluated how well all states have met alignment requirements, and their reports agree that there are deficiencies in alignment (American Federation of Teachers, 2006). Other organizations, such as Achieve, have evaluated individual states. Nevertheless, as soon as NCLB was passed, the Department of Education immediately began using existing test scores to hold schools accountable. Lists of schools “in need of improvement” were compiled, and the sanctions clock started to run. Measurement experts are explicit about what makes a test “valid” in an accountability system. Where alignment among curriculum, instruction, and assessment is incomplete, the assessment does not meet standards for validity, and we cannot rely on changes in test scores to judge whether schools have become more or less effective. Unfortunately, such scores are being used for sanctions whether the tests meet validity standards or not. Hold Schools Accountable for What Goes On in School Despite all the testing, our present accountability systems do not reliably sort out effective from ineffective schools. Our current methods simply do not measure the change in the knowledge of astudent from point A to point B—for example, from the beginning to the end of the school year. Thus they fail to reflect the educational progress over time of any student, class, or school. So what do our standardized tests now measure? They measure, for example, what students know about a subject at the end of the 8th grade. Our current accountability system compares such scores against a level—or cut point—that someone has judged as “proficient.” Then, it compares these scores with the scores of 8th graders 1 or 5 or 10 years ago. But comparing whatcertain students know now with what different students knew at the end of past school years tells us little about the quality of instruction. These past students may have known more or less when they entered 8th grade than did the 8th graders more recently tested. Different groups of students will enter 8th grade with a variety of backgrounds in schooling, preparation, and family resources. These differences will have provided each individual with more or fewer advantages forlearning. A test score at the end of 8th grade reflects all the learning a student has gleaned from the first 13 years or so of life. But to hold schools accountable, we need to know how much knowledge students gained in the course of a school year. And we still need to know what students in various groups know at each time we measure: total knowledge at a point in time. The latter data tell us how we are doing, as communities, as states, and as a nation in the entire learning enterprise. A number of large research studies (Barton, 2006) have measured school progress both ways: student gain during the school year and total knowledge at the end of the year. In each study, the correlation between the two measures was low. This means that some schools whose students are making considerable gains in knowledge in each grade are