TH E PRO FI CI EN CY I LLUSI O N October 2007 Copies of this report are available electronically at our website www edexcellence net Thomas B Fordham Institute 1701 K Street N W Suite 1000 Washington D C 20006 John Cronin Michael Dahlin Deborah Adkins and G Gage Kingsbury With a foreword by Chester E Finn Jr and Michael J Petrilli The Institute is neither connected with nor sponsored by Fordham University OCTOBER 2007 Thomas B Fordham Institute Table of Contents Foreword 2 Executive Summary 6 Introduction 8 National Findings 11 State Findings Arizona 47 Montana 128 California 54 Nevada 135 Colorado 61 New Hampshire 142 Delaware 68 New Jersey 149 Idaho 73 New Mexico 156 Illinois 78 North Dakota 163 Indiana 85 Ohio 170 Kansas 92 Rhode Island 175 Maine 97 South Carolina 180 Maryland 104 Texas 187 Massachusetts 109 Vermont 194 Michigan 114 Washington 198 Minnesota 121 Wisconsin 205 Appendix 1 212 Appendix 2 218 Appendix 3 219 Appendix 4 222 Appendix 5 223 Appendix 6 224 Appendix 7 226 Appendix 8 228 References 224 Table of Contents Foreword By Chester E Finn Jr and Michael J Petrilli No Child Left Behind made many promises one of the most important of them being a pledge to Mr and Mrs Smith that they would get an annual snapshot of how their little Susie is doing in school Mr and Mrs Taxpayer would get an honest appraisal of how their local schools and school system are faring Ms Brown Susie s teacher would get helpful feedback from her pupils annual testing data And the children themselves would benefit too As President Bush explained last year during a school visit One of the things that I think is most important about the No Child Left Behind Act is that when you measure particularly in the early grades it enables you to address an individual s problem today rather than try to wait until tomorrow My attitude is is that measuring early enables a school to correct problems early measuring is the gateway to success So far so good these are the ideas that underpin twenty years of sensible education reform But let s return to little Susie Smith and whether the information coming to her parents and teachers is truly reliable and trustworthy This fourth grader lives in suburban Detroit and her parents get word that she has passed Michigan s state test She s proficient in reading and math Mr and Mrs Smith understandably take this as good news their daughter must be on grade level and on track to do well in later grades of school maybe even go to college Would that it were so Unfortunately there s a lot that Mr and Mrs Smith don t know They don t know that Michigan set its proficiency passing score the score a student must attain in order to pass the test among the lowest in the land So Susie may be proficient in math in the eyes of Michigan education bureaucrats but she still could have scored worse than five sixths of the other fourth graders in the country Susie s parents and teachers also don t know that Michigan has set the bar particularly low for younger students such that Susie is likely to fail the state test by the time she gets to sixth grade and certainly when she reaches eighth grade even if she makes regular progress every year And they also don t know that proficiency on Michigan s state tests has little meaning outside the Wolverine State s borders if Susie lived in California or Massachusetts or South Carolina she would have missed the proficiency cut off by a mile Mr and Mrs Smith know that little Susie is proficient What they don t know is that proficient doesn t mean much 2 The Proficiency Illusion This is the proficiency illusion Standards based education reform is in deeper trouble than we knew both the Washington driven No Child Left Behind version and the older versions that most states undertook for themselves in the years since A Nation at Risk 1983 and the Charlottesville education summit 1989 It s in trouble for multiple reasons Foremost among these on the whole states do a bad job of setting and maintaining the standards that matter most those that define student proficiency for purposes of NCLB and states own results based accountability systems We ve known for years that there s a problem with many states academic standards the aspirational statements widely available on state websites of what students at various grade levels should know and be able to do in particular subjects Fordham has been appraising state standards since 1997 A few states do a super job yet our most recent comprehensive review 2006 found that two thirds of schoolchildren in America attend class in states with mediocre or worse expectations for what their students should learn Instead of setting forth a coherent sequence of skills and content that comprise the essential learnings of a given subject and doing so in concrete cumulative terms that send clear signals to educators parents and policymakers many states settle for nebulous content lite standards of scant value to those who are supposed to benefit from them That s a serious problem striking at the very heart of resultsbased educational accountability If the desired outcomes of schooling aren t well stated what is the likelihood that they will be produced Yet that problem turns out to be just the opening chapter of an alarming tale For we also understood that when it comes to the real traction of standards based education reform a state s posted academic standards aren t the most important element What really drives behavior determines results and shapes how performance is reported and understood is the passing level also known as the cut score on the state s actual tests At day s end most people define educational success by how many kids pass the state test and how many fail No matter what the aspirational statements set forth as goals the rubber meets the road when the testing program determines that Susie or Michelle or Caleb or Tyrone or Rosa is or is not proficient as determined by their scores on state assessments three key questions they sought to answer are straightforward and crucial How hard is it to pass each state s tests The advent of high stakes testing in general and No Child Left Behind in particular have underscored this When NCLB asks whether a school or district is making adequate yearly progress in a given year what it s really asking is whether an acceptable number of children scored at or above the proficient level as specified on the state s tests and how many failed to do so What We Asked In the present