CHAPTER 4 EXPERIENCE The Myth of Expanding Expertise [It is] important, perhaps imperative, that psychology begin to assemble a body of persuasive evidence bear- ing on the value of specific educational and training experience. -American Psychological Association, 1982' The empirical data indicate that mental health professionals' accura- cy of judgment does not increase with increasing clinical experience, just as their success as psychotherapists does not. There are good logi- cal and empirical reasons why experience does not help in this context, even though we may all "learn from experience" in other contexts. Moreover, there are good psychological reasons why the professionals incorrectly believe that experience does enhance their purported expertise, when it doesn't. The major reasons involve selec- tive recall, selective interpretation, and assumptions about what is likely to be true even though it isn't observed. Why does the American Psychological Association believe that assembling "persuasive evidence" is imperative, as expressed in the quotation that opens this chapter? The reason is not that evidence wasn't assembled, but that the evidence assembled was negative. The body of evidence at the time about psychologists in particular indicat- ed that there was little of any value in their training and experience for their practice. In 1989, Howard Garb summarized the evidence in a Psychological Bulletin article2: Professional clinicians make somewhat better judgments than do nonprofessionals, but that can easily be explained in terms of differences in such characteristics as intelligence and by the fact that people who have learned how to use valid tech- niques employ them better than people who haven't learned to use them. That's not a surprising conclusion, but what may be surprisingExperience 107 is that once the rudiments of the techniques have been mastered, their accuracy does not increase with additional experience using them. That is a very important finding. Any selective advantage that the professional has over the nonprofessional lies in their mastering the basics of a valid technique or two. They learn, for example, that the proper source books about the statistical evaluation of MMPI pro- files will help them evaluate those profiles. They learn the principles of a particular behavioral technique, which leads to its proper use, and such techniques work? The accuracy of the judgment of professional psy- chologists and other mental health workers is limited, however, by the accu- racy of the techniques they employ. That's no different from any other applied field, but what has happened in psychology is that for intu- itively compelling reasons, the myth has arisen that through experi- ence per se a professional can develop accurate use of a "pet" tech- nique that research has shown to be invalid, such as the Rorschach Ink Blot Test. Moreover, however often professional psychologists dis- avow the "medical model," the myth has arisen that the continued practice of a valid technique results in improvement, by analogy with medical procedures such as surgery. The research evidence supports neither of these myths. Garb's generalization that experience does not improve perfor- mance was based on a survey of the research literature evaluating the performance of clinicians who employed a broad variety of tech- niques. One area in which we might expect there to be an exception is in evaluating neurological impairment. Measures of specific types of intellectual functioning have been carefully devised over many years. Ever since the success of tests of general intelligence used to screen United States military recruits in World War I, psychologists have been interested in differentiating various types of intelligence and intellectual functioning. Along with tests of overall "level" of intelli- gence (IQ), tests to evaluate specific types of intellectual abilities have proliferated, and many of them have been well validated. The use of such tests to evaluate the results of brain injury has likewise proliferated, and indeed many tests considered in isolation do deter- mine abilities that are associated with such injury. Using such tests to determine how someone's abilities have changed as the result of brain injury is much more difficult, except in those few cases where the same test was administered prior to the damage-and even then any108 Tk Claims of tk Mental Health Experts Versus tk Evidence changes in performance must be evaluated with reference to the inherent degree of instability in the test results and to any factors other than the injury that might have occurred in the meantime. Given that they generally use such valid tests, neuropsychologists evaluating brain injury might well be expected to benefit from experi- ence in their trade. Not so. In a recent study (published after Garb's review) Faust and his colleagues asked "a nationally representative sample of clinical neuropsychologists" to evaluate the written results of tests of ten peo- ple known to have suffered from specific types of brain injury, or known to have suffered none.4 Faust and colleagues concluded: Except for a possible tendency among more experienced practition- ers to overdiagnose abnormality, no systematic relations were obtained between training, experience, and accuracy across a series of neuropsychological judgments. Comparable results were obtained when analysis was limited to the top versus bottom 20% (in terms of experience) of the sample. This and other studies raise doubts that clinical ne~ro~sychologists train and practice under conditions conducive to experiential learning. Why did the American Psychological Association's committee believe it "imperative" to "assemble a body of evidence" for some- thing that isn't true? The reason I propose is that the success of psy- chology and often mental health professions stems from the public's belief that experience does enhance professionals' performance. After all, it does in many other professions, and therefore it must in the mental health professions as well. But although professional psycholo- gy is proclaimed to be based on "the science of psychology," it nonetheless sees a need to provide evidence that experience enhances performance, rather than admit that it doesn't and implement changes in its practices accordingly.