Evaluating Psychological Interventions Empirically Supported Treatments Needles and Shots Example Cognitive-Behavioral TherapyShould we evaluate the effectiveness of psychological interventions?Slide 3Slide 4How should we evaluate and measure effectiveness?Three suggested methods for measure effectivenessWithin-subjects designsSlide 8Between-subjects designsSlide 10Slide 11Meta-AnalysisHow do we determine if a treatment is good enough (i.e., valid)?Criteria for “Well-Established Treatments” or Best SupportSlide 15Slide 16Criteria for “Probably Efficacious Treatments” or PromisingSlide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24What is Distress?GoalsLiterature ReviewSlide 28Slide 29Slide 30Slide 31Slide 32Slide 33ConclusionsHow many empirically supported treatments do you think there are for children and adolescents who have psychological/psychiatric disorders?Slide 36Slide 37Slide 38Slide 39Cognitive Behavioral TherapyWhat is CBT?Early Foundations of CBT (behavioral aspects)Slide 43Slide 44Early Foundations of CBT (cognitive aspects)Cognitive TheoriesCognitive TheorySlide 48Slide 49From Cognitive Theory to InterventionD-dispute irrational beliefs Cognitive ReframingPossible Components of CBT or Techniques used as part of CBTBehavioral StrategiesSlide 54Slide 55Slide 56Slide 57Slide 58Slide 59Slide 60Behavioral/Cognitive StrategiesSlide 62Slide 63Slide 64Slide 65Progressive Muscle Relaxation In-Class ExerciseHAPPY HALLOWEENEvaluating Psychological InterventionsEmpirically Supported Treatments Needles and Shots Example Cognitive-Behavioral TherapyGregg Selke, Ph.D.PSY 4930October 31st, 2006Should we evaluate the effectiveness of psychological interventions?CriticismsPatients are too heterogeneous to be evaluated statisticallyPsychotherapeutic interventions are too individualized to be evaluated empiricallyIssues and desired outcome is different for every patientDifficult to define, quantify, and measure “process” of therapy (e.g., rapport, empathy, transference)Fear that “lists” of “effective” treatments will be used by managed care to determine what will and will not be paid for.Should we evaluate the effectiveness of psychological interventions?Yes, and here is why!Estimated >400 forms of psychotherapy1. SpecificityMatching which interventions are most effective to specific problems2. Ethical ObligationResponsibility to clients to use best treatments (supported by research)Should we evaluate the effectiveness of psychological interventions?3. Field AdvancementIncreases credibility of fieldRefines our clinical skills and treatmentsBetter outcomes & cost efficiency4. Support for Theory behind InterventionEvaluates validity of theoretical basis of an intervention under evaluation (e.g., Cognitive Therapy: depression is due to underlying negative thoughts and beliefs, so if person becomes less depressed after changing negative thought patterns, theory supported)How should we evaluate and measure effectiveness?Rigorous Experimental Methods1. Random Assignment to treatment groupsReduces risk of ending up with more severe patients in one group2. Using appropriate control or comparison group(s)3. Using valid and reliable outcome measuresE.g., most sensitive test of depression4. Consistency of therapist(s) across patientsPre- and Post-treatment evaluation1. Within-subjects designs2. Between-subjects designs3. Meta-analysisThree suggested methods for measure effectivenessWithin-subjects designsIndividual acts as own control by undergoing each intervention or non-intervention conditionSingle-Case & Group Experimental Designs1) A-B-A-B design (A=no treatment; B=treatment)Ethics of withdrawing treatment (enuresis vs. depression)Not possible to withdraw some treatments (Cognitive Therapy)Within-subjects designsSingle-Case & Group Experimental Designs2) Multiple Baseline DesignDoes not require withdrawal or reversal of interventionStepwise introduction of components of treatment (A  B  C)E.G., hypothetical treatment for ODDComponent A reduce verbal abuseComponent B reduce noncomplianceComponent C reduce aggressionCan not definitively rule out improvements just due to passage of timeBetween-subjects designsGroups of individuals undergo different or no interventionsIncreases support for improvement actually being due to treatment 1) Nonrandomized Control Group Studies2 “naturally occurring” groups are comparedE.g., ADHD: behavior therapy vs. medsPoor design! No way of knowing if groups differed (in severity, SES) before interventionsBetween-subjects designs2) Randomized Clinical TrialsSubjects are randomly assigned to different conditions/interventions likelihood groups will not differ systematically, or differences will occur more equally across groupsBetween-subjects designs2) Types of Randomized Clinical Trialsa) No-treatment control group (ethical issues)b) Wait-list control group (get treatment later)c) Placebo control group (e.g., double-blind trials)E.g., nonspecific support in psychologyd) Comparing Multiple Interventions E.g., behavior therapy, meds, behavior therapy+meds, wait-list control, & placeboMeta-Analysis“Studies of studies”Statistical procedure to combine the findings of multiple studiesUses the effect sizes (how big the average change due to treatment was), and gives more weight to studies with larger samplesAdvantage: Studies do not have to use the same measuresHow do we determine if a treatment is good enough (i.e., valid)? APA Division 12 and 53Guidelines for identifying and promoting empirically “validated” or supported treatments in psychology. Defining Interventions as1.1.Best Support (“Well-Established Best Support (“Well-Established Treatments”)Treatments”)2.2.Promising (“Probably Efficacious Promising (“Probably Efficacious Treatments”)Treatments”)Criteria for “Well-Established Treatments” or Best SupportI. At least two good between group design experiments demonstrating efficacy in one or more of the following ways:a. Superior to pill placebo, psychological placebo, or another treatment.b. Equivalent to an already established treatment in experiments with adequate statistical power (about 30 per group; cf. Kazdin & Bass, 1989).LCriteria for “Well-Established Treatments” or Best SupportORII. A large series of single case design experiments (n > 9) demonstrating