1 STAT 252 Handout 4 Winter 2010 Statistical Significance for 2×2 Tables Example 1: Dolphin Therapy? Swimming with dolphins can certainly be fun, but is it also therapeutic for patients suffering from clinical depression? To investigate this possibility, researchers recruited 30 subjects aged 18-65 with a clinical diagnosis of mild to moderate depression. Subjects were required to discontinue use of any antidepressant drugs or psychotherapy four weeks prior to the experiment, and throughout the experiment. These 30 subjects went to an island off the coast of Honduras, where they were randomly assigned to one of two treatment groups. Both groups engaged in the same amount of swimming and snorkeling each day, but one group (the animal care program) did so in the presence of bottlenose dolphins and the other group (outdoor nature program) did not. At the end of two weeks, each subjects’ level of depression was evaluated, as it had been at the beginning of the study (Antonioli and Reveley, 2005). a) Why did the researchers include a comparison group in this study? Why didn’t they just see how many patients showed substantial improvement when given the dolphin therapy? The following table summarizes the results of this experiment: Animal care program(dolphin therapy) Outdoor nature program(control group) TotalShowed substantial improvement 10 3 13 Did not show substantial improvement 5 12 17 Total 15 15 30 Do the data appear to support the claim that dolphin therapy is effective? A useful first step is to calculate the proportion who improved in each group. b) Calculate these proportions. Did the dolphin therapy group have a higher proportion who showed substantial improvement than the control group? Yes indeed, the proportion who improved is higher in the dolphin therapy group (10/15 ≈ .667, or 66.7%) than in the control group (3/15 = .200, or 20.0%). But is it possible that this difference (.667 vs. .200) could happen even if dolphin therapy was not effective, simply due simply to the random nature of putting subjects into groups (i.e., the luck of the draw)? c) Is that possible? Explain.2 Well, yes, it is possible that the researchers were unlucky and happened to “draw” more of the subjects who were going to improve into the dolphin therapy group. But if 13 of the 30 people were going to improve regardless of whether they swam with dolphins or not, we would have expected 6 or 7 to end up in each group; how unlikely is a 10/3 split by this random assignment process alone? If the answer is that this observed difference would be very surprising if dolphin therapy were not effective, then we would have strong evidence to conclude that dolphin therapy is effective. Why? Because otherwise, we would have to believe that a rare event just happened to occur in this experiment. So, the key question now is how we determine whether the observed difference between the groups is surprising under the assumption that dolphin therapy is not effective. (We will call this assumption the null model. This is the model saying that subjects do not react differently to the treatments.) We want to know, if the null model is true, how often would the random assignment process alone would lead to such a large difference between the treatment groups. We will answer this question by replicating the randomization process all over again, but in a situation where we know that dolphin therapy is not effective (assuming the null model is true). We’ll start with 13 “improvers” and 17 non-improvers, and we’ll randomly assign 15 of these 30 subjects to the dolphin therapy group and the remaining 15 to the control group. Now the practical question is, how do we do this random assignment? One answer is to use cards, such as playing cards. We could take 30 cards, designate 13 of them to represent improvers the other 17 as non-improvers, shuffle them up, and randomly deal out 15 to be the dolphin therapy group. [For example, you might use a regular deck of 52 playing cards, let 13 face cards (jacks, queens, kings, plus one of the aces) represent improvers, and let 17 non-face cards (twos through fives, plus one six) represent non-improvers.] Then once we’ve done this random assignment, we could construct the 2×2 table to show the results to see how many improvers we end up with in each group, where clearly nothing different happened to those in “group A” and those in “group B” – any differences that arise are due to the random assignment process alone. Technical time-saving note: This procedure ensures that the row and column totals to be the same in all of these tables. In other words, we always have 13 improvers and 17 non-improvers, and we always have 15 in the dolphin therapy group and 15 in the control group. A practical benefit of this is that we do not have to calculate the difference in group proportions each time to determine if the result is as extreme as the actual data. All we have to do is check whether there are 10 or more improvers in the dolphin therapy group. This is the case for none of these repetitions, so 0 of the 5 repetitions thus far have produced a result as extreme as the actual data. d) Conduct this randomization process (shuffling the 30 cards and dealing 15 to each group, counting the number of improvers in each group) two times, working with your partner(s). Report the resulting 2×2 tables: Repetition 1: Repetition 2:3 e) Combine your simulation results with your classmates, focusing on the number of improvers in the dolphin therapy group. Produce a well-labeled dotplot. f) Granted, we have not done an extensive number of repetitions, but how’s it looking so far? Does it seem like the actual experimental results (the observed 10/3 split) would be surprising to arise purely from the random assignment process under the null model that dolphin therapy is not effective? Explain. We really need to do this simulated random assignment process hundreds, preferably thousands of times. This would be very tedious and time-consuming with cards, so let’s turn to technology. g) Log onto the computer and open Internet Explorer. Go to the “Dolphins” applet, available from links on our course webpage (www.rossmanchance.com/applets/Dolphins/Dolphins.html). Click on “Randomize” and notice that the applet does what you have done: shuffle the 30 cards and deal out 15 for the “dolphin