Lecture 5: NoiseLast time•We had a look at how we would generalize the notion of a box plot -- It led us to the idea of “data depth” and a new way to look at things like the median •We then introduced a particular kind of study, the randomized controlled trial, and discussed how simulation, and in particular re-randomization, could be used as a basis for drawing conclusions about the studyToday•We are going to finish up the clinical trial discussion by examining a competing strategy for analysis from Neyman and Pearson -- We’ll see two very different views of how one learns (or doesn’t) from data•We will then discuss random number generation -- Because we rely so heavily on simulation it’s worth discussing how a computer generates random numbers in the first place•We end with a second kind of randomized trial, but one with a much more recent history...Significance testing•In Hill’s Tuberculosis study, of the 55 patients receiving Streptomycin, 4 died (4 out of 55 or 7% of those patients) -- While in of the 52 patients in the bed rest group, 14 died (14 out of 52 or 27%)•To formally test this result, we started with the null hypothesis that Streptomycin was no more effective at preventing death from tuberculosis than bed rest•Under that assumption, we computed a P-value of 0.006 -- This is the chance of seeing 4 or fewer deaths assigned to the Streptomycin group at random•This, one could argue, is very strong evidence against the null -- It is possible that the null is true, and if it is, Hill witnessed an event that should occur in only 1 out of 157 random assignmentsSignificance testing•The setup for the Vioxx trials was largely the same, but with one important difference -- Here we were assuming that the control was safe and we were not expecting Vioxx to reduce the number of cardiovascular adverse events (CE)•In the Bombardier study from a previous lecture, 19 out of 4029 (0.5%) of the patients in the control group experienced CE, while 45/4047 (1.1%) of patients in the Vioxx group experienced CE•Our null hypothesis was that Vioxx and the control had the same propensity to cause CE -- Under that assumption, we computed a P-value of 0.0000004, the chance of seeing 45 or more deaths assigned to the Vioxx group at random•This, one could argue, is amazingly strong evidence against the null -- It is possible that the null is true, and if it is, the researchers witnessed an event that should occur in only 1 out of 2.5M random assignmentSignificance testing•In both the Vioxx and the Hill trials, we conducted a one-sided significance test, that is, we defined the notion of “extreme” in one direction -- for Hill, we considered extreme to mean fewer deaths, while for Vioxx, we took it to mean more•In both cases, you could argue that our notion of extreme came from the problem we were considering: In Hill’s case, there had been previous laboratory work and a few small patient trials that suggested Streptomycin should be better than bed rest; and for Vioxx, the entire class of COX-2 inhibitors had been suspected of causing heart problems and it is generally believed that taking the commonly prescribed doses of naproxen carries no increased risk of CE•Therefore, when we sought evidence against the null (that bed rest and Streptomycin were the same in terms of their ability to treat tuberculosis, or that Vioxx and naproxen had the same impacts on the cardiovascular system) we looked for evidence against the null in one direction -- Had we seen many more deaths under Streptomycin or many fewer incidents of CE under Vioxx, the researchers might have questioned the experimental setup rather than take it as evidence against the nullSignificance testing•Our decision to introduce tests in this way was largely for clarity -- In many (most?) cases, we don’t have quite as much information about the situation and we would be willing to accept departures in either direction as evidence against the null•That brings us to theCannon study for Vioxx (our last one, I promise) -- Unlike the Bombardier trial, the control here was diclofenac, a substance that many believe increases the chance of adverse cardiovascular events•In this case, we aren’t sure what direction to expect (does diclofenac carry a higher risk of CE than Vioxx?) and so we would conduct a two-sided test -- Again, we would take as evidence against the null an extreme event that sees either drug carrying a higher riskSignificance testing•In this study, 9 out of 268 (3.4%) of the patients in the control group experienced CE, while 10/516 (1.9%) of patients in the Vioxx group experienced CE -- Our null hypothesis is that Vioxx and diclofenac had the same propensity to induce cardiovascular adverse events•268 51625995061076919TreatmentStatusCENo CEDRSignificance testing•Our null hypothesis is that both treatment and control carry the same risk of heart attack and we again take as our test statistic the number of heart attacks in the Vioxx group •Under the null hypothesis, the 19 heart attacks would have occurred no matter which group they had been randomized into and our observed data (10 deaths in the Vioxx group) is simply the result of random assignment -- To test this we generate a series of re-randomizations and compare our observed test statistic (10 deaths in the Vioxx group) to the values from the different re-randomized tables•On the next slide we give the null distribution of our test statistic -- Identify where our observed test statistic is on the x-axis...4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19proportion of tablescount of (CE and R)0.00 0.05 0.10 0.15Significance testing•The very small numbers in this distribution (1, 2, 3...) correspond to small numbers of heart attacks in the Vioxx group and a correspondingly large number in the control group -- This would mean the control drug diclofenac is associated with a greater risk of heart attack•Large numbers (16, 17, 18...) correspond to more deaths under Vioxx than under the control -- This would mean Vioxx is associated with a greater risk of heart attack than the control drug, diclofenac•If we have no reason to suspect which drug should be more harmful, both large and small numbers (both tails of the null distribution) would provide evidence against the null hypothesis that the treatment and control are the sameSignificance testing•We now define as extreme counts in the Vioxx group that have
View Full Document
Unlocking...