1/21/2010 36-402/608 ADA-II H. SeltmanBreakout #4 CommentsQuestion 1: A manual rank sum calculation requires finding the sum of all of the ranks.Without using a calculator or computer can you find the sum of the integers from 1 to1000?This relates to a story told about Carl Friedrich Gauss. In elementary school the studentswere asked to add all of the integers from 1 to 100. He did it very quickly by realizingthat you can write the sum as (1+100) + (2+99) + (3+98) + · · · + (50+51) = 50*101= 5050. In general the sum of n ranks is (n + 1)n/2.Question 2:Sleuth data problem 29 (Exercise and Walking Times) presents an experiment reported inthe journal Science. The subjects are 12 one-week-old infants recruited as a “conveniencesample” from white, middle class families. The infants were randomly assigned the “ac-tive” group which had stimulation of the walking reflex via four 3-minute long exercisesessions daily from week 2 to week 8. The control group received no stimulation. Thetime of first walking was recorded in months.9.5 10 12 12 14ActiveControlMonths till WalkingCount0 1 2 3 4 5> with(d25, t.test(months~group))Welch Two Sample t-testt = -1.8481, df = 9.976, p-value = 0.09442alternative hypothesis: true difference in means is not equal to 095 percent confidence interval:-3.4929271 0.3262604sample estimates:mean in group Active mean in group Control10.12500 11.70833> with(d25, wilcox.test(months~group, conf.int=TRUE))Wilcoxon rank sum test with continuity correctionW = 9, p-value = 0.1705alternative hypothesis: true location shift is not equal to 095 percent confidence interval:-3.500013 0.750049sample estimates:difference in location-2.000005Warning messages:1: In wilcox.test.default(x = c(9.75, 9.5, 9.5, 9, 10, 13), y = c(9, :cannot compute exact p-value with tiesQuestion 2: Which two tests were done? How do you know that they are both potentiallyappropriate? Which test is likely to be more reliable in the problem and why? Commenton internal and external validity if you know what they are.We see an independent samples (unpaired, two-sample) t-test and a Wilcoxon rank-sumtest (Mann Whitney U test). Either can be used for two indpendent samples (with un-correlated errors). The t-test assumes Normality, but is quite robust to moderate degreesof non-Normality. Both tests assume equal variance. The t-test requires a quantitativeoutcome, while the rank-sum test also works for ordinal data. Although it’s hard to besure with such a small amount of data, there may be a high degree of non-Normality, sothe rank sum test may be preferred.Internal validity is assured by random assignment of treatment, as in this problem. Thisresults in the only difference (on average) between the two groups being due to treatment,as opposed to additional differences due to confounding variables, as in an observationalstudy. Good internal validity allows one to make causal conclusions.External validity refers to how well the tested sample represents the population of interest.A convenience sample my have poor external validity unless the effect we are studying doesnot vary within the population. Poor external validity means that we cannot generalizeour results to a population of interest.Question 3: Permutation test for Walking data### Permuatation test of median difference# For fun, find total number of possible randomizations of 12 items into 2 groupschoose(12,2) # [1] 662# Function to test median differences by permutationfm = function(y, x, nsim=1000, plotit=FALSE) {if (!is.numeric(x)) stop("x must be numeric")if (length(table(x))!=2) stop("x doesn’t have 2 values")vals = as.numeric(names(table(x)))xsim = rep(1:2, table(x))fsim = function(dummy) {xperm=sample(xsim)stat=abs(median(y[xperm==1])-median(y[xperm==2]))}rslt = sapply(rep(1,nsim), fsim)stat = abs(median(y[x==vals[1]])-median(y[x==vals[2]]))p.value=mean(rslt>=stat)if (plotit) {hist(rslt, main="permutation test", xlab="absolute median difference")abline(v=stat)}return(p.value)}fm(d25$months,as.numeric(d25$group), 10000, plotit=TRUE) # 0.2479 (takes 12 seconds)fm(d25$months,as.numeric(d25$group), 100000) # 0.24366fm(d25$months,as.numeric(d25$group), 1000000) # 0.242174 (takes tens of minutes)The top of the fm() function just allows the “x” input, which defines which group “y”belonged to, to be any two numbers. Variable “vals” will be the two possible “x” val-ues. Variable “xsim” is just the numbers 1 and 2 (representing group assignment) repli-cated the right number of times to match the data but in no meaningful order, e.g.,1,1,1,1,1,1,2,2,2,2,2,2 for our data. Function fsim() computes the absolute value of thedifference of medians for the two groups for one particular random group assignment.The sapply() computes a large number (nsim) of these differences. Variable “stat” isthe absolute value of the difference of medians for the two groups for the real data.3permutation testabsolute median differenceFrequency0.0 0.5 1.0 1.5 2.0 2.5 3.00 500 1000 1500 2000Question 3: How is the permutation p-value calculated? How do you interpret theplot? What does the p-value mean? Without computational details, how is the exactpermutation p-value different from the approximate one calculated here? Do you thinkthis test has more or less power than the Wilcoxon test?The perputation p-value is calculated by computing the distribution of the statistic ofinterest (here median difference across treatment groups) over many or all possible ran-domizations of treatment application. The p.value is the probability of seeing the observedstatistic or one more extreme (less like what is expected under H0) based on the computed(empiric) randomization distribution.The plot shows the randomization distribution and observed statistic. Due to the useof absolute value, more extreme corresponds to higher values, so the p-value is the areaunder the curve above the observed value. A small p-value means that the observedstatistic is unlikely under random treatment assignment with an ineffectual treatement,so the treatment probably affects the outcome.An exact p-value would come from computing the distribution of the desired statistic oncefor each possible randomization.The test probably has lower power than the Wilcoxon test, because it only looks at themedians, while the Wilcoxon test looks at all of the ranks.4Question 4: Tail feather experimentWiebe and Bortolotti (2002) examined color in the tail feathers of northern flickers. Someof the