Benjamin Neale March 4th, 2010 International Twin Workshop, Boulder, CO Adapted by Hermine Maes HGEN619 2010 What is power? What affects power? How do we calculate power? What is simulation? Why do we simulate? How do we simulate? Definitions of power The probability that the test will reject the null hypothesis if the alternative hypothesis is true The chance the your statistical test will yield a significant result when the effect you are testing exists Null Hypothesis Alternative Hypothesis Distribution of test statistics Null Hypothesis ◦ The baseline hypothesis, generally assumed to be the absence of the tested effect Alternative Hypothesis ◦ The hypothesis for the presence of an effect Distribution of test statistics ◦ The frequencies of the values of the tests statistics under the null and the alternative We are going to simulate a normal distribution using R We can do this with a single line of code, but let’s break it up R has functions for many distributions Normal, χ2, gamma, beta (others) Let’s start by looking at the random normal function: rnorm()In R: ?rnorm rnorm( n, mean = 0, sd = 1) Function name Number of Observations to simulate Mean of distribution with default value Standard deviation of distribution with default value This script will plot 4 samples from the normal distribution Look for changes in shape Thoughts?-3 -2 -1 0 1 2 3 40.0 0.2 0.4100 observationsTrait ValueDensity-4 -2 0 20.0 0.1 0.2 0.3 0.41,000 observationsTrait ValueDensity-4 -2 0 2 40.0 0.1 0.2 0.3 0.410,000 observationsTrait ValueDensity-4 -2 0 2 4 60.0 0.1 0.2 0.3 0.41,000,000 observationsTrait ValueDensity Sampling variance ◦ We saw that the ‘normal’ distribution from 100 observations looks stranger than for 1,000,000 observations Where else may this sampling variance happen? How certain are we that we have created a good distribution? Rather than just simulating the normal distribution, let’s simulate what our estimate of a mean looks like as a function of sample size We will run the R script mean_estimate_sim.R This script will plot 4 samples from the normal distribution Look for changes in shape Thoughts?-1.0 -0.5 0.0 0.5 1.00.0 0.4 0.8 1.210 sample size mean estimateEstimate of MeanFrequency-1.0 -0.5 0.0 0.5 1.00 1 2 3 4100 sample size mean estimateEstimate of MeanFrequency-1.0 -0.5 0.0 0.5 1.00 2 4 6 8 121,000 sample size mean estimateEstimate of MeanFrequency-1.0 -0.5 0.0 0.5 1.00 10 20 3010,000 sample size mean estimateEstimate of MeanFrequency We see an inverse relationship between sample size and the variance of the estimate This variability in the estimate can be calculated from theory SEx = s/√n SEx is the standard error, s is the sample standard deviation, and n is the sample sizeWhat does this variability mean? Again—this is where you commentThe sampling variability in my estimate affects my ability to declare a parameter as significant (or significantly different)The probability that the test will reject the null hypothesis if the alternative hypothesis is true Mean different from 0 hypotheses: ◦ ho (null hypothesis) is µ=0 ◦ ha (alternative hypothesis) is µ ≠ 0 Two-sided test, where µ > 0 or µ < 0 are one-sided Null hypothesis usually assumes no effect Alternative hypothesis is the idea being testedReject H0 Fail to reject H0 H0 is true α 1-α Ha is true 1-β#β α=type 1 error rate β=type 2 error rate 1-β=statistical powerRejection of H0 Non-rejection of H0 H0 true HA true Nonsignificant result (1- α) Type II error at rate β Significant result (1-β) Type I error at rate α Statistical Analysis Truth The probability of rejection of the null hypothesis depends on: ◦ The significance criterion (α)#◦ The sample size (N) ◦ The effect size (Δ) The probability of detecting a given effect size in a population from a sample size, N, using a significance criterion, α.T alpha 0.05 Sampling distribution if HA were true Sampling distribution if H0 were true β α Power:! 1 - β Standard Case Non-centrality parameterT β α Increased effect size Sampling distribution if H0 were true Sampling distribution if HA were true alpha 0.05 Power:! 1 - β Non-centrality parameterT alpha 0.05 Sampling distribution if HA were true Sampling distribution if H0 were true β α Power:! 1 - β Standard Case Non-centrality parameterT β α More conservative α Sampling distribution if H0 were true Sampling distribution if HA were true alpha 0.05 Power:! 1 - β Non-centrality parameterT β α Less conservative α Sampling distribution if H0 were true Sampling distribution if HA were true alpha 0.05 Power:! 1 - β Non-centrality parameterT alpha 0.05 Sampling distribution if HA were true Sampling distribution if H0 were true β α Power:! 1 - β Standard Case Non-centrality parameterT β α Increased sample size Sampling distribution if H0 were true Sampling distribution if HA were true alpha 0.05 Power:! 1 - β Non-centrality parameterT β α POWER:!!1!)!β!↑!Increased sample size Sample size scales linearly with NCP Sampling distribution if H0 were true Sampling distribution if HA were true alpha 0.05 Non-centrality parameter Type of Data ◦ Continuous > Ordinal > Binary ◦ Do not turn “true” binary variables into continuous Multivariate analysis Remove confounders and biases MZ:DZ ratio Larger effect sizes ◦ Reduce heterogeneity Larger sample sizes Change significance threshold ◦ False positives may become problematicYou can do this by hand But machines can be much more fun Martin, Eaves, Kearsey, and Davies Power of the Twin Study, Heredity, 1978 Just like we simulated estimates of means we can simulate chi squares from dropping C We get to play God [well more than usual] ◦ We fix the means and variances as parameters to simulate ◦ We fit the model ACE model ◦ We drop C ◦ We generate our alternative sampling distribution of statistics This script does not produce anything, but rather creates a function We will make this function and then run the function once to see what happens Then we can generate a few more
View Full Document
Unlocking...