1PowerPower and Sample Size and Sample SizeAdapted from:Adapted from:Boulder 2004Boulder 2004Benjamin NealeBenjamin NealeShaun PurcellShaun PurcellI HAVE THEPOWER!!!OverviewOverviewIntroduce Concept of Power viaIntroduce Concept of Power viaCorrelation Coefficient (Correlation Coefficient (ρρ) Example) ExampleDiscuss Factors Contributing toDiscuss Factors Contributing toPowerPowerPractical:Practical:••Simulating data as a means ofSimulating data as a means ofcomputing powercomputing power••Using Mx for Power CalculationsUsing Mx for Power Calculations2Simple exampleSimple exampleInvestigate the linear relationship betweenInvestigate the linear relationship betweentwo random variables X and Y: two random variables X and Y: ρρ=0 vs. =0 vs. ρρ≠≠00using the Pearson correlation coefficient.using the Pearson correlation coefficient. Sample subjects at random from Sample subjects at random frompopulationpopulation Measure X andY Measure X andY Calculate the measure of association Calculate the measure of association ρρ Test whether Test whether ρρ ≠≠ 0. 0.How to Test How to Test ρρ ≠≠ 0 0Assume data are normallyAssume data are normallydistributeddistributedDefine a null-hypothesis (Define a null-hypothesis (ρρ = 0) = 0)Choose an Choose an αα level (usually .05) level (usually .05)Use the (null) distribution of the testUse the (null) distribution of the teststatistic associated with statistic associated with ρρ=0=0t=t=ρρ √√ [(N-2)/(1- [(N-2)/(1-ρρ22)])]3How to Test How to Test ρρ ≠≠ 0 0Sample N=40Sample N=40r=.303, t=1.867, df=38, r=.303, t=1.867, df=38, pp=.06 =.06 αα=.05=.05Because observed Because observed pp > > αα, we fail to, we fail toreject reject ρρ = 0 = 0Have we drawn the correct conclusionHave we drawn the correct conclusionthat p is genuinely zero?that p is genuinely zero?αα= type I error rate= type I error rateprobability of deciding probability of deciding ρρ ≠≠ 0 0(while in truth (while in truth ρρ=0)=0)αα is often chosen to equalis often chosen to equal.05...why?.05...why?DOGMA4N=40, r=0, nrep=1000, centralN=40, r=0, nrep=1000, centralt(38),t(38),αα=0.05 (critical value 2.04)=0.05 (critical value 2.04)Observed non-nullObserved non-nulldistribution (distribution (ρρ=.2) and=.2) andnull distributionnull distribution5In 23% of tests that In 23% of tests that ρρ=0, |t|>2.024=0, |t|>2.024((αα=0.05), and thus correctly=0.05), and thus correctlyconclude that conclude that ρρ = = 0. 0.The probability of correctlyThe probability of correctlyrejecting the null-hypothesis (rejecting the null-hypothesis (ρρ=0)=0)is 1-is 1-ββ, , known as the power.known as the power.Hypothesis TestingHypothesis TestingCorrelation Coefficient hypotheses:Correlation Coefficient hypotheses:hhoo (null hypothesis) is (null hypothesis) is ρρ=0=0hha a (alternative hypothesis) is (alternative hypothesis) is ρρ 00Two-sided test, where Two-sided test, where ρρ > 0 or > 0 or ρρ < 0 are < 0 areone-sidedone-sidedNull hypothesis usually assumes noNull hypothesis usually assumes noeffecteffectAlternative hypothesis is the ideaAlternative hypothesis is the ideabeing testedbeing tested6Summary of PossibleSummary of PossibleResultsResultsH-0 trueH-0 trueH-0H-0falsefalseaccept H-0accept H-01-1-ααββreject H-0reject H-0 αα1-1-ββαα=type 1 error rate=type 1 error rateββ=type 2 error rate=type 2 error rate1-1-ββ=statistical power=statistical powerRejection of H0Non-rejection of H0H0 trueHA trueSTATISTICSR E A L I T YNonsignificant result(1- α)Type II error at rate βSignificant result(1-β)Type I error at rate α7PowerPowerThe probability of rejectingThe probability of rejectingthe null-hypothesis dependsthe null-hypothesis dependson:on:the significance criterion (the significance criterion (αα))the sample size (N)the sample size (N)the effect size (NCP)the effect size (NCP)“The probability of detecting a given effect size in a population from a sample of size N, using significance criterion α”P(T)Talpha 0.05Samplingdistribution if HAwere trueSamplingdistribution ifH0 were trueβαPOWER = 1 - βStandard CaseEffect Size (NCP)8P(T)Talpha 0.1Samplingdistribution if HAwere trueSamplingdistribution ifH0 were truePOWER = 1 - β ↑Impact of less conservativeImpact of less conservativeαβαP(T)Talpha 0.01Samplingdistribution if HAwere trueSamplingdistribution ifH0 were truePOWER = 1 - β↓Impact of moreImpact of moreconservative conservative αβα9P(T)Talpha 0.05βαImpact of increased sample sizeReduced varianceof samplingdistribution if HA istrueSamplingdistribution ifH0 is truePOWER = 1 - β↑P(T)Talpha 0.05Samplingdistribution if HAwere trueSamplingdistribution ifH0 were trueβαPOWER = 1 - β↑Impact of increase in Effect SizeEffect Size (NCP)↑10Summary: Factors aff ectingSummary: Factors aff ectingpowerpowerEffect SizeEffect SizeSample SizeSample SizeAlpha LevelAlpha Level<Beware the False Positive!!!><Beware the False Positive!!!>Type of Data:Type of Data:Binary, Ordinal, ContinuousBinary, Ordinal, ContinuousResearch DesignResearch DesignUses of power calculationsUses of power calculationsPlanning a studyPlanning a studyPossibly to reflect on ns trend resultPossibly to reflect on ns trend resultNo need if significance is achievedNo need if significance is achievedTo determine chances of studyTo determine chances of studysuccesssuccess11Power Calculations viaPower Calculations viaSimulationSimulationSimulate Data under theorized modelSimulate Data under theorized modelCalculate Statistics and Perform TestCalculate Statistics and Perform TestGiven Given αα, how many tests p < , how many tests p < ααPower = (#hits)/(#tests)Power = (#hits)/(#tests)Practical: Empirical Power 1Practical: Empirical Power 1Simulate Data under a model onlineSimulate Data under a model onlineFit an ACE model, and test for CFit an ACE model, and test for CCollate fit statistics on boardCollate fit statistics on board12Practical: Empirical Power 2Practical: Empirical Power 2First getFirst gethttp://www.vipbg.vcu.edu/neale/gen619/phttp://www.vipbg.vcu.edu/neale/gen619/power/power-raw.mx and put it into yourower/power-raw.mx and put it into yourdirectorydirectorySecond, open this script in Mx, and
View Full Document
Unlocking...