MASON PSYC 612 - Lecture 10: Planned Comparisons

Unformatted text preview:

PSYC 612, SPRING 2009Lecture 10: Planned ComparisonsLecture Week: 3/17/2009Contents1 Preliminary Questions 12 Part I: Introduce Multiple Comparisons (50 minutes; 2 minute break) 12.1 Purpose: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.2 Objectives: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.3 Describ e the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.4 Statistical Power and Alpha Inflation . . . . . . . . . . . . . . . . . . . . . . . . . . 32.5 Multiple Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.5.1 A prior comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.5.2 Post-hoc comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Part II: Beyond the readings (5 minutes; 2 minute break) 73.1 Purpose: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 Objectives: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Part III: Introduct ion to Linear or Matrix Algebra (cont.) (20 minut es; timepermitting) 71 Preliminary Ques tions•Any questions on mediation or moderation?•Did you read the Klockars and Sax text?•Did you enjoy your spring break?2 Part I: Introduce Multiple C omparisons (50 minut es; 2minute break)12.1 Purpose:Provide the background for multiple comparisons2.2 Objectives:1. Describe the problem2. Disucss the relevance of power and alpha inflation3. Introduce several solutions2.3 Describe the ProblemWe often analyze nominal data containing multiple levels (e.g., dog, cat, fish, and bird as levels ofpets). There are no problems with doing so but we need to be careful. Consider the simple datasetbelow. I have four sets of pet owners and their corresponding lifespans in years. I want to knowwhether owning a particular type of pet extends life expectancy.pet Longevity1 dog 90.742 dog 86.693 dog 86.674 dog 102.035 dog 71.146 cat 77.087 cat 66.868 cat 75.709 cat 101.6510 cat 89.9611 fish 64.5112 fish 73.2913 fish 68.6414 fish 63.0015 fish 71.1816 bird 60.2817 bird 72.2718 bird 72.8119 bird 64.1720 bird 86.02How might we analyze the between group differences given that there are four groups? If werun a simple, one-way ANOVA, we get the following results.Notice that the main effect is significant. What does t hat mean? The main effect for pet is anomnibus test tha t tests the following hypothesis:H0: µdog= µcat= µfish= µbird2Df Sum Sq Mean Sq F value Pr(>F)pet 3 1250.52 416.84 3.92 0.0283Residuals 16 1700.50 106.28So yo u can see that if the p-value is less than .05, we reject the null that all four pet ownerslive the same number o f years. OK, fine. Pet owners appear to differ in life expectancy. So if Iwere to recommend a pet if you wanted to live longer than other pet owners, which pet would Irecommend? The ANOVA results merely tell us that the four pets differ but not which pet is thesup erior pet. We are not interested in that omnibus but rather the simple effect comparing o ne petspecies with anot her pet species. The omnibus just t ells us if there is a difference among the groupsthat leads us to reject the null hypothesis.Instead of running a one-way ANOVA, I could just run multiple ((k2− k)/2) t-tests. So if wehad 10 different types of pets, we would have 45 t-tests. That seems like a problem, no?2.4 Statistical Power and Alpha InflationMultiple comparisons present problems for both statistical power and alpha inflation. Recall thatstatistical power is the probability that we reject the null given that the null is false. Also rememberthat β is the Type II error rate and power is 1 − β or the probability tha t you will no t make aType II error. As the number of tests increase, your power increases. You might think this is agood thing because high statistical power is good but it is not necessarily good because of how youmight have come to reject the null. Multiple tests inflate the alpha and, as a result, make it mucheasier to reject the null. Think of the p-value as a hurdle. If the critical p-value increases, then thehurdle gets lower. Multiple tests increase the effective, experimentwise error. In o ther words, thep-cricital va lue increases. Multiple tests increase power by increasing alpha. So t he question thatremains is how do we run multiple comparisons without these untoward effects?There are two issues to address. First, how do we run multiple tests. I intend to cover that inthe following section. The second issue is how to protect against the untoward effects of multipletests. I intend to cover that in Part II of the lecture today.2.5 Multiple ComparisonsMultiple comparisons come in two different flavors - a priori comparisons and post-hoc comparisons.I detail each below in that order.2.5.1 A prior comparisonsWe may choose to run comparisons according to a specific theoretical question or according to ahypo thesis. My or ig inal example data offers us a glimpse at this situation. Suppose I expected dogowners to be the most likely to live long lives and I wanted to compare dog owners to the ownersof all other pets. Those multiple comparisons are said to be a priori because we are posing thosequestions before we conduct the a nalysis. A priori comparisons are ones where specific comparisonsare made at the exclusion of other comparisons. We would not run all possible comparisons amongour pet owners if we had a priori multiple comparisons in mind. Comparing dog owners with allother owners requires a simple “dummy code” procedure whereby each pet type gets compared todogs. The codes for this type of contra st look like this:3cat fish birddog 0 0 0cat 1 0 0fish 0 1 0bird 0 0 1What the heck does this mean? Well to setup the contrasts, we need to create new variables.The first variable is called “cat”, the second is called “fish” and the third is call “bird.” Note thatthere are …


View Full Document

MASON PSYC 612 - Lecture 10: Planned Comparisons

Download Lecture 10: Planned Comparisons
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Lecture 10: Planned Comparisons and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Lecture 10: Planned Comparisons 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?