Exam Guide 3 etc 1 What are the properties of a good survey question e g wording should be simple Use simple words Make the stem of a question short and easy to understand Avoid vague questions Don t ask for too much information in one question Avoid check all that apply items Avoid questions that ask for more than one thing Soften impact of sensitive questions What are demographics 2 Where should different types of questions go in your survey e g where should you put items that are asking about similar concepts where should you put sensitive questions Organize questions into a coherent visually pleasing format Do not present demographic items first Use an interesting question as your first item Keep related items together continuity Be aware that question order can make a difference Place sensitive or objectionable items after less sensitive objectionable items Establish a logical navigational path 3 What are the ways to increase the reliability of your questionnaire e g a survey with more questions is typically more reliable Increase number of items but be careful not to make the survey overlong Standardize administration procedures Be careful of scoring errors check twice Make sure questions or answers are clear piloting helps 4 Be able to visually identify different types of survey questions Open Ended Restricted Partially Open Ended Rating Scales and know the properties advantages and disadvantages of each Open Ended Introduces a topic and allows each participant to respond in his own words Information more complete advantage Participant may not understand what you are looking for dis Some answers omitted inadvertently dis Summarizing data difficult dis Restricted example In the past year how many faces have you met none 1 5 6 10 ect Limited set of responses advantage Ordered sets of responses gives quantitative information advantage Missing alternatives dis Information not as rich dis Partially Open Ended example Who introduced you to the American you most recently met face to face family co worker classmate other be specific Rating Scales Participants rate the question on a Likert or Likert typed scale Scale of 1 10 how much do you like something Interval level information advantage Easy to understand and answer advantage Response set dis 5 Please review validity with respect to surveys Content validity e g does a political survey ask about all relevant Concurrent Validity e g do the results agree with another survey measuring the same dimension administered at the same time Criterion related validity is your survey predictive of behavior or other topics beliefs 6 There will be multiple questions about variance What are the components of the variance in your data e g total variance systematic variance error variance Why is error variance a problem Where does error variance come from What are the 4 ways to deal with it control it randomize it across conditions increase effectiveness of your IV and stats Error Variance Variability in scores caused by extraneous variables or participant variability Reduce Error Variance Treat all participants the same Match participants on variables that have an effect on DV Increase Effectiveness of IV Use levels of IV that are very different E g Effect of Alcohol on Memory Sober vs 05 BAC Sober vs 10 BAC Randomize Error Variance Across Group Participants have an equal chance of being in any group Error variance between groups will tend to balance out But not always which is why we need Statistical Analysis What is the probability that the size of the effect we observed happened just by chance as a result of error variance Stats programs give us a p value 5 chance typically considered acceptable 7 Know the properties and advantages and disadvantages of Randomized 2 Group designs Randomized Multi Group designs Matched Pair Matched Multi Group designs Within Participant Designs and Factorial designs Randomized 2 Group Designs Advantages Simple Relatively few participants required Data and stats easy easy to interpret No pre testing required to ensure equality of groups we rely on randomization Disadvantages performance May be insensitive to effects when participants differ greatly in their Doesn t yield a large amount of info Differences but not shape of function relating IV and DV Randomized Multi Group Design The logic of the 2 Group Design can be extended to 3 or more groups may be necessary to rule out alternate explanations Matched Group Design Controls participant related variability by matching groups on characteristics that influence performance ex age has a large effect n cognitive ability we may want to math groups on age BY accounting for this participant related variability we may be more likely to see the effect of the IV By matching participants on characteristics that influence performance this is less of an explanation for why groups might differ Systematic variance easier to observe Matched Pairs Design Measure your sample find pairs of people who match on characteristics that might influence performance For each pair randomly assign which person goes into which group Example Kansas Video Game Intervention Study Matched Multi Group Designs Find similar participants then randomly assign these participants to groups Disadvantages More difficult to implement Need to measure all participants before study If you have many groups may be difficult to find a lot of people who match Requires the use of slightly less powerful statistics So make sure the thing you are matching on really does influence DV Within Subjects Designs Taking matching to the extreme Think of a between subjects design as matching each participant with someone who is just like them You re matching each participant with him or herself AKA Repeated Measures Design Each participant exposed to all levels of the IV DISADVANTAGE More demands put on participants longer study more complicated study may increase drop out rate 8 Know specifically what advantage a matched design provides and when it provides it How does it control for error variance There is a specific example of when a matched design actually hurts your ability to detect an effect When does this happen and why Matched Group designs help to control for error variance By matching participants on characteristics that influence performance this is less of an explanation for why groups might differ Systematic variance easier to observe 9 Please know about carryover effects and the ways to deal with them
View Full Document
Unlocking...