PSYC300 Exam 2 Study Guide Professor Bernat Book Chapters 4 5 and 6 Internal Validity the experiment tests what it intended to If an experimental design has internal validity it means that variation in the dependent variable could only be due to variation in the independent variable Threats extraneous variable or confounding variable a different variable than the independent variable could be causing the variation in the dependent variable Other threats o History something happens between when the first measurement and second measurement are taken that accounts for the variation o Maturation change in the participants between tests o Testing participants behavior in the second test may change after they have already been tested once o Instrumentation instruments used to take the measurements change over time o Statistical regression people regress towards the mean over time i e if they score an extreme the first time they are tested they are more likely to be closer to the average the second time they are tested o Biased selection the participants in each group of the experiment need to equivalent or it will influence the dependent variable measurements o Experimental mortality participants drop out External Validity the degree to which results generalize beyond your sample and research setting Threats highly controlled laboratory setting restricted populations pretests demand characteristics experimenter bias subject selection bias To increase external validity o Increase sample size so that it is more representative of the entire population o Decrease experimenter bias ways to do this are listed on page 4 of the study guide Internal validity is more important in basic research while external validity is more important in applied research Factors to consider when choosing a measurement Reliability obtain similar results if experiment is repeated Validity the measurement measures what you intended it to To assess reliability To assess validity Test retest find the correlation between the score at time 1 and the score at time 2 o Best for stable characteristics but a problem with learning effects Equivalent Forms same as test retest EXCEPT different forms used each time o Successive measurements o Assume that form A and form B are equal administer form A the first time and from B the second time Split Half correlate a person s score on one half of the items with their score on the other half of the items o Internal consistency o If the scale is reliable correlation between two halves will approach r 1 00 Interrater consistency between raters o Measure the difference between rater A and rater B Construct Validity the extent to which a measured variable actually measures the conceptual variable it is designed to measure Subjective Validity o Face validity it looks like it should work Is the measured variable an adequate measure of the conceptual variable Problems unscientific weak o Content validity how well the content of a test measured what it was intended to Objective validity o Convergent the extent to which a measured variable is found to be related to other measured variables designed to measure the same conceptual variable o Divergent the extent to which a measured variable is found to be unrelated to other measured variables designed to assess different conceptual variables Criterion Validity how well a test score can be used to infer an individuals performance on some criterion measure Predictive Validity predict future behavior o Ex if a job aptitude test can predict actual job performance Concurrent validity predict current behavior o Ex if a self report test can predict an individual s current mood Reliability and Error Actual score True Score random error Reliability true score actual score o The actual score in their measured score from the test o True score is their actual measurement of the variable in real life o Ex a measure score would be your IQ score but a true score would be your true IQ o You want a measure with as little error as possible Scales of Measurement Nominal Scale o Categories have different names order does not matter o Reflects qualitative differences gender ethnicity o Can be assigned numbers male 1 female 2 o Statistical procedures non parametric Ordinal Scale Interval Scale o Categories have different names and are organized sequentially o Can determine differences and direction of the differences o Cannot say how big the differences are o Statistical procedures non parametric o Categories organized sequentially all categories are the same size o Has an arbitrary zero point zero of whatever is being measured does NOT mean an absence of what is being measured ex temperature in Celsius o Can determine the size of the differences o Statistical procedures parametric Ratio Scale o Equal ordered categories with a non arbitrary zero point weight percent correct o Zero is the complete absence of the variable being measured o Appropriate statistics parametric Factors affecting choice of measurement Information yielded o Nominal scale least information o Ordinal scale adds crude information o Interval ration scales most information Statistical tests available o Nonparametric tests used for nominal and ordinal data are less powerful than parametric tests used for interval and ration data o Use scale that allows you to use the most powerful statistical test Ecological Validity o Using a measure that reflects what people do in the real world have scales that have ecological validity o Choose ecological validity even if it means loss of information Difference between measures Overall threats to validity Categorical qualitative data cannot be measured with numbers i e gender ethnicity Continuous Quantitative measures can be measured numerically i e age test scores Reactivity the participant changing their answers or behavior because they know they are being observed o Social desirability natural tendency for research participants to present themselves in a positive or socially acceptable way to the research o Self promotion occurs when research participants respond in ways that they think will make them look good o Reduce threat of reactivity by Administering other self report measures that measure the tendency of the participant to lie or self promote Use a cover story tell the participants that one thing is being measured when the scale is really measuring something else Experimenter Bias experimenter creates the bias by communicating their expectations