PSYC 455: Research Methods
42 Cards in this Set
| Front | Back |
|---|---|
|
Scientific Method
|
Define the Question & Concepts (using Theory & observation)
Develop Hypotheses (predictions, use logic)
Collect Data to Test Hypotheses
Revise Theory
* Science is self-correcting.
|
|
Four Pillars of Research Methods
|
Internal Validity
Statistical Conclusion Validity
External Validity
Construct Validity
|
|
Internal Validity
|
Causation, correlation, experiments, threats to internal validity, multiple regression
whether the inference is true that one variable (X) is the cause of another variable (Y)
|
|
Internal validity
Independent variable (X)
|
cause, antecedent, predictor, manipulated variable
|
|
Internal validity
Dependent variable
(Y)
|
measured outcome, effect, criterion
|
|
Internal Validity
3 Criteria for inferring Causation
|
X is related to Y
X precedes Y
Rule out alternative explanations for relationship between X and Y (no confounds)
|
|
Internal Validity
Confound
|
a third variable that provides an alternative explanation for the relationship between X and Y
Z is a common cause of X and Y
Spurious relationship between X and Y – relationship is not causal, it’s only due to Z
|
|
How to increase confidence in internal validity?
|
Experiments (the ideal!):
Control group – group of participants who receive no treatment
Random assignment – way of placing participants into treatment group or control group that equalizes all participants in terms of their chances of receiving the treatment
|
|
Threats to Internal Validity
|
History
Maturation
Testing
Instrumentation
Regression-to-the-Mean
|
|
Multiple Regression
|
statistical technique that uses several predictor (X) variables to predict one outcome (Y) variable.
|
|
Statistical Conclusion Validity
|
(Hypothesis testing flaws) – extent to which our conclusion about whether the hypothesis is supported (e.g., rejecting the null hypothesis) is correct
|
|
Descriptive statistics
|
mean, median, mode; variance, skewness; normal (bell) curve
|
|
Inferential statistics
|
use the data at hand to draw conclusions about some larger population (hypothesis testing: p < .05?)
sample correlation (r) => population correlation (ρ)
|
|
Sample
|
group of people from whom you can get data (200 college students at UIUC who were willing to fill out your survey)
|
|
Population
|
larger group of people, whom you believe the sample represents (all 32,000 UIUC college students)
|
|
Inferential Statistics
Null hypothesis
|
(Ho: ρ = 0) – population correlation between X and Y is = zero
|
|
Inferential Statistics
Alternative hypothesis
|
population correlation between X and Y is ≠ zero (different from zero)
|
|
inferential statistics:
p-value
|
given than the null hypothesis is true (ρ < or = 0), what is the probability of getting a smaple correlation (r) as big as the one we got?
if p< .05 reject the null hypothesis
but p=.05 means there is still a 5% chance we reject the null, even when the null is true - an inferential err…
|
|
Type I Error
|
when there is no effect, but our statistical test says there is an effect
a.k.a. “Mirage”
Falsely rejecting the null
False Positive
|
|
Type II Error
|
when there is an effect, but our statistical test says there is no effect
a.k.a. “Blindness”
Falsely accepting the null
False Negative
|
|
Statistical Power = 1
|
Type II Error rate
(probability of detecting true effects)
|
|
Which is worse, Type I Error or Type II Error?
|
Faulty theories (Type I)
Missed Opportunities (Type II)
|
|
Reliability
|
consistency of measurement; freedom from random error
EX:
Internal consistency
Test-retest
Interrater reliability
|
|
Reliability
Internal consistency
|
(multiple items on the same survey instrument are correlated/measure the same thing: Cronbach’s alpha > .7)
|
|
Reliability
Test-retest
|
(Time1 measure correlates with a Time2 measure of the same thing)
|
|
Reliability
Interrater reliability
|
(Two raters’ ratings of the same thing are correlated; performance ratings)
|
|
Reliability
Reliability places a ceiling on validity!
|
Maximum possible correlation between X and Y (rXY) depends upon the reliability of X (rXX) and reliability of Y (rYY)
rXX = .70, rYY = .70, then max rXY = .70
rXX = .70, rYY = .55, then max rXY = .62
|
|
External Validity
|
whether results from your sample are generalizable to another population, setting, set of measures, etc.
|
|
Random Sampling
|
way of selecting participants that gives every member of the population an equal chance of being selected.
Helps support external validity: drawing inferences from sample to population
|
|
Probability Sampling
|
Simple random sampling
Need a complete list of the population members
Stratified random sampling
Select randomly within demographic groups; ensures equal representation in the sample
Cluster sampling
Select groups (orgs., neighborhoods) nonrandomly, then sample within each group [sav…
|
|
Non-Probability Sampling
|
(Does not ensure equal representation)
Convenience sampling
When you have problems with data access
Snowball sampling
When population is hard to find
Nonrandom sampling is bad for external validity—who is the population?
|
|
Meta-Analysis
|
statistical technique for combining data across a bunch of past studies
Get the average correlation, across samples
Compare correlation size across different types of samples
|
|
Advantages of Meta-Analysis
|
Best way to establish external validity
Can actually test whether the relationship between X and Y varies across populations.
Can assess Moderator Variables that influence the correlation between X and Y
Longitudinal vs. Cross-sectional designs
Simple vs. Complex Job Types
Statistica…
|
|
Moderator Variables
|
The relationship between X and Y depends upon M, such that the X-Y relationship is [stronger, weaker] when M is high than when M is low.
Ex) Job market conditions moderate the relationship between job satisfaction and turnover, such that the satisfaction-turnover relationship is weaker w…
|
|
Construct Validity
|
correspondence between a measure or manipulation and the concept that is presumably being measured or manipulated
|
|
Construct Validity has two basic principles:
|
Convergent validity – two measures designed to assess the same construct are strongly correlated
Discriminant validity – two measures designed to assess two different constructs are not strongly correlated
|
|
Construct Validity
Construct
|
psychological concept
(latent, unobservable)
EX: Job Satisfaction
|
|
Construct Validity
Indicator variable
|
observed measure designed to assess the construct
(e.g., survey response)
EX: I'm satisfied with my job.
|
|
Construct Validity
Factor Analysis
|
Factor = latent construct (job satisfaction; represented by an oval)
Indicators = measured variables (the squares)
Factor Loadings = rel’ns between factor & indicators (the arrows)
|
|
Construct Validity
Convergent validity
|
two measures designed to assess the same construct are strongly correlated
(similar items have large loadings on the same latent factor)
|
|
Construct Validity
Discriminant validity
|
two measures designed to assess two different constructs are not strongly correlated
(estimated correlation between two latent factors is less than 1.0.)
|
|
Construct Validity
Common method variance (CMV)
|
source of bias in the correlation between X and Y when both are measured via self-report.
* Correlations between 2 self-reported variables are often inflated!
“Percept-percept inflation”
|
PSYC 455: Exam 3