COMM 301: Empirical Research in CommunicationExperimentsSlide 3Experiments Advantages vs. DisadvantagesA note on causality (remember lect2 ?)Factors influencing the amount of control in experimentManipulation of independent variableEnsuring group equivalenceHow to ensure group equivalenceHow to ensure group equivalence (cont.)Slide 11Control of intervening variablesExperimental designsSettingAmount of Control – High ControlPretest-posttest control group designSlide 17Slide 18Solomon four-group designSolomon four-group design interpretation exerciseAmount of Control – Moderate ControlSlide 22Slide 23Amount of Control – Low Control11COMM 301:Empirical Research in Communication Lecture 10Kwan M Lee22Experiments•Things to know by the end of the lecture:–Know advantages and disadvantages of experiments–Know how to achieve experimental control to get causal inference•Factors influencing control•Specific techniques to ensure control–Know the various experimental designs, in terms of•Understand logic behind each design•structure, how they look like•advantages and disadvantages•How to apply33Experiments•Experiment–method that allows us to evaluate the influence of some independent variables on some dependent variables–while controlling for other intervening variables44ExperimentsAdvantages vs. Disadvantages•Advantages–Allow causal inference–Replication possible by someone else.•Disadvantages–Lack ecological isomorphism–Weak generalizability55A note on causality (remember lect2 ?)•3 requirements for causal relationships–Temporal ordering: cause precedes effect in time–Meaningful correlation: must have a theoretical foundations for observed correlations–No alternative causes (hypotheses): correlation cannot be explained by other factors•Control: the use of techniques for systematically ruling out alternative causes–Alternative hypotheses exercise•E.g. no wash on an exam day 1) no wash is lucky vs. 2) more time spend on reading66Factors influencing the amount of control in experiment•Manipulation of independent variable•Ensuring group equivalence•Control of intervening variables77Manipulation of independent variable•Active manipulation–Researcher determines participants’ level of exposure to independent variable.–Give some assurance that if dependent variable changes, it is due to the independent variable–E.g., default experiment setting•Passive observation–Cannot be under control of researcher–Natural setting•E.g.,Comparison of two existing classes which have different teaching methods88Ensuring group equivalence•Experiments use different groups of participants•treatment group (treatment)•control group (no treatment)•comparison groups (when two or more treatment groups) •Group equivalence assumption–Must ensure that before imposing the treatment, the groups are equivalent with regard to the dependent variables.–How to insure?  see next slides99How to ensure group equivalence•Random assignment–Participants assigned to treatment or control, or comparison groups by randomized method.–No guarantee of getting equivalent groups•But it usually works!–Can use statistical testing to find out how likely the group is non-equivalent.•Comparisons of variables not related to dependent variables1010How to ensure group equivalence (cont.)•Pretesting–Groups are measured on the dependent variable(s) before any treatment.–Expectation is that the groups will be similar•If not, non-equivalent with regard to DVs1111How to ensure group equivalence (cont.)•Matching •Participants in the groups are matched on characteristics important to dependent variable(s) (i.e. matched on intervening variables)–Constancy matching•All participants in all groups kept uniform on the characteristic thought to influence the dependent variable.•e.g. holding gender constant: test only males–Pairing•Each participant in a group is matched with other participants in other group on the variable(s) thought to affect the dependent variable(s)•e.g. for each male (female) in treatment group, there is a male (female) in the control group – i.e., equal # of males and females1212Control of intervening variables•Eliminate the influence of:–participants (through ensuring group equivalence)–settings (through ensuring settings are different only with regard to manipulations)–individual researchers•use of double-blind procedures•use of scripts.–other factors influencing internal validity (e.g., maturation, attrition.)1313Experimental designs•Experimental research designs are categorized along four dimensions–Research settings–Amount of control–Number of independent variables–Different subjects vs. Same subjects in cells1414Setting•Experimental design setting–Laboratory•Research setting created by the researcher, maintains strong control over the setting •Sacrifice ecological isomorphism–Field•Naturally occurring research setting, with little researcher control•Enhance ecological isomorphism•e.g. new training program at a company1515Amount of Control – High Control•Pretest posttest control group design–Good control for group equivalence•Randomized assignment to treatment and control groups•Pre-testing–Handle most threats to validity well.–Limitation: test sensitization–See Graph in p. 861616Pretest-posttest control group design•T: treatment group•C: control group•R: Random Assignment•Pr: Pretest•X : Treatment (intervention)•- : No treatment (intervention)•Po: Posttest1717Amount of Control – High Control•Posttest only control group–Randomized assignment to treatment and control groups for group equivalence–No pre-testing, sacrifice some group equivalence–Attempts to counter test sensitization in pretest posttest control group design1818Amount of Control – High Control•Solomon four-group design–Combines pretest posttest control group and posttest only control group–Highest level of control•Ensures group equivalence through randomized assignment and pretesting•Allows assessment of test sensitization’s impact (T1 vs. T2; C1 vs. C2; C1 [Pr vs. Po])–Limitations:•Costs, especially in subjects•Difficulties in interpreting contradicting results1919Solomon four-group design2020Solomon four-group design interpretation exerciseGroup 1 R Pr1 (60) X Po1 (82)Group 2 R X Po2 (74)Group 3 R Pr2 (62) Po3 (70)Group 4 R Po4