Experimental design Experiments are the most common and useful way to determine causality. The question is: how to design them?What is a good experiment? Hard to say... Experimentation is partially an art form! But, in general, a good experiment is one that controls for everything else -- testing only what you want it to test Planned well in advance Controlling for other plausible hypothesesThe first experiment An Early Experimental Design:“Dew it both ways” Judges (Shoftim) Chapter 6 And Gideon said to G-d: “If You will save Israel by my hand, as You have said, look, I will put a fleece of wool on the threshing-floor; if there be dew on the fleece only, and it be dry upon all the ground, then shall I know that You will save Israel by my hand, as You have said.” And it was so; for he rose up early on the next day, and pressed the fleece together, and wrung dew out of the fleece, a bowlful of water. And Gideon said to G-d:“Do not be angry with me, and I will speak just this once: let me try just once more, I ask You, with the fleece; let it now be dry only upon the fleece, and upon all the ground let there be dew.” And G-d did so that night; for it was dry upon the fleece only, and there was dew on all the ground.An experiment vs. experiments Any experiment is a step in a dialog Experiments build on each other No experiment is the ultimate oneThe art of setting experiments Industry standards Elegance Symmetry Many ways to measure the same idea What is convincing evidence?Single cell designs Not an experiment -- a study Parameter estimations Theory test Examples: economics, physics Creating a control group from the single cell Examples: ordering simulation in variety seekingRandom assignment Random assignment is the central aspect of experiments By randomly assigning subjects to groups we can assume no systematic differences between groups Assume they are the same on everything but the manipulation.The importance of manipulations By using a manipulation you can determine the direction of causality Based also on random assignment we can link the outcome to our manipulationOne-factor designsOne-factor designs XX.XX XX.XX Level 1 Level 2 DVOne-factor designs 80 60 40 20 0 Level 1 Level 2 80 60 40 20 0 Level 1 Level 2 Level 3 Level 4Multiple-factor designsMultiple-factor designs Factor 1 Factor 2 XX.XX XX.XX XX.XX XX.XX Level 1 Level 2 Level 1 Level 2The importance of interactions No interaction 100 Gender 1 Gender 2 80 20 40 60 0 Low HighThe importance of interactions Ordinal interaction Assumptions? 100 80 60 40 20 Gender 1 Gender 2 0 Low HighThe importance of interactions Dis-ordinal interactionAssumptions? 100 Gender 1 Gender 2 80 60 40 20 0 Low HighThe importance of interactions Ordinal Disordinal Interactions help us rule out alternative explanations Examples…..Within vs. between subject designs Which is better? Why What are the advantages of each? What are the disadvantages of each?Experimental design: summary Experimental design is about the design of the structure of the experiment The origins of this is in agriculture Without a good design it is hard to know for sure what we have learnedSummary Experiments Random assignment Controls Types of experiments Once cell, one factor, multiple factors The role of interactions Within and between subjects
View Full Document
Unlocking...