Survey Methodology 2 5 14 Establishing Causation Does X cause Y Watching TV Obesity Three requirements needed to establish causation Covariation Temporal precedence Non spuriousness Covariation As X changes Y remains the same No covariation or correlation present A change in X is CORRELATED with a change in Y Watching TV increases more likely to be obese Watching TV decreases less likely to be obese Correlation DOES NOT imply causation Temporal Precedence The cause X precedes the effect Y in time o Aka you are ruling out possibility of reverse causation Chicken or the egg Which came first o Does watching TV cause obesity OR do obese people tend to watch more television Non Spuriousness Y No plausible alternative explanation for relationship observed between X o AKA ruling out potential confounders o Third variable confounders o Example shoes size intelligence There is a positive correlation between shoe size and intelligence problem age could be a confounding variable as we age out shoe size gets bigger and we also get smarter from more schooling Age is creating a false relationship between these two variables Validity Internal validity External Validity Validity Trade Off Survey Research Why The extent to which we can accurately state that the independent variable cause the observed effect within the study aka it wasn t something else The ability to generalize to or across exemplars of a particular to the entire class of a particular Populations causes effects environment A study with high external validity REALISTIC Objective produce comparable information data across many people in a sample 1 Describe characteristics opinions behaviors o How much time do HS students spend on Facebook 2 Examining relationships between measured variables o Is exposure to the state of the union correlation with support for the president Example Research Question What is the relationship between exposure to local news programming and fear of crime Survey items o Has watching local TV news made you more afraid of crime This is bringing up the fear putting it in their head o INSTEAD how fearful are you of being a crime victim in your neighborhood AND How often do you watch local TV news Administering Surveys Types of surveys Mail cheap slowest lowest RR cannot clarify no interviewer influence Online cheapest fastest moderate RR cannot clarify no interviewer influence influence interviewer influence Telephone moderate fast moderate RR can clarify some interviewer Face to Face expensive slow to moderate highest RR can clarify most Survey Designs Cross sectional Surveys survey ONE sample at ONE point in time ex relationship between children s amount of TV and reading level can establish covariation cannot establish temporal order you don t know which comes first Longitudinal Surveys survey samples TWO OR MORE points in time can establish covariation and temporal order 1 Trend study same population different sample 2 Panel study same population same sample observes changes within individuals over time more precise difficult expensive attrition Surveys can only PARTLY address the threat of alternative explanations by Measuring potential confounders in surveys Adjusting for them in statistical models Cant measure everything relatively low internal validity BUT high external validity vs experiments Constructing Survey Items Survey is only good as the questions it asks o Strive for reliable and valid measures Types of Questions o Open ended items o Closed ended items o Graphic measures o Measurement scales Open ended items Ex what is your favorite TV show Ex What rules if any exist in your home regarding your children s media use Advantages o Allows for in depth responses o Allows for unforeseen types of answers o Good in pilot studies Disadvantages o Difficult to code and analyze o Could misinterpret responses Closed ended Items Provide list of choices o Mutually exclusive and exhaustive o Likert type scales and semantic differentials etc Advantages o Much easier to analyze o Allow for powerful statistical analyses Disadvantages o May miss other possible responses o Participant may be influenced by response options Useful for research wit children the mentally disabled elderly or those o Graphic Measures under high stress Measurement Scales a series of close ended items that reflect some underlying concept perceived credibility compliance with doctors orders o could consider al items together as One concept o OR could consider it s multidimensional scale Survey Measures Should 1 Capture variation Use continuous variables where possible Avoid phrasing extreme questions that most will answer the same way o Do you watch TV o I am in favor or people driving while intoxicated everyone will answer the same way 2 Address the variable in Hs RQ s you pose 3 minimize order effects 4 minimize potential social desirability effects Social Desirability Bias People will DENY LIE in order to save face People tend to over report and under report certain behaviors and attitudes to look good o How s your sex life o How much did you donate to charity last year Solution o Ensure anonymous data storage o Limit face to face interviews o Provide face saving alternatives in the questions themselves More Issues With Survey Items EXAMPLE 1 ORIGINAL QUESTION o Do you go out often PROBLEM home EXAMPLE 2 o Vague bars restaurants the gym o Difficult to recall frequency IMPROVED QUESTION o In a typical week about how many meals do you eat away from o Do you support raising taxes to fund social security and increase ORIGINAL QUESTION military spending PROBLEM o Double barreled IMPROVED QUESTION S o Do you support raising taxes to fund SS o Do you support raising taxes to increase MS EXAMPLE 3 To examine relationship btw education smoking ORIGINAL QUESTION o Do you think more educated people smoke less PROBLEM o Assumes beliefs reflect reality IMPROVED QUESTION S o What is your education level o Do you smoke cigarettes EXAMPLE 4 ORIGINAL QUESTION election PROBLEM o Did you do your patriotic duty and vote in the last presidential o Leading question implies you should vote o Vulnerable to social desirability bias IMPROVED QUESTION S o Were you able to vote in the last presidential election EXAMPLE 5 ORIGINAL QUESTION o When did you stop drinking soda PROBLEM o False premise assume stopped IMPROVED QUESTION S Filters o Have you ever consumed soda o If yes do you currently drink soda o If no when did you stop drinking soda Experimental