Slide 1Doing the Observation ExerciseTips_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 2Methodological Concepts in Assignments• Observation– Operationalizing a variable– Inter-subjective reliability• Experiment– Isolating causal relations by controlling extraneous variables– Logic of randomization• Questionnaire– Operationalizing a variable with multiple indicators– Construct validity (relations among different measures)_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 3Examples of Things to Study• People in public places• TV commercials• TV shows• Radio shows or ads• Magazine ads• News articles (from newspapers or electronic archives)_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 4Preliminary Unstructured Observation• Partners can do this separately, does not have to be at exactly the same time or on the same subjects• But agree on the type of object you are observing or type of place you are observing people in• You may observe together if you wish, but write your observations and ideas for research separately before discussing them_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 5Sampling Choices• You must end up trying to observe the sameunits of analysis.• You must end up with at least 30 cases (units of analysis)• Two options:– #1 You can sample by a rule (in which case defining the rule is operationalizing the sample and counts toward making the project “non-obvious)– #2 Select sample documents separately and then trade for observation or agree whom to observe in a given time period._____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 6Option 1: Sampling by Rule• Don’t “cheat”, create a rule and then follow it independently. Examples:• Observe everyone who comes through the door• Watch all commercials on channel 7 between 7 and 8 pm• Use keyword search to find all articles about murder in May of 2002_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 7Option 2: Pick Subjects Directly• To save time when you cannot observe together, you may each choose a sample of 15+ documents (e.g. magazine ads), observe and code the variable for them, then trade samples.• In field observation, you may agree which subject to observe for a given time period.• The observations of the variables must be 100% independent – you must not peek at the others’ coding or discuss it in any way._____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 8Testing Reliability• You must observe the same subjects independently, without checking each other’s work• If you sampled by rule, calculate sampling error and locate those subjects you both observed• Calculate the percentage error (disagreement) in coding the dependent variable_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 9Calculating Reliability -1N = total number of distinct people seen (by both or either) S1 = sample difference 1: number of times partner 1 observed a subject partner 2 did not see.S2 = sample difference 2: number of times partner 2 observed a subject partner 1 did not see. A = number you agree on: same subject, same variable code.C = coding difference: number of times you observed the same subject but coded the variable differently._____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 10Calculating Reliability - 2• Coding Error: CE = C/(A+C) (proportion you both saw that you disagree about in the variable).• Sample selection error: SE = (S1 + S2)/N (proportion of total cases that one person saw but not the other)._____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 11Evaluating Reliability• Perfect reliability is the goal, zero errors• But for this assignment, need to do a variable that is complex enough that this is not “easy”• Even 10% error is fairly high for reliability• Try to understand the source of all errors and how they could be avoided• If error is low, discuss what you did well in the procedures to produce low error_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Slide 12Conditional Percentages1) Dependent variable is qualitative2) Cross-tabulate the data3) Calculate percentages for the dependent variable separately within each category of the independent variable.4) Compare the percentages across categories of the independent
View Full Document
Unlocking...