Survey design&Taking measurementsLets try ...• Make up a question to ask anything you want to know about MIT students (do not use a yes / no question format)• Write down your question and its response scale• Examples ...• Turn to your neighbor and present your questions to each other– Why this question, why this response scale etc.How do people answer questions?3 cognitive stages in answering questions• "File drawer" view: – Attitudes are stable dispositions that are stored in long-term memory, and you just retrieve them• "The constructive" view:– Attitudes are constructed on the fly, computed when necessary from whatever information comes to mindinterpretquestionretrieve info&form judgmentmap judgmentonto response scaleDo people interpret the question as intended?•What brand of soft drink do you typically buy?–What is the time period involved? (How far back do I go … I bought Gatorade in the summer; Hi C when I was a kid)–Do purchases at movies, in restaurants, & sporting events count?– Does "you" refer to me or to my household?– What is a "soft drink" exactly?• do lemonade, iced tea, fruit punch, & mineral water count?– Are Coke and Cherry Coke the same brand? interpretquestionretrieve info&form judgmentmap judgmentonto response scaleCan people retrieve the relevant information?• How many ounces of alcohol did you consume last year?– What would be a better question to ask? interpretquestionretrieve info&form judgmentmap judgmentonto response scaleHow do people select responses?• How successful are you?– on a scale from 1 to 10?–•Is 10 the valedictorian of your high school? Ted Turner? Michael Jordan?interpretquestionretrieve info&form judgmentmap judgmentonto response scale3 cognitive stages in answering questionsTo design a good question you MUST be certain that people can1) Interpret your question as you want them to2) Access the relevant information3) Map their response on the scale in a meaningful and consistent (across and between people) wayinterpretquestionretrieve info&form judgmentmap judgmentonto response scaleEffects of scale formatsEffects of scale formatHow successful have you been in life ? Notat allVerymuch0 1 2 3 4 5 6 7 8 9 10Notat allVerymuch-5 -4 -3 -2 -1 0 1 2 3 4 5Response scale provide cues for the "normal" levels of a behavior _____ _____ _____ _____ _____ _____ up to .5 .5 to 1 1 to 1.5 1.5 to 2 2 to 2.5 more than 2.5 How many hours a day do you spend watching TV ? _____ _____ _____ _____ _____ _____ up to 2.5 2.5 to 3 3 to 3.5 3.5 to 4 4 to 4.5 more than 4.5 How many hours a day do you spend watching TV ? 7% 18% 26% 15% 18% 16%62% 23% 8% 5% 2% 0%Response scale provide cues for the "normal" levels of a behavior 0 1 2 3 4 5 6 7 8 9 10 >10How many times do you floss in a typical day ? 0 1 2 3 4 5 6 7 8 9 10 >10How many times do you floss in a typical year ? How likely are you to develop gum disease?How much would you pay for a bottle of Scope mouth wash? Other examples ….Effects of scales:•In many cases the scales we give people are not ones they are used to think about (even hours of watching TV)•In such cases people use the scales to determine the acceptable range and their subjective fit within the range to determine their responseThe importance of context1a. How bad is it to jaywalk?2. 0 1 2 3 4 5 6 7 8 9 10 not at very all bad bad 2. How bad is it to spray paint the side of a building?5. 0 1 2 3 4 5 6 7 8 9 10 not at very all bad bad– 1b. How bad is it to murder your professor?2. 0 1 2 3 4 5 6 7 8 9 10 not at very all bad bad 2. How bad is it to spray paint the side of a building?4. 0 1 2 3 4 5 6 7 8 9 10 not at very all bad bad–Ease of recall affects liking ICondition 1Condition 2List 10 reasons why you love XList 3 reasons why you love X1. __________2. __________3. __________4. __________5. __________6. __________7. __________8. __________9. __________10. _________1. __________2. __________3. __________Ease of recall affects liking IICondition 1Condition 2List 10 reasons to buy a BMWList 3 reasons to buy a BMW1. __________2. __________3. __________4. __________5. __________6. __________7. __________8. __________9. __________10. _________1. __________2. __________3. __________Effects of scales:• The constructive view of preferences:– People do not know but if you ask them they will give you an answer– And they will be sure about it– But …..• In some cases people”learn” about themselves from their own answersQuestion Order I1) How satisfied are you with your life overall? 2) How is the weather? How satisfied are you with your life overall?––In one of these conditions, was there a strong correlation between current weather (whether it was raining outside or sunny) and people's appraisal of their life as a whole.–In which of these two conditions was the correlation observed.Question Order II1a) How many dates did you have last month?1b) How satisfied are you with your life, in general?•2a) How satisfied are you with your life, in general?2b) How many dates did you have last month?Response language•Asking questions is a part of a discussion•People make assumptions about the intended meaning•Think about what the question suggests / assumes / implies in the context of a conversationSummary•The way we ask questions can have a large effect on how people understand the questions, interpret them, and ultimately answer them•Be careful!!!Extracting sensitive informationExtracting sensitive information I1) Do you happen to have done X in the last year?2a) As you know, many people have been doing X these days. Do you happen to have done X?2b) Do you know anyone who has done X? How about yourself?Extracting sensitive information IIPull a coin from your pocket. If it lands Heads, answer Question 1. If it lands Tails answer Question 2:1.Is your birthday between Jan 1st