SurveysPlan of StudyStatistical LiteracySlide 4Statistical LiteracySlide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16General IdeaSome new vocabularySlide 19Challenges to writing good questionsChallenge to writing a good question #1Challenge to writing a good question #2Challenge to writing a good question #3Challenge to writing a good question #4Challenge to writing a good question #5Steps to running a surveyThink about itMisspecifying target populationSurveys that use census as sampling frameSelecting samplesObtaining survey dataUnreliable methods of selecting samplesAdditional potential pitfallsThink about itExample of voluntary response surveyExamples of problematic survey designsHite’s surveyHite’s surveySlide 39Random sampling comment 1Random sampling comment 2Random sampling comment 3Random sampling comment 4Up ShotFPP Chapter 19SurveysPlan of Study Conducting good surveys is not trivial. We will cover these three main topics today.1. Issues in questionnaire design1. Methods for selecting units to survey1. Administration of surveysStatistical LiteracyStatistics are often used to validate arguments, philosophies, public policy decisions etc. It behooves us to be educated consumers of statisticsLying with statistics“There are three kinds of lies: lies, damned lies and statistics”“If you torture data enough it will tell you what you want it to.”Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists by Joel BestStatistics: Concepts and Controversies by David MooreStatistical LiteracyAn example from Joel Best’s bookIn dissertation prospectus a student reported the following that was found in a 1995 reputable peer-reviewed journal“Every year since 1950 the number of American children gunned down has doubled”Best says that “I think that it may be the worst – that is, the most inaccurate -- social statistic ever”What’s wrong with it?Statistical LiteracyUnderstand the measureTwo reports in same newspaper (2/90)Teens’ sexual activity on the rise.Teen’s sexual activity on the declineFirst article: avg. age of first intercourse (17.2 females, 16.5 males), which was youngerSecond article: avg. number of partners (6) and frequency (3 per month), which were lower.Two analyses really tackled different question but conclusions presented the same wayAlways find out what variable is being measured when judging or making statistical conclusions.Statistical LiteracyWhich U.S. states are the worst polluters?E.P.A. in 1993 ranked New Jersey 22nd worst in the nation in release of toxic chemicals. E.P.A used total pounds (38.6 million).When using pounds per square mile, New Jersey was the 4th worst in the nation. The two analyses assess pollution, but they use different variables.Pay attention to variable used when judging statistical claimsStatistical Literacy“Statistics are no substitute for judgments” Henry ClayStatistical LiteracyBeware Hidden AgendasSurvey paid for by disposable diaper company“It is estimated that disposable diapers account for less than 2% of the trash in today’s landfills. In contrasts, beverage containers, third-class mail and yard wastes are estimate to account for about 21% of the trash in landfills. Given this, in your opinion would it be fair to ban disposable diapers?”Question is prefaced with claims that only favor diapers which will lead respondents answersWhen judging surveys get exact wording of questionsStatistical LiteracyBeware hidden agendas 2“Levi’s 501 Report, a fall fashion survey conducted annually on 100 campuses” sponsored by Levi’sReport: 90% of college students chose Levi’s 501 jeans as “in” on campusLevi’s 501 jeans was the only type of blue jeans on the listStatistical LiteracyBeware of hidden agendas 3Advertisement for Triumph cigarettes:“TRIUMPH BEATS MERIT – an amazing 60% said that Triumph tastes as good or better than Merid.”Actual survey results36% Triumph, 40% Merit, 24% no pref.The company reported the survey results to make Triumph look as good as possible.Statistical LiteracyEconomic questionsDid wages increase during the Reagan-Bush years (1980-1992)?Average wages in private nonagriculturalProduction: $235 in 1980, $345 in 1992The equivalent 1980 wage adjusted for inflation was $388. People were worse off!Consider inflation when judging statistical statements about money.Statistical LiteracyConsider raw numbers“Planes get closer in midair as traffic control errors rise.”18% increase in errors, 1997 to 1998.Wow! Huge increase. Flying was much more dangerous in 1998 compared to 1997Actual error rates:5.5 errors per million 19984.8 errors per million 19977 more errors per million isn’t as badStatistical LiteracyUse the proper base80% of all accidents happen within 10 miles of homeWell duh! Most driving is within ten miles of home, so most accidents should occur within 10 miles of home.Statistical LiteracyUse proper base12th grade students ranking on math performance in various regions. (I.A.E.E.A. rankings 1991)Hong Kong ranked firstHungary ranked near the bottom of list.But …50% of Hungary’s 12th graders took math3% of Hong Kong’s 12th graders took mathIt is likely that only the mathematically inclined students took math in Hong Kong while math education was more universal in HungaryAverage score in Hungary should be lower when including performance of weaker math students.Statistical LiteracyGood and bad graphsStatistical LiteracyWhat to do?Ask yourself as many questions as possibleTry to think of all possible interpretationsIs it suspicious?If it is suspicious then investigateMake sure that it makes senseGeneral Idea Parameter StatisticInferenceSamplePopulationSome new vocabulary PopulationSampleParameterStatisticInferenceBiasSelection biasNon-response biasFrame coverage biasSimple random sampleQuota samplingConvenience samplingJudgment samplingVoluntary samplingProbably others that I’ve missedPlan of Study Conducting surveys is not trivial. We will cover these three main topics today.1. Issues in questionnaire design1. Methods for selecting units to survey1. Administration of surveysChallenges to writing good
View Full Document