1/18/10 1 FPP Chapter 19 Surveys Plan of Study  Conducting surveys is not trivial. We will cover these three main topics today. 1. Issues in questionnaire design 2. Methods for selecting units to survey 3. Administration of surveys1/18/10 2 Kind of a short (maybe long) tangent Statistical Literacy  Statistics are often used to validate arguments, philosophies, public policy decisions etc.  It behooves us to be educated consumers of statistics  Lying with statistics  “There are three kinds of lies: lies, damned lies and statistics”  “If you torture data enough it will tell you what you want it to.”  Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists by Joel Best  Statistics: Concepts and Controversies by David Moore1/18/10 3 Statistical Literacy  An example from Joel Best’s book  In dissertation prospectus a student reported the following that was found in a 1995 reputable peer-reviewed journal  “Every year since 1950 the number of American children gunned down has doubled”  Best says that “I think that it may be the worst – that is, the most inaccurate -- social statistic ever”  What’s wrong with it? Statistical Literacy  Understand the measure  Two reports in same newspaper (2/90)  Teens’ sexual activity on the rise.  Teen’s sexual activity on the decline  First article: avg. age of first intercourse (17.2 females, 16.5 males), which was younger  Second article: avg. number of partners (6) and frequency (3 per month), which were lower.  Two analyses really tackled different question but conclusions presented the same way  Always find out what variable is being measured when judging or making statistical conclusions.1/18/10 4 Statistical Literacy  Which U.S. states are the worst polluters?  E.P.A. in 1993 ranked New Jersey 22nd worst in the nation in release of toxic chemicals. E.P.A used total pounds (38.6 million).  When using pounds per square mile, New Jersey was the 4th worst in the nation.  The two analyses assess pollution, but they use different variables.  Pay attention to variable used when judging statistical claims Statistical Literacy  “Statistics are no substitute for judgments” Henry Clay1/18/10 5 Statistical Literacy  Beware Hidden Agendas  Survey paid for by disposable diaper company  “It is estimated that disposable diapers account for less than 2% of the trash in today’s landfills. In contrasts, beverage containers, third-class mail and yard wastes are estimate to account for about 21% of the trash in landfills. Given this, in your opinion would it be fair to ban disposable diapers?”  Question is prefaced with claims that only favor diapers which will lead respondents answers  When judging surveys get exact wording of questions Statistical Literacy  Beware hidden agendas 2  “Levi’s 501 Report, a fall fashion survey conducted annually on 100 campuses” sponsored by Levi’s  Report: 90% of college students chose Levi’s 501 jeans as “in” on campus  Levi’s 501 jeans was the only type of blue jeans on the list1/18/10 6 Statistical Literacy  Beware of hidden agendas 3  Advertisement for Triumph cigarettes:  “TRIUMPH BEATS MERIT – an amazing 60% said that Triumph tastes as good or better than Merid.”  Actual survey results  36% Triumph, 40% Merit, 24% no pref.  The company reported the survey results to make Triumph look as good as possible. Statistical Literacy  Economic questions  Did wages increase during the Reagan-Bush years (1980-1992)?  Average wages in private nonagricultural  Production: $235 in 1980, $345 in 1992  The equivalent 1980 wage adjusted for inflation was $388. People were worse off!  Consider inflation when judging statistical statements about money.1/18/10 7 Statistical Literacy  Consider raw numbers  “Planes get closer in midair as traffic control errors rise.”  18% increase in errors, 1997 to 1998.  Wow! Huge increase. Flying was much more dangerous in 1998 compared to 1997  Actual error rates:  5.5 errors per million 1998  4.8 errors per million 1997  7 more errors per million isn’t as bad Statistical Literacy  Use the proper base  80% of all accidents happen within 10 miles of home  Well duh! Most driving is within ten miles of home, so most accidents should occur within 10 miles of home.1/18/10 8 Statistical Literacy  Use proper base  12th grade students ranking on math performance in various regions. (I.A.E.E.A. rankings 1991)  Hungary ranked near the bottom of list.  Hong Kong ranked first  But …  50% of Hungary’s 12th graders took math  3% of Hong Kong’s 12th graders took math  It is likely that only the mathematically inclined students took math in Hong Kong while math education was more universal in Hungary  Average score in Hungary should be lower when including performance of weaker math students. Statistical Literacy  Good and bad graphs1/18/10 9 Statistical Literacy  What to do?  Ask yourself as many questions as possible  Try to think of all possible interpretations  Is it suspicious?  If it is suspicious then investigate  Make sure that it makes sense Statistical Literacy  Swine Flu1/18/10 10 General Idea Parameter Statistic Inference Sample Population Some new vocabulary  Population  Sample  Parameter  Statistic  Inference  Bias  Non-response bias  Response bias  Simple random sample  Convenience sampling  Frame coverage bias  Judgment sampling  Voluntary sampling  Probably others that I’ve missed1/18/10 11 Plan of Study  Conducting surveys is not trivial. We will cover these three main topics today. 1. Issues in questionnaire design 2. Methods for selecting units to survey 3. Administration of surveys Challenges to writing good questions 1. Defining objectives and specifying the kind of answers needed to meet objectives of the question 2. Ensuring all respondents have a shared, common understanding of the question 3. Ensuring people