Total = 20 slides1Sociology 201:Social Research Design24. Evaluation Research ITotal = 20 slides2Preview• Workbook assignments due: 12.1(15 pts), 12.3(10 pts), •The Logic of evaluation research•Examples•Group discussionTotal = 20 slides3Logic of evaluation research•We perceive a problem•We create an intervention to solve it•We should follow up with research to determine whether we were successfulTotal = 20 slides4Much social intervention•Breakup of AT&T•Lowered voting age•Raised drinking age•Tax reform•Seat belt laws•Lowered speed limit to 55•Child-proof caps•Revising Sociology curriculum at ChapmanTotal = 20 slides5Goals and purposes•What exactly are we trying to achieve?–This is often unclear•Were we successful?–This is often not testedTotal = 20 slides6Example: NHTSA study of driver training• Self-selection is at work– "Good" students less likely to have accidents and violations– "Good" students also more likely to take driver training– Controlling for kind of kid cancels out the apparent effect• Program may actually cause more accidents – Encourages students to get licenses earlierOverall, driver training students have fewer accidentsHowever. . .Total = 20 slides7Operationalizing success/failure•Let’s create an example together•What’s the problem?•Create an intervention•Specify success/failureTotal = 20 slides8Cost of Living Index Research• Problem: some places more expensive than others• Want comparable pay for federal employees• Measure the price of things• Illustrates the importance of definitions and careful field work• I was asked to evaluate the Civil Service Commission’s measurement of cost of living on Maui• I was evaluating an evaluation • Keep this in mind when you hear cost of living statisticsTotal = 20 slides9Cost of Living Logic•Various consumer items•Weighted by importance•DC = 100, Maui = 108, Honolulu = 118.3 (1976)•Yet commodities had to be shipped from Honolulu to Maui•How could things be cheaper on Maui?Total = 20 slides10The Explanation lay in Research ErrorsTotal = 20 slides11Sewing Machines• Prices: DC = $298.17, Maui = $157.73• Early studies: Sears (Maui) vs Singer (DC)• 1978 & 1979: Maui priced Singers, too• Top = $979.95 (1978) $1035 (1979)• Got worksheets from Washington• They added top-of-line for $950 (v 1035)•However, they divided seven prices by sixTotal = 20 slides12Bicycles•Instructions called for 3-speed bikes•These were hard to find anywhere•Washington priced almost all 10-speed bikes•Maui tracked down a 3-speedTotal = 20 slides13Phonograph records•1976: DC=5.06, Maui=5.06, Hon=6.11•Worksheets showed the researcher priced phonographs, not records ($100+)•Commission simply substituted DC price•Should have used the Honolulu priceTotal = 20 slides14Books•Instructions: price 150-200 page books•DC=$1.47, Maui=$1.24•Maui could only find children’s book that length–Averaged 186 pages•DC priced best-sellers, averaging 373 pagesTotal = 20 slides15Restaurants•DC=$9.29, Maui=3.34 for dinner•DC: mainline hotel and popular roast-beef place•Maui: Chinese restaurantTotal = 20 slides16Rice•DC=.42 cents/lb, Maui=.24•“Price 1 pound container”•Maui priced 25-lb and divided•Called the large “economy” sizeTotal = 20 slides17Same problem with other commodities• 3.5-ounce tube of toothpaste– DC=67, Maui=.55– DC priced 3, 3.5, and 7 oz tubes– Maui priced 7 and 10 oz tubes– 7-oz tube in Maui=1.09, DC=.59• Maui priced 20-exposure rather than 36-exposure film• Newspapers: local and out-of-town– Honolulu: Advertiser = local–Maui: Advertiser = out-of-townTotal = 20 slides18Haircuts•Maui didn’t include tip because he didn’t think it was appropriateTotal = 20 slides19Comment on Expert TestimonyTotal = 20 slides20Next Time• Continue Evaluation