Comparing ACS with Census Sample and Current Survey Data Partnership and Data Services Training Susan Love April 12 2005 Some Sources for Comparison Census long form samples in general and Census 2000 in particular Current Population Survey and Annual Social and Economic Supplement American Housing Survey What should we compare Sampling frames and sample design Data collection time frames and modes Interviewer requirements Respondent requirements Data completeness Weighting and estimation methods Final results at comparable geographies ACS and the Decennial Census How do the samples compare Similarities Systematic samples of MAF addresses Differential sampling rates for areas based on population size Sampling Rates Governmental Unit Size Number of Occupied Housing Units 0 200 201 800 801 1200 1201 2000 Over 2000 Census 2000 Sampling Rates 50 0 50 0 25 0 16 7 12 5 ACS 5 year Sampling Rates 50 0 35 0 17 5 12 0 8 5 How do the samples compare Differences ACS samples every year and spreads sample over 12 months census samples once a decade and uses the entire sample at the same time ACS subsamples for personal visit followup court ruled against census subsampling Census sample estimates based on about 18 million housing units ACS 5 year estimates based on about 11 million housing units How does data collection compare Similarities Primary method is self response to a paper questionnaire Content is the same How does data collection compare Differences ACS nonresponse FU uses CATI and CAPI instruments past censuses have used only paper questionnaires ACS data collected only from household members census data often collected from neighbors ACS interviews conducted by experienced and wellsupervised staff census enumerations conducted by inexperienced temporary workforce What is the impact on the data ACS estimates have higher sampling error measures are released for each estimate and shown as 90 confidence limits or margins of error in every table Similar sampling error measures for census long form sample estimates have not been provided What is the impact on the data ACS estimates have lower potential nonsampling error Unit nonresponse no data for a case Item nonresponse data missing for an item These and other measures are released on ACS s Quality Measures website www census gov acs www UseData sse index htm Unit Nonresponse Response Rates Census 2000 Sample ACS 2000 91 2 95 1 2001 96 7 2002 2003 97 7 96 7 100 unit nr rate 11 Item Nonresponse Summary Allocation Rates Census 2000 Sample 2000 ACS 10 4 6 6 Population items Mail 8 7 8 1 Population items Followup 12 9 4 5 Housing items Mail 11 5 5 9 Housing items Followup 14 9 7 1 amount of data missing from interviews All modes items 12 What do the estimates represent Most ACS estimates are said to be the average characteristics of an area over a calendar year Census sample estimates are often said to be characteristics of an area as of Census Day Is this so What do the estimates represent Both the ACS interview date and the census enumeration date play important roles ACS uses the interview date as the single reference point or as the end of a reference period for all data collection The Census 2000 sample did too except for income migration and agricultural sales Weighted Census 2000 Sample and 2000 ACS Housing Units by Response Week 25 000 000 20 000 000 15 000 000 10 000 000 5 000 000 Interview Week in Calendar Year 2000 Census 2000 Sample 2000 ACS 52 49 46 43 40 37 34 31 28 25 22 19 16 13 10 7 4 0 1 Weighted Housing Units 30 000 000 What do the Census 2000 sample estimates represent Census 2000 data collection was a sequential process of 3 operations over a 6 month period Data from these operations were processed together weighted and tabulated The characteristic estimates are the sum of the peaks and valleys of the enumerations not Census Day Census 2000 Data Collection Design M o n t h o f C e n s u s 2 0 0 0 E n u m e r a tio n M a rc h M a il A p r il M ay NRFU June J u ly C IF U August What do the ACS estimates represent ACS data collection is a sequential process of 3 operations on each of 12 monthly samples over 3 month periods All 3 operations take place every month on different sample panels a continuous series of interviews Data collected in a calendar year are grouped by interview month a weighting adjustment is applied to smooth overall monthly differences Summed estimates are considered the average characteristics of all areas for the calendar year ACS Data Collection Design M o n th o f In te rv ie w ACS S a m p le P a n e l N ovem ber D ecem ber J a n u a ry F e b ru a ry M a rc h A p ril M ay June 1999 2000 N o vem ber D ecem ber M a il J a n u a ry F e b ru a ry M a rc h A p ril M ay June CATI CAPI M a il CATI CAPI M a il CATI CAPI M a il CATI CAPI M a il CATI CAPI M a il CATI CAPI M a il CATI M a il How did the 2000 ACS and Census 2000 results compare Comparisons with Census 2000 Four reports by Bureau staff compared the 2000 ACS data with Census 2000 data at the national level Two staff reports and four reports by outside researchers compared three year averages from the 1999 2001 ACS test sites with Census 2000 at county and tract levels www census gov acs www advMeth Reports Comparisons with Census 2000 ACS profile distributions were compared to Census 2000 profile distributions without group quarters pop ACS distributions were surprisingly similar to Census 2000 Only about 9 of the profile table estimates differed by more than 1 percentage point Self response data from mail returns were responsible for the consistent results Comparisons with Census 2000 Characteristics differing the most Race and relationship Table 1 Ancestry disability high school graduates Table 2 Labor force participation median household and family income and poverty rates Table 3 Year built rooms and house heating fuel Table 4 Summary of National Level Comparisons of General Demographic and Housing Characteristics Table 1 Percentage Point Diference C2SS Census 2000 1 0 or greater 0 5 to 0 9 0 1 to 0 4 0 0 0 1 to 0 4 0 5 to 0 9 1 0 or greater 0 5 10 15 20 25 30 Percent of National Level Comparisons n 94 35 General Demographic Characteristics ACS estimated more White and fewer Some Other Race experience and training of FRs ACS estimated fewer married couple families ACS weighting edit difference marital status no longer 100 census item Item nonresponse Race Census 3 9 ACS 2 4 Relationship Census 2 2 ACS relationship 1 6 Summary of National