Survey Design and Weighting material drawn from Korn Graubard 1999 A Some basic design considerations and their implications 1 Many considerations when conducting a survey a want the final results to be representative of an analysis population b may want the survey to include specific subpopulations i racial ethnic diversity ii economic diversity iii age diversity iv programmatic diversity c want to keep costs down by surveying many people in a limited number of areas d may have other design issues e g AddHealth samples within schools and also samples peer networks e possible differential unit non response 2 These design issues affect subsequent statistical analyses a affect the representativeness of the observations b also affect the independence of the observations 3 Consider some general issues a clustering selecting subjects from a few areas may lead to spatial correlation in the data i although this might not lead to biased estimates of population parameters it could affect calculations of standard errors ii essentially there is less variation than a sample taken completely at random 1 b oversampling of particular populations which is done to ensure adequate representation of those individual populations leads to a sample that doesn t represent the general population c different rates of unit non response can also lead to a loss in representativeness 4 In analyses we address these issues by a including survey weights b including design variables B Sampling plans 1 Simple random sampling a consider a given population of size N b choose a subset of n individuals where each possible subset is equally likely to be sampled c individuals are chosen without replacement we won t refer to this distinction subsequently d the ratio n N is called the sampling rate or inclusion probability e most sample estimators such as the usual mean and variance estimators assume this type of sampling 2 Stratified simple random sampling a population is first divided into mutually exclusive and exhaustive strata b simple random sampling is then carried out within each strata c sampling rates may will likely vary across strata i for example if the populations of the strata differ but the sample sizes don t the sampling rates will 2 vary ii sampling rates may vary for other reasons d subsequent population estimators would weight each observation by the inverse of its sampling rate e if the observations are more homogeneous within strata than across them stratified random sampling can reduce the variance of the population estimates 3 multi stage sampling a population is first divided into cells or primary sampling units PSUs usually on the basis of geography b a sample of those units is taken c sampling of individuals then takes place within those selected PSUs d advantages i reduces the costs of conducting a survey by restricting it to a smaller number of areas very important if in person interviews are used ii also may be the only feasible way to construct a sampling frame e disadvantage is that observations within clusters may be correlated f weighting is again used to adjust estimates weights are the product of i inverse of the PSU inclusion probability and ii sampling rate within the PSU g here we ve described a two stage process but the process can be carried out iteratively 4 Unit non response a this is a distinct issue from the sampling plan 3 however the correction would be similar b within PSUs and clusters we may will have unit non response c estimated response probabilities can be formed possibly conditioning on additional information d the inverse of these probabilities can then be multiplied times the other components of the weight calculation to form a new weight 5 In all of the cases except simple random sampling we have unequal sampling rates for individuals within the overall population but can use weights to obtain unbiased estimates of the population parameters C Poststratification 1 Another strategy to improve the accuracy and representativeness of a sample is to poststratify the sample or the sampling weights 2 In poststratification weights are developed so that the totals or proportions of different types of respondents match known population figures 3 The advantage of this technique is that it brings in additional information about the population a it can be effective in dealing with differential nonresponse and under sampling b it also helps to make surveys more comparable by reweighting them to a common analysis population D Use of weights in statistical analyses 1 If observations are sampled unequally respond 4 unequally or can be adjusted to reflect the larger population it makes sense to use weights to reduce biases 2 Weights are usually the product of a inverse sampling probabilities sometimes referred to as the base weight note these sampling probabilities can themselves be products of probabilities if multistage sampling is used b inverse response probabilities sometimes referred to as non response adjustments and c poststratification adjustments 3 For most cross sectional statistics calculations for weighted estimates are straightforward a let Xi be a variable for subject i and let Wi be the associated weight b the weighted mean is 1 Xw Wi X i W i c let xi and yi be deviations of variables Xi and Yi from their weighted means the slope coefficient from a weighted regression of Yi on Xi is Wx y w i i 2 i Wi xi 4 In SAS most statistical procedures include a WEIGHT statement a syntax WEIGHT weight variable 5 b the weight variable would be the SAS variable containing the weights 5 If the sampling design indicates that weights should be included you should include them right E An alternative to weighting 1 An alternative to weighting is to model the survey design in your statistical procedure 2 In a multivariate model this is accomplished by including measures of the characteristics that enter the weighting procedure as additional explanatory variables in an unweighted model a coefficients on these variables will confound genuine effects and survey design effects b however coefficients on the other variables should be purged of the influence of design effects c this assumes that you have modeled the design correctly d it also assumes that the design variables don t introduce other problems 3 Weights are often based on characteristics that we would include in models anyway such as age race ethnicity age and socioeconomic status 4 Suggests that weights might not be especially