Unformatted text preview:

Reconsidering the Measurement of Political Knowlege Matt Levendusky 1 1 April 04 A basic feature of representative democracy is that citizens use elections as a means of controlling elected officials A critical maintained hypothesis implicit in that idea is that citizens know enough about politics and public affairs to hold politicians accountable for their behavior in office If citizens cannot pass even this basic test then representative government seems an unsure proposition In short understanding how elections function in a democracy rests upon being able to evaluate what citizens know about politics and government In order to answer such questions empirically we need a high quality measure of citizen s political knowledge However despite its importance our measures of political sophistication 2 are not as accurate as one would like Traditionally most scholars choose one of two routes either relying on a single item most typically the interviewer s subjective assessment of the respondent s level of political sophistication Bartels 1996 or alternatively constructing a knowledge scale built from several factual items Zaller 1992 Mondak 1999 Mondak 2001 Yet both methods have associated drawbacks First if we use a single indicator we never get any idea of the uncertainty with which we measure a given concept If there is considerable error in our proxy variable then we risk biasing our inferences of interest the classic errors in variables result If we use a multi item scale we don t know that every item should be given the same weight some items might do a better job of tapping into the underlying quantity of interest So the question still remains how can we more accurately measure political sophistication Here I conceptualize political sophistication think of it as a continuous but unobservable variable Even though we do not observe political sophistication directly we do observe various indicators of it and we can use these indicators to measure this underlying unobservable quantity of political sophistication Here the observable indicators are factual information items from the 1 This is part of a larger joint project with Simon Jackman Any errors contained here are my own Essentially political sophistication is how well you conceptualize and understand politics do you know the rules of the game the way various pieces of an ideology coherently fit together etc 2 1 quadrennial National Election Study NES from 1980 2000 Using these indicators we can develop a model of political sophistication What does this sort of model look like To actually model political sophistication I estimate a two parameter item response IRT model The model takes the following form P yij 1 i F aj i bj Where j indexes items and i indexes respondents Here we ve modeled the probability that individual i correctly answers item j as a function of two model parameters aj and bj and a latent trait i So here i is the individual specific unobserved trait and the ys are the observable items the factual information items drawn from the NES surveys Since the items used here are binary the F in the above distribution is the logistic CDF In the above model aj the slope is termed the item discrimination parameter and bj the intercept the item difficulty parameter The discrimination parameter measures how well the item distinguishes between individuals possessing differing amounts of the latent trait An item with high discrimination distinguishes respondents with low amounts of the latent trait from those with middling and those with high levels The difficulty parameter on the other hand measure the difficulty of a given item The interpretation of this parameter is straightforward a more difficult item is harder for individuals at all levels of political information than an easier item and it is easier for people with higher levels of the latent trait to answer it than those with lower levels Johnson Albert 1999 184 5 Together these two parameters describe how well an item measures the underlying latent trait However items that can be scored correct or incorrect are not the only sources of information about political sophistication in the NES The NES also asks the person conducting each interview to rate the respondents level of political knowledge on a 5point scale from Very High to Very Low Indeed this is perhaps the most common 2 single indicator item used to measure political sophistication In all of the studies using this item there is an untested assumption that every interviewer uses the scale in exactly the same way That is that for interviewer A a Very High is the same as a Very High from interviewer B To test this hypothesis I include a random effect term for p every interviewer That is we can write out the ordinal logit link as i i p where p is the random effects term given to interviewer p Let Zi be the ranking given by judge p to individual i Then p P r Zi Very Low F 1t i p p P r Zi Fairly Low F 2t i F 1t i p P r Zi Very High 1 F 4t i Here p indexes interviewers i indexes individuals t indexes surveys and F is the logistic CDF Here I ve restricted the discrimination parameter the parameter on i to be 1 This is done to achieve identification of this part of the model by ensuring that respondents who receive a higher interviewer rating receive higher values of the latent trait ceteris paribus thereby fixing the direction of the latent scale Also it gives us overtime comparability That is this model restriction allows us to compare respondents across time since we impose this restriction in all years Otherwise we couldn t be sure that scores from 1980 respondents were directly comparable to those from 1996 The interviewer effects terms allow us to see if each judge uses the scale in the same manner The model estimates a different set of thresholds per year the t terms and assumes that each judge uses these thresholds but then allows for each judge to shift these cutpoints by differing amounts the p terms That is I allow for the possibility that each judge may have a higher lower threshold for a given category than his her peers This allows me to test explicitly the idea that all judges are using the scale the 3 same way Estimation of this sort of a model would be taxing in a traditional framework Since we re estimating a parameter for every respondent plus parameters on the items and judges we re estimating well over 12 000 total parameters Trying to find the global maximum of a 12 000 dimension hyper


View Full Document

Stanford POLISCI 353 - Study Notes

Download Study Notes
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Study Notes and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Study Notes 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?