CSUN PSY 427 - Test Administration (24 pages)

Previewing pages 1, 2, 23, 24 of 24 page document View the full content.
View Full Document

Test Administration



Previewing pages 1, 2, 23, 24 of actual document.

View the full content.
View Full Document
View Full Document

Test Administration

85 views


Pages:
24
School:
California State University, Northridge
Course:
Psy 427 - Intro Psy Testing
Unformatted text preview:

Psy 427 Cal State Northridge Andrew Ainsworth PhD Psy 427 Cal State Northridge When we talk about reliability we are interested in random sources of error Observed Score True Score Error When tests are actually administered however there are other sources of error aside from random error 2 Psy 427 Cal State Northridge Rapport Ethnicity Language Training of Test Administrators Expectancy Effects Use of Reinforcers Computer Assisted Testing Subject Variables 3 Psy 427 Cal State Northridge Importance of establishing rapport Feldman Sullivan 1960 WISC Enhanced rapport vs neutral rapport younger children through grade 3 did not benefit from enhanced rapport Older children grades 5 9 produced higher IQ scores under enhanced rapport Enhanced Rapport mean IQ of 122 Neutral Rapport mean IQ of 109 4 Psy 427 Cal State Northridge Children score lower on IQ test when the administrator made disapproving comments I thought you could do better then when administrators made neutral or positive comments Witmer Bernstein and Dunham 1971 Children unfamiliar with the administrator did significantly worse on a reading test compared to children familiar with the administrator DeRosa and Patalano 1991 5 Psy 427 Cal State Northridge Importance of establishing rapport Fuchs Fuchs 1986 meta analysis 22 studies involving 1489 children 4 IQ point increase when the examiner was familiar with the test taker in general 7 6 IQ point increase when familiarity and lower SES co occurred 6 Psy 427 Cal State Northridge Importance of establishing rapport Self report vs interview of attitudinal surveys People disclose MORE on self report than they do to interviewers People disclose MORE to computers than they do to human interviewers Conclusions Rapport is important in situations that are not viewed as personal or those typically subject to social desirability 7 Psy 427 Cal State Northridge Should children of one ethnicity be tested only by test administrators of the same ethnicity Majority of studies have found nonsignificant effects for cross ethnic administration of most intelligence tests The only significant findings have been when paraprofessionals have administered the tests Why no differences Standardized procedures 8 Psy 427 Cal State Northridge How valid are tests given in English to bilingual or Limited English Proficient LEP individuals What about translating tests Language Standard of practice administer a test in the most proficient language BUT what about the normative sample How comparable are the scores from these individuals Can IRT help Interpreters another potential source of bias 9 Psy 427 Cal State Northridge Administration and scoring errors are a large source of bias Typical graduate training 2 4 administrations of a test in class importance of fieldwork placements majority of testing practice obtained in fieldwork placements Error rates on WAIS administrations decrease after 10 administrations 10 Psy 427 Cal State Northridge Also known as Rosenthal effects Robert Rosenthal Harvard University Subjects perform in a manner consistent with experimenter s test administrator s expectations works with humans works with rats Effects not limited to experiments also occurs on standardized tests students asked to score ambiguous responses will give more points to people they like or think are bright People find what they expect 11 Psy 427 Cal State Northridge Expectancy and test administration Rosenthal expectancy effects are triggered by non verbal cues and the experimenter administrator may not even be aware Expectancy effects have small and varied influence on test outcomes careful study is required 12 Psy 427 Cal State Northridge No clear and consistent difference between studies using reinforcement showing positive or negative effects Individual studies however are compelling Terrell Taylor Terrell 1978 found a 17 6 point increase in IQ scores when AfricanAmerican children were given culturally appropriate feedback by African American test administrators 13 Psy 427 Cal State Northridge General guidelines Check with the testing manual first Generally OK to reward EFFORT not answers 14 Psy 427 Cal State Northridge Advantages The obvious connection to Item Response Theory and the ability to tailor tests to a persons ability Highly Standardized Precision of Timing Lessened Dependence on Human Testers Pacing no need to rush respondents Control of Bias from the test administrator etc 15 Psy 427 Cal State Northridge Computer adaptive versions of tests have shown no large differences between computer assisted and paperand pencil versions Computer versions can be more accurate and take less time e g IRT and CAT Some people enjoy the computer format and even prefer it 16 Psy 427 Cal State Northridge One study Locke and Gilbert 1995 showed that when respondents are asked about sensitive material e g MMPI drinking etc they were more honest when the tests were administered via computer vs questionnaire or interview CAT has been applied to the MMPI personnel selection and cognitive tests among others 17 Psy 427 Cal State Northridge The big concern with computer aided testing is that it will lead to the computer generated reports landing in the wrong inexperienced hands and misinterpreted 18 Psy 427 Cal State Northridge The state of the subject can also be a source of error when administering a test Illness Insomnia Test anxiety Drugs prescription and recreational Hormones e g menstruation variations in perceptual motor coordination varied with cycle better away from menses effects reverse for other tasks 19 Psy 427 Cal State Northridge Issues that arise when people judges act as the testing instrument Human judges are subject to problems that add to the error when assessing respondents Reactivity Drift Expectancies same as with test administration Deception 20 Psy 427 Cal State Northridge The reliability of behavioral assessments is usually assessed using inter rater reliability consistency among raters or by having supervisor make periodic checks Reliability tends to be highest when the judges know they are being evaluated either by supervisor or against one another This increase is called REACTIVITY 21 Psy 427 Cal State Northridge One study Reid 1970 showed that the reliability of observers rating decreased 25 when they were told that they would not be compared to a standard Many studies report inter rater reliability but care should be exercised because these numbers typically are calculated during training


View Full Document

Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...
Login

Join to view Test Administration and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Test Administration and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?