TestingTesting: MorrowGraham Thurgood (English 232) 151 Testing Test types: Contrasting types of ESL testsKnowledge Performance (or Skills) testsSubjective ObjectiveProductive ReceptiveLanguage subskills Communication skillsNorm-referenced Criterion-referencedDiscrete-point tests IntegrativeProficiency Achievement Testing: Morrow Keith Morrow. 1979. Communicative language testing: revolution orevolution? The Communicative Approach to Language Teaching, Edited byC.J. Brumfit and K. Johnson. Oxford University Press. Pp. 143-157. I. Reliability One of the most significant features of psychometric tests as opposed to thoseof 'pre-scientific' days is the development of the twin concepts of reliabilityand validity.The basis of the reliability claimed by Lado is objectivity. The ratherobvious point has, however, not escaped observers (Pilliner, 1968; Robinson,1973) that Lado's tests are objective only in terms of actual assessment. Interms of the evaluation of the numerical score yielded, and perhaps moreimportantly, in terms of the construction of thetest itself, subjective factorsplay a large part.It has been equally noted by observers that an insistence on testingprocedures which can be objectively assessed has a number of implicationsfor the data yielded. Robinson (op. cit.) identifies three areas of differencebetween testing procedures designed to yield data which can be objectivelyassessed and those which are open to subjective assessment.1. The amount of language produced by the student. In an objective test,students may actually produce no language at all. Their role may beGraham Thurgood (English 232) 152 limited to selecting alternatives rather than producing language.2. Thus the type of ability which is being tested is crucially different. In asubjective test the candidate's ability to produce language is a crucialfactor; in an objective test the ability to recognize appropriate forms issufficient.3. The norms of language use are established on different grounds. In anobjective test the candidate must base his responses upon the languageof the examiner; in a subjective test, the norms may be his own,deriving from his own use of the language. Thus an objective test canreveal only differences and similarities between the language norms ofthe examiner and candidate; it can tell us nothing of the norms whichthe candidate himself would apply in a use situation.The above factors lead to what Davies (1978) has called the reliability-validity 'tension'. Attempts to increase the reliability of tests have led testdesigners to take an over-restrictive view of what it is that they are testing.Other terms of importance: norm referencing, criterion referencing,Graham Thurgood (English 232) 153 II. Validity The idea that language test designers should concern themselves withvalidity-in other words that they should ask themselves whether they [147]are actually testing what they think they are testing, and whether what theyare testing is what they ought to be testing-is clearly an attractive one. Butunfortunately, because of the 'tension' referred to above, designers workingwithin the tradition we are discussing seem than totally convincing.Five types of validity which a language test may claim are traditionallyidentified (cf. Davies, 1968).Face the test looks like a good one.Content the test accurately reflects the syllabus on which it is based.Predictive the test accurately predicts performance in some subsequentsituation.Concurrent the test gives similar results to existing tests which havealready been validated.Construct the test reflects accurately the principles of a valid theory offoreign language learning.Statistical techniques for assessing validity in these terms have beendeveloped to a high, and often esoteric level of sophistication. Butunfortunately, with two exceptions (face, and possibly predictive) the types ofvalidity outline above are all ultimately circular. Starting from a certain set ofassumptions about the nature of language and language learning will lead tolanguage tests which are perfectly valid in terms of these assumptions, butwhose value must inevitably be called into question if the basic assumptionsthemselves are challenged. Thus a test which perfectly satisfies criteria ofcontent, construct or concurrent validity may nonetheless fail to show in anyinteresting way how well a candidate can perform in or use the targetlanguage. This may occur quite simply if the construct of the languagelearning theory, and the content of the syllabus are themselves not related tothis aim, or if the test is validated against other language tests which do notconcern themselves with this objective. There is clearly no such thing intesting as 'absolute' validity. Validity exists only in terms of specified criteria,and if the criteria turn out to be the wrong ones, then validity claimed interms of them turns out to be spurious. Caveat emptor.Graham Thurgood (English 232) 154 Notes:The material discussed above was not presented for you to simplyaccept; in fact, if you simply accept it, then I would be somewhatdisappointed in you. In the discussion of testing, what we were discussing was the meanings of the TECHNICAL TERMS reliability and validity. We were NOT saying that if a test was reliable and valid in a technical sense that you yourself would necessarily think it was a reliable and valid test in a non-technical sense. Quite the opposite. I suspect that many tests that are RELIABLE and VALID do not really seem reliable and valid to you and me. So what we did in class was discuss what test makers mean when they use these terms. One of the main points was to see what the difference was between how they use these terms and what they mean by them and how you and I use these terms and what we mean by them. I should hope that you do not blindly accept the test makers uses of the word reliable and