Statistics and Measurement (Understandingand Quantifying Measurement Uncertainty)The general question to be addressed here is "How do statistical methodsinform measurement/metrology?" Some answers will be phrased in terms ofmethods for separating measurement variation from process variation (includingappropriate confidence intervals), methods for "Gauge R&R" (again includingappropriate confidence intervals), and the use of simple linear regression predic-tion limits to assess measurement uncertainty on the basis of a linear calibrationstudy.1Basic Issues in Metrology• Validity (am I really tracking what I want to track?)• Precision (consistency of measurement)• Accuracy (getting the "right" answer on average)2Not AccurateNot PreciseAccurateNot PreciseNot AccuratePreciseAccuratePreciseFigure 1: Measurement/Target Shooting Analogy3A Simple Measurement Model and Basic Statisti-cal MethodsA Basic Statistical/Probabilistic Model for Measurement: What is measured,y, is the measurand, x, plus a normal random measurement error, ,withmeanβ and standard deviation σmeasurement.y = x + Pictorially:4Figure 2: A Basic Statistical Measurement Model5Notice that under this model, based on m repeat measurements of a sin-gle measurand, y1,y2,...,ym,withsamplemeany and sample standarddeviation s• if I apply the t confidence interval for a mean, I get an inference forx + β = measurand plus biasthat is,— in the event that the measurement device is known to be well-calibrated(one is sure that β =0, there is no systematic error), the limitsy ±ts/√m based on ν = m − 1 df are limits for x— in the event that what is being measured is a standard for which x isknown, one may use the limits(y − x) ± ts√m6to estimate the device bias, β• if I apply the χ2confidence interval for a standard deviation, I get aninferenceforthesizeofthemeasurement"noise,"σmeasurementFor measurements on multiple measurands (e.g. on different batches or partsproduced by a production process), we extend the basic measurement model byassuming that x varies/is random. (Variation in x is "real" process variation.)In fact, if we assume that the measurand is itself normal with mean μxandstandard deviation σxand independent of the measurement error, we then havey = x + with meanμy= μx+ β7and standard deviationσy=qσ2x+ σ2measurement>σx(so observed variation in y is larger than the actual process variation becauseof measurement noise).Under this model for single measurements made on n different measurandsy1,y2,...,ynwith sample mean y and sample standard deviation sy, the limitsy ± tsy/√n (for t based on n−1 degrees of freedom) are limits for μx+β,themean of the distribution of true values plus bias. Note also that the quantitysyestimates σy, that really isn’t of fundamental interest. But sinceσx=r³σ2x+ σ2measurement´− σ2measurementan estimate of specimen-to-specimen variation (free of measurement noise)based on a sample of m observations on a single unit and a sample of n8observations each on different units is (see display (2.3), page 20 of SQAME )cσx=rmax³0,s2y− s2´Example Below are m =5measurements made by a single analyst on a singlesample of material. (You may think of these as measured concentrations ofsome constituent.)1.0025,.9820, 1.0105, 1.0110,.9960These have meany =1.0004 and s = .0120. Consulting a χ2table usingν =5− 1=4df, we can find a 95% confidence interval for σmeasurement⎛⎝.0120s411.143,.0120s4.484⎞⎠i.e. (.0072,.0345)9(One moral here is that ordinary small sample sizes give very wide confidencelimits for a standard deviation.) Consulting a t table also using 4 df, we canfind 95% confidence limits for the true value for the specimen plus instrumentbias (x + β)1.0004 ± 2.776.0120√4i.e. 1.0004 ± .0167Suppose that subsequently, samples from n =20different batches are ana-lyzed. The t confidence interval.9954 ± 2.093.0300√20i.e. .9954 ± .0140is for μx+β,theprocessmeanplus any measurement instrument bias/systematicerror. An estimate of real process standard deviation iscσx=rmax³0,s2y− s2´=rmax³0, (.0300)2− (.0120)2´= .027510andthisvaluecanusedtomakeconfidence limits. To do so, we need aSatterthwaite "approximate degrees of freedom"ˆν =cσx4s4yn − 1+s4m − 1=(.0275)4(.0300)419+(.0120)44=11.96rounding down to ˆν =11, an approximate 95% confidence interval for the realprocess standard deviation, σx,is⎛⎝.0275s1121.920,.0275s113.816⎞⎠i.e. (.0195,.0467)11Gauge R&R Studies and Partitioning MeasurementVariation Where Multiple Analysts Make Measure-mentsThere can be "operator/analyst variability" that should be considered part ofmeasurement imprecision.• "Repeatability" variation is variation characteristic of one operator/analystremeasuring one specimen• "Reproducibility" variation is variation characteristic of many operatorsmeasuring a single specimen once each (exclusive of repeatability variation)12In a typical (balanced data) Gauge R&R study, each of I itemsismeasuredmtimes by each of J operators. For example, a typical data layout for I =2parts, J =3operators and m =2repeats per "cell" might be represented as13Operator1231111112yy121122yy131132yyPart2211212yy221222yy231232yyFigure 3: A Gauge R&R Layout for I =2Parts, J =3Operators and m =2Repeats per “Cell”14Typical analyses of Gauge R&R studies are based on the so-called "two-wayrandom effects" model. Withyijk= the kth measurement made by operator j on specimen ithe model is thatyijk= μ + αi+ βj+ αβij+ ijkwhere• μ is an (unknown) constant, an average (over all possible operators andall possible parts/specimens) measurement• the αiarenormalwithmean0andvarianceσ2α, (random) effects ofdifferent parts/specimens15• the βjarenormalwithmean0andvarianceσ2β, (random) effects ofdifferent operators• the αβijarenormalwithmean0andvarianceσ2αβ, (random) joint effectspeculiar to particular part/operator combinations• the ijkarenormalwithmean0andvarianceσ2, (random) measurementerrorsσ2α,σ2β,σ2αβ, and σ2are called "variance components" and their sizes governhow much variability is seen in the measurements yijk16Example "Thought Experiment" generating a Gauge R&R data setOperator12 3y111= y121= y131=1y112= y122= y132=Party211= y221= y231=2y212= y222= y232=In this (two-way random effects) model• σ measures within-cell/repeatability variation17• σreproducibility=qσ2β+ σ2αβis the standard deviation that would
View Full Document