OutlineRegression FundamentalsMeasures of Model Goodness – R2Least Squares RegressionLeast Squares Regression, cont.Precision of Estimate: Variance in bConfidence Interval for bExample RegressionLack of Fit Error vs. Pure ErrorRegression: Mean Centered ModelsRegression: Mean Centered ModelsPolynomial RegressionRegression Example: Growth Rate DataGrowth Rate – First Order ModelGrowth Rate – Second Order ModelPolynomial Regression In ExcelPolynomial RegressionOutlineProcess OptimizationMethods for OptimizationBasic Optimization Problem3D ProblemAnalyticalSparse Data Procedure – Iterative Experiments/Model ConstructionExtension to 3DLinear Model Gradient FollowingSteepest DescentVarious SurfacesA Procedure for DOE/Optimization(1) DOE Procedure(2) RSM Procedure(3) Optimization ProcedureConfirming ExperimentsOptimization Confirmation ProcedureExperimental OptimizationOn-Line Optimization Continuous Optimization: EVOPSummaryMIT OpenCourseWare ____________http://ocw.mit.edu 2.830J / 6.780J / ESD.63J Control of Manufacturing Processes (SMA 6303)Spring 2008For information about citing these materials or our Terms of Use, visit: ________________http://ocw.mit.edu/terms.2.830J/6.780J/ESD.63J 1ManufacturingControl of Manufacturing ProcessesSubject 2.830/6.780/ESD.63Spring 2008Lecture #15Response Surface Modeling andProcess OptimizationApril 8, 20082.830J/6.780J/ESD.63J 2ManufacturingOutline•Last Time– Fractional Factorial Designs– Aliasing Patterns– Implications for Model Construction• Today– Response Surface Modeling (RSM)• Regression analysis, confidence intervals– Process Optimization using DOE and RSMReading: May & Spanos, Ch. 8.1 – 8.32.830J/6.780J/ESD.63J 3ManufacturingRegression Fundamentals• Use least square error as measure of goodness to estimate coefficients in a model• One parameter model:– Model form– Squared error– Estimation using normal equations– Estimate of experimental error– Precision of estimate: variance in b– Confidence interval for β– Analysis of variance: significance of b– Lack of fit vs. pure error• Polynomial regression2.830J/6.780J/ESD.63J 4ManufacturingMeasures of Model Goodness – R2• Goodness of fit – R2– Question considered: how much better does the model do than justusing the grand average?– Think of this as the fraction of squared deviations (from the grand average) in the data which is captured by the model• Adjusted R2– For “fair” comparison between models with different numbers of coefficients, an alternative is often used– Think of this as (1 – variance remaining in the residual). Recall νR= νD- νT2.830J/6.780J/ESD.63J 5ManufacturingLeast Squares Regression• We use least-squares to estimate coefficients in typical regression models• One-Parameter Model:• Goal is to estimate β with “best” b• How define “best”?– That b which minimizes sum of squared error between prediction and data– The residual sum of squares (for the best estimate) is2.830J/6.780J/ESD.63J 6ManufacturingLeast Squares Regression, cont.• Least squares estimation via normal equations– For linear problems, we need not calculate SS(β); rather, direct solution for b is possible– Recognize that vector of residuals will be normal to vector of x values at the least squares estimate• Estimate of experimental error– Assuming model structure is adequate, estimate s2of σ2can be obtained:2.830J/6.780J/ESD.63J 7ManufacturingPrecision of Estimate: Variance in b• We can calculate the variance in our estimate of the slope, b:• Why?2.830J/6.780J/ESD.63J 8ManufacturingConfidence Interval for β• Once we have the standard error in b, we can calculate confidence intervals to some desired(1-α)100% level of confidence• Analysis of variance– Test hypothesis: – If confidence interval for β includes 0, then β not significant– Degrees of freedom (need in order to use t distribution)p = # parameters estimated by least squares2.830J/6.780J/ESD.63J 9ManufacturingExample RegressionModelErrorC. TotalSource189DF8836.644064.66958901.3135Sum of Squares8836.648.08Mean Square1093.146F Ratio<.0001Prob > FTested against reduced model: Y=0Analysis of VarianceInterceptageTermZeroed 00.500983Estimate00.015152Std Error.33.06t Ratio.<.0001Prob>|t|Parameter EstimatesageSource1Nparm1DF8836.6440Sum of Squares1093.146F Ratio<.0001Prob > FEffect TestsWhole Model01020304050income Leverage Residuals0 25 50 75 100age Leverage, P<.0001Age Income86.1622 9.8835 14.3540 24.0657 30.3473 32.1778 42.1887 43.2398 48.76• Note that this simple model assumes an intercept of zero – model must go through origin• We can relax this requirement2.830J/6.780J/ESD.63J 10ManufacturingLack of Fit Error vs. Pure Error• Sometimes we have replicated data– E.g. multiple runs at same x values in a designed experiment• We can decompose the residual error contributions• This allows us to TEST for lack of fit– By “lack of fit” we mean evidence that the linear model form is inadequateWhereSSR= residual sum of squares errorSSL= lack of fit squared errorSSE= pure replicate error2.830J/6.780J/ESD.63J 11ManufacturingRegression: Mean Centered Models• Model form• Estimate by2.830J/6.780J/ESD.63J 12ManufacturingRegression: Mean Centered Models• Confidence Intervals• Our confidence interval on output y widens as we get further from the center of our data!2.830J/6.780J/ESD.63J 13ManufacturingPolynomial Regression• We may believe that a higher order model structure applies. Polynomial forms are also linear in the coefficients and can be fit with least squares• Example: Growth rate dataCurvature included through x2term2.830J/6.780J/ESD.63J 14ManufacturingRegression Example: Growth Rate Data• Replicate data provides opportunity to check for lack of fitObservationNumberAmount of Supplement(grams) x12345678910Growth Rate(coded units) y153035857565Growth rate data1010202025252573789091878691Figures by MIT OpenCourseWare.95908580757065605 10 15 20 25 30 35 40yxFit meanLinear fitPolynomial fit degree = 2Bivariate fit of y by x2.830J/6.780J/ESD.63J 15ManufacturingGrowth Rate – First Order Model• Mean significant, but linear term not• Clear evidence of lack of fitSource Sum of SquaresDegrees of FreedomMean SquareModelResidualTotalSM = 67,428.6SR = 686.4ST = 68,115.0SL = 659.40SE = 27.0{{{{mean: 67,404.1extra for linear: 24.510885.844{{21167,404.1164.856.7524.5ratio = 24.42lack of fitpure errorAnalysis of variance for