DOC PREVIEW
UAH CPE 619 - Simple Linear Regression Models

This preview shows page 1-2-3-23-24-25-26-46-47-48 out of 48 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

CPE 619 Simple Linear Regression ModelsOverviewRegressionSimple Linear Regression ModelsDefinition of a Good ModelGood Model (cont’d)Slide 7Slide 8Estimation of Model ParametersExample 14.1Example 14.1 (cont’d)Slide 12Derivation of Regression ParametersDerivation (cont’d)Derivation (cont’d)Allocation of VariationAllocation of Variation (cont’d)Slide 18Allocation of Variation (cont’d)Example 14.2Standard Deviation of ErrorsStandard Deviation of Errors (cont’d)Example 14.3Confidence Intervals for Regression ParamsConfidence Intervals (cont’d)Example 14.4Example 14.4 (cont’d)Case Study 14.1: Remote Procedure CallCase Study 14.1 (cont’d)Slide 30Slide 31Slide 32Confidence Intervals for PredictionsCI for Predictions (cont’d)Slide 35Example 14.5Example 14.5 (cont’d)Slide 38Visual Tests for Regression Assumptions1. Linear Relationship: Visual Test2. Independent Errors: Visual TestIndependent Errors (cont’d)3. Normally Distributed Errors: Test4. Constant Standard Deviation of ErrorsExample 14.6Example 14.7: RPC PerformanceSummaryHomework #5CPE 619Simple Linear Regression ModelsAleksandar MilenkovićThe LaCASA LaboratoryElectrical and Computer Engineering DepartmentThe University of Alabama in Huntsvillehttp://www.ece.uah.edu/~milenkahttp://www.ece.uah.edu/~lacasa2OverviewDefinition of a Good ModelEstimation of Model ParametersAllocation of VariationStandard Deviation of ErrorsConfidence Intervals for Regression ParametersConfidence Intervals for PredictionsVisual Tests for Verifying Regression Assumption3RegressionExpensive (and sometimes impossible) to measure performance across all possible input valuesInstead, measure performance for limited inputs and use it to produce model over range of input valuesBuild regression model4Simple Linear Regression ModelsRegression Model: Predict a response for a given set of predictor variablesResponse Variable: Estimated variablePredictor Variables: Variables used to predict the responseLinear Regression Models: Response is a linear function of predictorsSimple Linear Regression Models: Only one predictor5Definition of a Good ModelxyxyxyGood Good Bad6Good Model (cont’d)Regression models attempt to minimize the distance measured vertically between the observation point and the model line (or curve)The length of the line segment is called residual, modeling error, or simply error The negative and positive errors should cancel out  Zero overall error Many lines will satisfy this criterion7Good Model (cont’d)Choose the line that minimizes the sum of squares of the errors where, is the predicted response when the predictor variable is x. The parameter b0 and b1 are fixed regression parameters to be determined from the dataGiven n observation pairs {(x1, y1), …, (xn, yn)}, the estimated response for the ith observation is:The error is:8Good Model (cont’d)The best linear model minimizes the sum of squared errors (SSE):subject to the constraint that the mean error is zero:This is equivalent to minimizing the variance of errors9Estimation of Model ParametersRegression parameters that give minimum error variance are:where,and10Example 14.1The number of disk I/O's and processor times of seven programs were measured as: (14, 2), (16, 5), (27, 7), (42, 9), (39, 10), (50, 13), (83, 20)For this data: n=7,  xy=3375,  x=271,  x2=13,855,  y=66,  y2=828, = 38.71, = 9.43. Therefore,The desired linear model is:11Example 14.1 (cont’d)12Example 14.1 (cont’d)Error Computation13Derivation of Regression ParametersThe error in the ith observation is:For a sample of n observations, the mean error is:Setting mean error to zero, we obtain:Substituting b0 in the error expression, we get:14 Derivation (cont’d) The sum of squared errors SSE is:1nSSE15Derivation (cont’d)Differentiating this equation with respect to b1 and equating the result to zero:That is,11n16Allocation of VariationHow to predict the response without regression => use the mean responseError variance without regression = Variance of the responseand17Allocation of Variation (cont’d)The sum of squared errors without regression would be:This is called total sum of squares or (SST). It is a measure of y's variability and is called variation of y. SST can be computed as follows:Where, SSY is the sum of squares of y (or  y2). SS0 is the sum of squares of and is equal to .18Allocation of Variation (cont’d)The difference between SST and SSE is the sum of squares explained by the regression. It is called SSR:orThe fraction of the variation that is explained determines the goodness of the regression and is called the coefficient of determination, R2:19 Allocation of Variation (cont’d)The higher the value of R2, the better the regression. R2=1  Perfect fit R2=0  No fitCoefficient of Determination = {Correlation Coefficient (x,y)}2Shortcut formula for SSE:20Example 14.2For the disk I/O-CPU time data of Example 14.1:The regression explains 97% of CPU time's variation.21Standard Deviation of ErrorsSince errors are obtained after calculating two regression parameters from the data, errors have n-2 degrees of freedomSSE/(n-2) is called mean squared errors or (MSE). Standard deviation of errors = square root of MSE. SSY has n degrees of freedom since it is obtained from n independent observations without estimating any parametersSS0 has just one degree of freedom since it can be computed simply from SST has n-1 degrees of freedom, since one parameter must be calculated from the data before SST can be computed22Standard Deviation of Errors (cont’d)SSR, which is the difference between SST and SSE, has the remaining one degree of freedomOverall,Notice that the degrees of freedom add just the way the sums of squares do23Example 14.3For the disk I/O-CPU data of Example 14.1, the degrees of freedom of the sums are:The mean squared error is:The standard deviation of errors is:24Confidence Intervals for Regression ParamsRegression coefficients b0 and b1 are estimates from a single sample of size n  they are random  Using another sample, the estimates may be differentIf 0 and 1 are true parameters of the population. That is,Computed coefficients b0 and b1 are estimates of 0 and 1 (the mean values),


View Full Document

UAH CPE 619 - Simple Linear Regression Models

Download Simple Linear Regression Models
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Simple Linear Regression Models and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Simple Linear Regression Models 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?