Regressio n Analysis by Dr Mahamood Usman Khan Ph D IIT ISM Dhanbad M Phil M Sc AMU Regression Regression analysis is used to estimate the relationship between the response and predictors Regression analysis relationship independent variables between is used the to estimate dependent the and We want to predict the value of a variable based on the value of another variable It is used for predictions and forecasting Simple Regression Model Explanation 1 Xf error Y Actual relation Estimated relation Residual e ENDEAVOR Minimizing ERROR Simple Regression Model Explanation 2 Estimation of Parameters LS Method y 0 1 x The least square estimates of n i 1 1 x i yx i y n i 1 x i 2 x n 0 and xy i i 1 n i i 1 1 are n x y i 1 i xn 2 2 x i Cov Var yx x y x 1 y 0 The estimated regression line is x y 1 0 x Real Life Situations Businesses often use regression analysis to understand the relationship between advertising spending and revenue Agricultural scientists often use regression analysis to measure the effect of fertilizer and water on crop yields Medical researchers often use regression analysis to understand the relationship between drug dosage and blood pressure of patients The relationship between global warming and the melting of glaciers Forecasting through Regression revenue 0 1 ad spending The coefficient 0 would represent total expected revenue when ad spending is zero The coefficient 1 would represent the average change in total revenue when ad spending is increased by one unit e g one dollar If 1 is negative it would mean that more ad spending is associated with less revenue If 1 is close to zero it would mean that ad spending has little effect on revenue And if 1 is positive it would mean more ad spending is associated with more revenue Depending on the value of 1 a company may decide to either decrease or increase their ad spending Regression of Y on X X on Y Regression of Y on X Regression of X on Y Why two regression lines Regression Coefficients N N XY X X 2 2 X Y N XY YN X 2 2 Y Y Coefficient of determination 2R Residual Observed value predicted value Residuals e y y A residual is the vertical distance between a data point and the regression line Each data point has one residual They are Positive if they are above the regression line Negative if they are below the regression line Zero if the regression line actually passes through the point The sum of the residuals always equals zero Consequently the mean of residuals is also equal to zero Example 1 Problem For the data given below Compute the followings i Regression of Y on X ii Regression of X on Y iii Regression Coefficients iv Coefficient of determination v Residuals vi Standard error of estimates Solution Example 2 Solution Example 3 Multiple Regression General Formula for Multiple Regression Regression Model With Two Independent Variables y b 0 xb 11 xb 22 Normal Equations for Solving the Regression Coefficients Least Square Estimate of Regression Coefficient The regression coefficient of the regression model consisting two independent variables are estimated using the Least Square Estimation Method given as follows Visualization Example Solution ThankYou