Final Exam Review Model vs Data December 15 2006 Model vs Data In most experiments we can control a set of independent variables x and can measure the value of the dependent variable y For such a system we can propose a model which relates the value of the dependent variable to the values of the independent variables as shown in Equation 1 y f x 1 The aim of the generating a model for the system is to obtain an answer to the following three questions For what value of is the deviation between model and data minimum Is the model consistent with the data What are the error bars on the values of parameters In the context of models we classify models as linear or non linear Linear models depend on the parameters linearly For example the log of the rate of an arrhenius reaction is linear in the parameters log A and ERa This model is linear even tough log A is not linearly dependent on the dependent variable temperature T Ea 2 RT Usually in solving these problems we make the following two assumption log k log A 1 The dependent variable y that is being measured is distributed nor mally a gaussian distribution around its mean value This distri bution can be due to many factors which are not in control of the experimentalist 2 On the other hand the independent variables x are known exactly Cite as Sandeep Sharma course materials for 10 34 Numerical Methods Applied to Chemical Engineering Fall 2006 MIT OpenCourseWare http ocw mit edu Massachusetts Institute of Technology Downloaded on DD Month YYYY 1 Best values of parameters Let us assume that the model that we have is correct and that the standard deviation of the random errors in the measurement of the dependent variable y is Then the probability of getting a measured value yi is given by the Equation 3 yi f xi 2 p yi exp 2 2 3 If we perform N di erent experiments then the probability of obtaining a vector y measurements of the dependent variables is given in Equation 4 N yi f xi 2 p y exp 2 2 i 1 N 1 exp 2 yi f xi 2 2 i 1 4 The probability of getting this vector y becomes highest when sum of squares of errors become minimum Thus the idea of minimizing the sum of squares of errors is based on the assumption that the errors in the mea surement are normally distributed Central limit theorem ensures that this assumption is justi ed if we assume that each measurement is obtained by performing many repeats If the model is linear then we have an analytical solution for the best t values of the parameters If the model is non linear in parameters then there can potentially be multiple local minima and we have to be careful The linear model can be written as shown in Equation 5 y X 5 The solution to the model is given by the expression in Equation 6 X T X 1 X T y 6 A potentially better method of solving minimization problem is by per forming SVD decomposition of X recall SVD of X gives three matrices U and V which are related to X as X U V The best t values of are given in Equation 7 N i 1 Ui y Vi i 7 Cite as Sandeep Sharma course materials for 10 34 Numerical Methods Applied to Chemical Engineering Fall 2006 MIT OpenCourseWare http ocw mit edu Massachusetts Institute of Technology Downloaded on DD Month YYYY 2 Model consistency Let be the variance in the measured data Then the probability that we get a vector y of measured values is given by Equation 4 Now we de ne a parameter 2 as 2 N yi f xi 2 i 1 The least square method of calculating the parameters is nothing but the same as minimizing the value of 2 Also 2 is the a sum of N normally distributed quantities with mean 0 and variance 1 This 2 is itself a random variable and is distributed as chi square distribution with N dim degrees of freedom This chi square distribution can be used to quantify the goodness of the t The probability of a model being correct is given by the area under the curve of a chi square pdf between the abscissa 2 and inf Con dence intervals If we know the value of we can assume that y is distirubted normally around its mean value y with a variance We can then go ahead and calculate the approximate probability distribution functions of the parameters From the probability distribution functions of the parameters we can calculate the 95 con dence intervals for the parameters If the model is linear in parameters then one would expect that if the pdf of y was normal then the pdf of the parameters would also be normal This is infact true and for more than one parameter we obtain a higher dimension gaussian The covariance matrix for the parameters cov 2 X T X M 1 The 95 con dence intervals for a parameter is given in Equation 8 i M j Z2 5 X T X M 1 jj 1 2 8 An interesting thing to note in the above equation is that the error bars on a parameter i depends on the matrix X and thus by cleverly chosing our experimental conditions we can use a X that minimizes the error bars on the parameter of interest When the model is not linear we can still use Equation 8 to calculate the error bars on the parameters only in this case we can generate a linearised design matrix using Equation 9 and then use Equation 7 to calculate the error bars on the matrix Xi j f xi j 9 This is the same way that matlab function nlinfit and nlparci work Note that by using this linearized design matrix we lose the information Cite as Sandeep Sharma course materials for 10 34 Numerical Methods Applied to Chemical Engineering Fall 2006 MIT OpenCourseWare http ocw mit edu Massachusetts Institute of Technology Downloaded on DD Month YYYY 3 of the covariance between di erent parameters The graphical way to look at any pair of parameters is to plot the 2 value for a range of these two parameters To convert the 2 plot to a plot of probability we just calculate the value of 2 2 1 2 2min and this value of 2 is distributed with a chi square distribution of 2 degrees of freedom An example of this was worked out in the homework 9 Cite as Sandeep Sharma course materials for 10 34 Numerical Methods Applied to Chemical Engineering Fall 2006 MIT OpenCourseWare http ocw mit edu Massachusetts Institute of Technology Downloaded on DD Month YYYY 4