1. Ecological Models To manage fish and wildlife based on science, we need to predict... to predict we need to model. 1. What is a model? An abstraction that symbolizes the operation of processes in nature. It is no more than a hypothesis clearly stated. a. Modeling Terms Model Variables **Response variable (Y): what we are interested in predicting Observed data: y1, y2, y3, …yn **Predictor variable (X): any auxiliary information we could use to predict Y Observed data: x1, x2, x3, …xn Model Structure **Probability function: Function (equation) that maps the probability that a random draw (event) from a discrete random variable Y (e.g., number of offspring, number of dots on a die) will take the value of y. f(y) = Pr{Y = y} In this class, we will be dealing with Stochastic models Stochastic versus deterministic model – There is randomness Frequentist – data are random events from some assumed model (i.e., probability dist.) Talk about random events: length of trout, time series… Capital letter means it’s a random variable Lower-case means it’s an observation**Probability density function (pdf): For a continuous random variable Y (length, mass, population density) the area under the pdf (equation) between 2 points yL, yU is the probability that a random draw (event) y will lie between yL and yU **Parameter(s): Specifies the mathematical relationships within the model including those between the predictor variable(s) and the response **Examples of Models... Y ~ normal()2,µ σ ( )22( )2221| , ,2yf y normal eµσµ σπσ− −= × 5.5; 1µ σ= =; e = 2.718…; pi = 3.14….. 00.050.10.150.20.250.30.350.40.452 4 6 8yprobability density What is the probability of getting a value between 6 and 7? Write on overhead. Point out parameters and variables2. Parameter Estimatation Maximum Likelihood (R. A. Fisher 1922): We can use our observed data to find the most likely parameter values for a particular model... What are we doing? Trying to find the values of the parameters that maximize the probability of observing what we observed. The value of the parameter(s) that makes the likelihood as large as possible. 3. Model Selection To get good predictions (remember this is our goal!), we need a model that will closely approximate reality How can we measure how close our model is to reality (i.e., truth)? Metric for measuring distance. Absolute Difference AD = ( ) ( )yg y f y−∫ Integrated squared error ISE = ()2( ) ( )yg y f y−∫ Kullback-Leibler distance (Kullback and Leibler 1951) KL = ()( ) ln ( ) ln ( )yg y g y f y× −∫ **Note: I use g(y) to represent truth and f(y) to represent an approximating model. Integral means “SUM”…Estimates of K-L Distance without knowledge of whole truth g(y), use observed data: particular values of Y on g(y) 1.1.1. Model likelihood (similar to R2) Just as we used the log-likelihood to find the parameter values that got us closest to the data, we can use logˆL value for each model to assess how close different models are to the data. logˆL (~R2) doesn’t always give us a good idea of how close we are to truth because of the problem of overfitting y = 3.9302x + 0.4736R2 = 0.529600.511.522.530 0.1 0.2 0.3 0.4 0.5 y = 4319x5 - 4070.4x4 + 1213.6x3 - 84.117x2 - 7.6193x + 1.2873R2 = 0.786100.511.522.530 0.1 0.2 0.3 0.4 0.5 • The linear regression is too biased so predictions will be poor; it is underfit o underfit – there are enough data to specify more meaningful relationships (i.e., parameters) • The fifth order polynomial IS unbiased and is in fact the “true” model that generated the data… so why is it bad? o Can’t trust the parameter estimates – they’re too variable to make good predictions o The model is overfit – there are not enough data to specify the relationships (i.e., parameters) y = 22.007x2 - 4.8728x + 1.0311R2 = 0.723200.511.522.530 0.1 0.2 0.3 0.4 0.5 The 2nd order polynomial IS biased but it predicts better because of lower varianceNumber of Parameters in ModelBias2Variance ***BOTH underfit and overfit models predict poorly*** Because of the problem of overfitting, we cannot use the logˆL (r2) to measure how close we are to the truth. 1.1.2. Information theoretic criteria (ITC) ***Idea is that we CAN use the logˆL to estimate the relative KL distance IF we correct for overfitting*** 1.1.2.1. Akaike’s Information Criterion (AIC) ˆ2 log 2AIC L K= − ⋅
View Full Document
Unlocking...