Lecture Summaries ARE 261, Fall 10, Karp & TraegerLecture Summaries ARE 263, Fall 2010Final version, December 4th, 2010Lecture 1: The lecture introduced the concept of state (or stock) versuscontrol (or flow) variables. We discussed the open-loop solution to a dy-namic optimization problem, which is the time sequence of o p t i m al controlvariables. We derived the Euler equation by analyzing a perturbat io n of an(assumed to be) optimal policy sequence. Here, we changed policies in twosubsequent periods so that the state variables were not affected beyond thetime of the periods of perturbation. We obtained the Euler equation fromthe fact that the (first order) present value change of such a perturb a t i on hasto be zero. Then, we introd u ced a simple dynamic programming equationin two steps. First, we looked at a decision maker optimizing over leisureand consumption. We found that the decision can be broken up into twosequential decisions. We then applied the same idea to a truly intertemporalfishing model. We found th at we solve these models backwards in time, eventhough policies are implemented forward in time. We arrived at the T perioddynamic programming equation for the fishery problem.Lecture 2: We der i ved the Euler equation for the fishing pr oblem from thedynamic programmin g equation (D PE) . For th i s purpose, we employed theenvelope theorem giving us an equation of motion for the co-stat e variable.In addition, we used the necessary condition for maxim i zi n g the r.h.s. of theDPE. We obtained the Euler equation from inserting this necessary condi-tion for the current and the subsequent period into the equation of motionfor the co-st a t e. This procedure wil l give us th e Euler equation in moregeneral settings as well. We then turned to a capital investment p r ob l em .We switched to this ex am p l e because its linear equation of motion simplifiedour subsequent analysis. We derived the Euler equat i on and examined thesteady state (using the algebraic Euler equation) . We cannot solve for t hecontrol ru l e in general. However, we lea r n ed how t o derive an approximationof the control rule in the neighborhood of the steady stat e. This methodinvolved taking an implicit derivative of th e Euler equ a t io n aft er p l u g gi n g inour o p t i m al control as a general function of the stock (control rule, closedform). Evaluating the resulting approximation at the st eady state resulted intwo candidates (for the linear approximation to the control rule). Using the1Lecture Summaries ARE 261, Fall 10, Karp & Traegerequation of motion, our assumption that the steady state should be (locally)stable made us select the negative ro ot as the right approximation.Lecture 3: We presented the stati c “taxes versus quotas” model in which afirm knows the realization of a random cost shock. The regulator knows onlythe distr i b u t i on of this cost shock. Emissions a r e random and abatementis deterministic under taxes; emissions are deterministic and abatem ent israndom under th e quota. Social abatement costs are a convex function ofabatement and environmental damages are a convex function of emissions.We reviewed Jensen’s Inequality and applied it to show that expected abate-ment costs are greater under the quota, and expected environmental damagesare greater under the tax. Using a linear-quadratic model in which the costshock affects the intercept bu t not the sl ope of marginal co st , we showed thatthe regulator prefers taxes to quotas if and only if the margi n a l abatementcost is steeper than the margina l environmental damage. We defined thePrinciple of Certainty Equivalence.We then turned to a linear-quadratic control problem with a finite timehorizon. Using the dynamic programming equation and an inductive proof,we sh owed that the value function is quadratic in the state variable and thatthe control rule is linear in the state variabl e. We obtained difference twoequations, one of them, the Riccati difference equation , for the two par a m -eters of the value function. The the solution to these equations determinethe coefficients of the value func ti o n and the control rule. We discussedseveral features of the solution. In particular, we noted that the varianceof the random term affects the value of the program but not the optimalcontrol rule. This independence is an example of the Principle of CertaintyEquivalence.Lecture 4: We reviewed the basics of the linear-quadratic control probl em ,and proved that the maximand (the right side of the DPE) is concave in thecontrol variable. We then took the limit of the finite horizon pr oblem t oobtain the autonomous contr ol problem and the solution to that problem.In the infinite horizon model we work with th e Riccati algebraic rather thanthe Riccati difference equation. We reviewed the condition for determiningstability of a steady state of a difference equation. We discussed several waysof identifying the correct root of the algebraic Riccati equation. We thenconsidered two variations of the LQ problem, one with multiplicative riskand the second with a risk-sensitive controller. For these two variations, the2Lecture Summaries ARE 261, Fall 10, Karp & TraegerPrinciple of Certainty Equivalence does not hold: the optimal contr ol ruledepends on the variance of the random sho ck.We then showed that the static “prices versus quotas” problem discussedin the previous lecture can be generalized, using the LQ control problem, todescribe the problem of a stock rather than a flow pollutant. The policyranking depends on the magnitudes of the slopes of marginal abatement costsand margin a l damages, and also on the discount factor and on a parameterthat measures the persistence of the pollut a nt.Lecture 5: We provided a quick review of the effect of uncertainty on opti-mal decisio n s, comparing optimal actio n s with and without uncertainty. Wereviewed t h e meaning of Jensen ’ s Inequality and defined “prudence”, a char-acteristic of p r efer en ces that depends on the third derivative of the utilityfunction. Using the first and second order con d i t i on s for optimality, and aspecific two period model, we showed that the effect of great er uncertaintydepends on whether the decision maker is prudent greater. We empha si zeda gr a p h i ca l constructi on , that shows clearly the role of the second order con-dition in determining the comparative statics.We then began to con si d er the effect of