P1maxmTt0ertU x(t),y(t) dts.t.˙x  f(x)  y state equationxt0given initial conditionx(T)  xTterminal conditionLarry KarpNotes for DynamicsSeptember 19, 2001II. General Dynamic Problem in Resources, Calculus of Variations1) Description of dynamic optimization problem2) Statement of necessary condition (Euler equation)3) Interpretation of Euler Equation4) Autonomous problems and steady states5) Derivation of Euler Equation6) Special functional forms7) Necessary and sufficient conditions8) Different boundary conditions for dynamic problems9)An example: extraction of a non-renewable resource1) Examples of control problems in natural resourcesx = stock of resources, x = initial condition0y ="harvest"f(x) = natural growthU(x,y) flow of payoff - the integrandr = discount ratee = discount factor-rtT = final time x = terminal conditionTIn this problem, x is the state variable, y is the control variable, and dx/dt = f(x) -y is the equationof motion.++/-fisheries U(x, y)F(t,x,˙x) / w f(x)  ˙x  c x,f(x)  ˙xertP2maxmTt0F(t,x,˙x)dts.t. x0givenxTgivenWhen using the Maximum Principle, we typically retain the control variable y, and we do1not eliminate the equation of motion.2:2-+fisheries (compet) = p(t)y - c (x, y)fisheries (monopoly) = p(y) @ y-c(x, y)fisheries (social planner) = w(y)-c(x,y)wherewN/p(y)____________________pollution x = f(x)+g <---contribution to pollution+U(x,g)= v (g) - D (x)benefit damagefunction functionnonrenewable resource f(x) / 0 x =-yFor the C.O.V. problem write integrand as F (t, x, x )usingy = f(x)-x . For example, for fisherycontrolled by social planner:I can write P1 asI have eliminated the constraint x =f(x)-y by substitution. I get rid of one variable (y) and oneconstraint and write the problem in terms of x and its derivative dx/dt. It is common to eliminatethe control variable (here y) when using COV to solve a dynamic problem. That is, instead ofchoosing the optimal trajectory for the control variable y (harvest), we explicitly choose dx/dt.12. Necessary ConditionFx(t,x,˙x) ddtF˙x(t,x,˙x) F˙xt F˙xxdxdt F˙x ˙xd2xdt2 F˙xt F˙xx˙x F˙x ˙x¨xmt1tFx(τ)dτ mt1tdF˙x(τ)  F˙x(t1)  F˙x(t) mt1tFx(τ ,x(τ),˙x(τ))dτ F˙x(t,x(t1), x(t1)  F˙x(t,xt,˙x(t))2:3(Subscripts indicate partial derivatives.) A necessary condition for P2 is that the optimal x solves*the Euler equation:This is a second order ODE, and we have 2 B.C.'s, x(()=x and x(T)=x This is a Atwo point( Tboundary value problem@ (TPBVP)3. Interpretation of Euler EquationInterpretation of Euler equation: Integrate, usingor,=Benefit over t,t of having Leaving one more fish in sea at t and1the extra fish in the stock taking it out at t (reallocating consumption)1Mention case where F / 0. This inequality would hold if, for example, a larger stock doesxnot make it any cheaper to catch fish, and the growth is independent of the stock of fish. Theindependence of growth on the stock of fish is not plausible. However, the independence of thegrowth on the stock does hold for a nonrenewable resource, such as oil, or for a resource withexogenous growth (e.g. the water in an aquifer, where additions to the stock might be independentof the amount of water currently in the aquifer).If F / 0 the Euler equation implies that the present value of the marginal utility ofxconsumption is constant.Continue interpretation of Euler Equation for fish example with no stock dependent costsx =f(x)-ymaxmT0e rtU f(x)  ˙xdt x0,xTgivenddt ertUN(y)  ertUN(y) fN(x)ertUN(yt) mt1tersU N(ys)fN(xs)ds e rt1UN(yt1)ddtertUN(y)rertUN(y)  ertUNN(y)˙y  ertUN(y)fN(x)UNN(y)UN(y)˙y  fN(x)  r2:4Here y is the harvest (consumption) and U(y) is utility of consuming fish.Euler equation isIf I integrate EE over (t, t )Iget1LHS = PDV of consuming extra fish at tConsider the following perturbation of the optimal trajectory. If I refrained from eating the marginalfish at time t, and then ate it at time t , I can eat the Aextra fish@ produced by this marginal fish over1the interval (t,t ). The marginal fish (that I refrain from eating at time t) increases the growth by f1N(x ). When I eat these extra fish, I obtain a PDV of marginal utility of eUN(y ) f N(x ) at time s <ts ss 1-rs(and I still have the marginal fish at time s). I eat the marginal fish at t . At optimum, there is no1benefit of reallocating consumption in this manner. In other words, the perturbation that I describedyields no increase in utility.In the problem above, time enters only through the discount factor, and because of the upper limitof integration -- the finite horizon -- T. This fact makes it possible to eliminate t from the Eulerequation, as shown below.If I write out Euler equation (which requires that I find and cancel e I have:-rtThis expression is independent of time.ddtUN(y)UN(y)Y UNNUN>0Y ˙y >0 iff fN(x)  r >0˙yy fN(x)  r2:5I can write LHS of the last version of the Euler Equation asIn words, the EE says that along the optimal trajectory, the proportionate rate of change of marginalutility must equal the difference between growth rate of capital stock (fish) and "rate of impatience"concave utilitySpecial case U(y)=lny, Euler equation is4. Autonomous Problems and Steady StatesConsider the previous fish problem (where harvest costs do not depend on the stock of fish), butreplace the upper limit of integration T by infinity . The optimal trajectory to this infinite horizonproblem converges to a steady state. At this steady state (where y is constant) it must be the casefN(x) = r. In other words, the steady state for the infinite horizon version of this problem isindependent of the utility function, U. The steady state requires that the marginal growth rate of fishequals the discount rate.The control problem is said to be autonomous if and only if the horizon is infinite and time entersexplicitly only through a constant discount rate. In this case, the optimal control depends only onthe state, not on time.Think about why this time-independence requires an infinite horizon. If there were a finite horizon,the amount of Atime-to-go@ would affect the optimal control. If there is a finite horizon, but timedoes not enter the integrand except via discounting, we saw from the previous section that the EulerEquation is independent of time. However, with a finite horizon, the amount of time until we reachthe