9-05

Notation: To distinguish the problem's "d" constant from time-differential dt, I write the latter "d" upright. Also, let me write $LaTeX: F(t,x,y)=\big(ax^2+2bxy+cy^2+d\cdot t^2y\big)e^{-rt}$ as a function of (t,x,y), while for part (a),
LaTeX: q(x,y)=ax^2+2bxy+cy^2 and
$LaTeX: Q(x,y)=(q(x,y)+d\cdot t^2y)e^{-rt}$ ; that is, Q is as F but considered to be a function of only (x,y), with t as a parameter.

(a): Concavity of Q?

We are asked for concavity wrt. (x,y), and then t is a constant; then $LaTeX: t^2d\cdot y$ is linear and does not affect concavity, and neither does the positive multiplicative constant e^-rt. So Q has the same concavity property as the quadratic form q(x,y), which is negative semidefinite and hence concave iff $LaTeX: \underline{\underline{a\leq 0\ \wedge\ c\leq 0\ \wedge\ ac\geq b^2}}$ .

(b): Euler equation associated to the problem $LaTeX: \displaystyle\max\int_0^TF(t,x,\dot x)\, \mathrm dt$ with $LaTeX: x(0)=x_0,\ x(T)=x_T$ ?

We need $LaTeX: \frac{\partial F}{\partial x}=\frac{\partial q}{\partial x}e^{-rt}=(2ax+2by)e^{-rt}$ and $LaTeX: \frac{\partial F}{\partial y}=\big(\frac{\partial q}{\partial y}+t^2d)\cdot e^{-rt}=(2bx+2cy+t^2d)e^{-rt}$ and $LaTeX: \frac{\mathrm d}{\mathrm d t}\Big[(2bx+2c\dot x+t^2d)e^{-rt}\Big]=(2b\dot x+2c\ddot x+2td)e^{-rt}-r(2bx+2c\dot x+t^2d)e^{-rt}$ .

Inserting: $LaTeX: (2ax+2b\dot x)e^{-rt}=(2b\dot x+2c\ddot x+2td)e^{-rt}-r(2bx+2c\dot x+t^2d)e^{-rt}$ from which we cancel the exponential and 2bx' and divide by 2
$LaTeX: \underline{\underline{0=c\ddot x+td-r(bx+c\dot x+\tfrac d2 t^2)-ax}}$

Comment: This is enough, although it can be prettified by gathering together the -(a+rb) in front of x, and get the terms containing t over to the other side.
Now, you can ask whether it is permissible to do as the compendium solution p. 38, and divide by c which is not assumed nonzero; If c=0, the problem really degenerates: you don't even get a differential equation for x, just an algebraic one. (How to fit that to initial and terminal conditions?)
So although I generally frown upon dividing by anything unspecified, this is actually a case where the Euler equation is equivalent to on p.38 except when the problem is quite a bit meaningless.

Part (c): Solve when T=1, r=x₀=x₁=0, a=-9, 2b=2, c=-1, d=3.

ac-b^2=9-1>0, so the following will indeed be the solution.

Insert into the Euler equation, we get $LaTeX: \ddot x-9x=3t$ . Characteristic roots -3 and 3, and particular solution -t/3. Fit $LaTeX: Ae^{3t}+Be^{-3t}-t/3$ to initial and terminal conditions: t=0 yields B=-A and t=1 yields $LaTeX: A\cdot(e^{3}-e^{-3})-1/3=0$ .
Solution: $LaTeX: x^*=\underline{\underline{\frac13\Big(\frac{e^{3t}-e^{-3t}}{e^3-e^{-3}}-t\Big)}}$ .

Part (d)

Let me first do as much as possible without invoking the terminal constraint, and fill in those afterwards. The problem has running income = F(t,x,u) = LaTeX: -9x^2+2xu-u^2+3 t^2u and Hamiltonian
LaTeX: H(t,x,u,p)=-9x^2+2xu-u^2+3 t^2u+pu . This is concave in (x,u), so whatever satisfies the below necessary conditions will indeed solve the problem.

We have the necessary conditions:

The optimal control maximizes (over all real u).
- Carrying out the maximization of the strictly concave quadratic, we get
The adjoint process satisfies the differential equation with transversality conditions as follows:
Problem (i): p(1)=0
Problem (ii) p(1)≥0 with equality if x*(1)>2.
- Inserting u* from cond'n 1 gives $LaTeX: \dot p=18x^*-2(x^*+(p+3t^2)/2))=16x^*-p-3t^2$
x* satisfies with the appropriate boundary conditions.
That shall lead to the Euler equation as follows:
- Inserting: $LaTeX: \dot x^*=x^*+(p+3t^2)/2$ .
- Differentiating once more: $LaTeX: \ddot x^*=\dot x^*+\tfrac12 \dot p+3t$ , which when we insert, becomes $LaTeX: \ddot x^*=\dot x^*+\tfrac12 \big(16x^*-p-3t^2\big)+3t$ and then because p+3t^2=2(u*-x*), we get that x* satisfies the second-order differential equation $LaTeX: \ddot x=\dot x+8x-(\dot x-x)+3t$ ...
- ... which is the Euler equation, no surprise!

So as above, we have the Euler equation for the state variable: the general solution is $LaTeX: Ae^{3t}+Be^{-3t}-t/3$ , and we eliminate one constant by putting x(0)=0: B=-A. So for some constant A, the optimal x* is $LaTeX: A\cdot(e^{3t}-e^{-3t})-t/3$ .

For the two problems at hand, we need p(1). Because p+3t^2=2(u-x)=2(x'-x), we have
$LaTeX: p(t)=-3t^2+2\Big[A(3e^{3t}-(-3)e^{-3t})-\tfrac13 - A(e^{3t}-e^{-3t})+\tfrac13t \Big]$ and so
$LaTeX: p(1)=-3+2A\Big[2e^3+4e^{-3}\Big]$ . To complete the solution:

(i) if x(1) is free, then p(1)=0 and $LaTeX: A=3/\big[4e^3+8e^{-3}\big]$ . Then the optimal path is
$LaTeX: \displaystyle x^*(t)=A\cdot(e^{3t}-e^{-3t})-t/3=\underline{\underline{3 \cdot \frac{e^{3t}-e^{-3t}}{4e^3+8e^{-3}}-\frac t3}}$
(if I have calculated correctly, but it looks like last year's solution ... and the method should be correct)
(ii) x(1)≥2. Two things could happen: > or =. But let us rather split into the following two possible outcomes:
- p(1)=0. Reason to do that: because we then know that x* must be as in the previous item, namely $LaTeX: 3 \cdot \frac{e^{3t}-e^{-3t}}{4e^3+8e^{-3}}-\frac t3$ . But is that admissible? Put t=1, we see that the numerator first ratio is <3/4, so clearly not; x(1) will be <2! Therefore, this case is impossible.
- p(1)>0 and thus x(1)=2. We are back to $LaTeX: A\cdot(e^{3t}-e^{-3t})-t/3$ , but unlike part (c) we shall now fit it to x(1)=2. That gives A=(7/3)/(e^3-e^(-3)), and before concluding, we need to check that this A does indeed give p(1)>0: we have $LaTeX: \displaystyle p(1)=-3+2A\Big[2e^3+4e^{-3}\Big]=\frac {14}{3}\cdot \frac{2e^3+4e^{-3}}{e^3-e^{-3}}-3=\frac {1}{3}\cdot \frac{28e^3+56e^{-3}-9(e^3-e^{-3})}{e^3-e^{-3}}>0$
  and so the following is indeed the solution:
  $LaTeX: \displaystyle \underline{\underline{x^*(t)=\frac 73\cdot \frac{e^{3t}-e^{-3t}}{e^3-e^{-3}}-\frac 13t}}$
  - NOTE: Did we really have to calculate p(1) completely? Nah ... We see that the A that gave p(1)=0 gave too low x(1), and we see that the formula for x*(t) = A*[positive] is increasing in A. Therefore, we have increased A. And also the formula for p(1)=-3+A*[positive] is increasing in A, so increasing A gives larger p(1) than the zero we had.
[And just to mention the case that part (d) does not ask for: the one where x(1) is given by an equality. That case goes like the lattermost except that we don't need to check that p(1)>0.]