14.

451 Notes
Samuel Isaac Grondahl
September 15, 2014

1
1.1

Mathematical Preliminaries
Theorem of the Maximum

Assumption. Let X Rn and Y Rm , and f : X Y R continuous, and let : X Y nonempty,

compact valued, and continuous correspondence. Then x, f (x, ) attains a maximum within (x) so that we
can define a function h : X R and a correspondence G : X Y as follows:
h(x) := max f (x, y), x
y(x)

G(x) := {y (x) | f (x, y) = h(y)}, x

Theorem (SLP 3.6 - Theorem of the Maximum). Under the aforementioned assumption, a maximum exists,
h is continuous, and G is compact-valued and u.h.c.
Note: in this case, f is the function in the Bellman equation that is the sum of F (x, y) + v(y) (perhaps
bad notation, but this is how it applies).
Assumption. Let be non-empty, compact valued, and continuous. Next ,consider a continuous function
f and a sequence of continuous functions fn that converge uniformly in the sup norm to f . Finally, suppose
that all these functions are strictly concave in y, x, and that (x) is stritly convex x. It follows that the
following are well-defined and continuous:
g(x) := arg max f (x, y)
y(x)

gn (x) := arg max fn (x, y), n

y(x)

Theorem (SLP 3.8). Under the aforementioned assumptions, gn converges to g pointwise. If in addition,
X is compact, then gn converges to g uniformly.
We are going to use this result in the following way. Remember, f is the whole thing in the Bellman
equation. So now with T as a contraction mapping, then for an arbitrary starting function v0 , we will get a
sequence of function vn (with vn = T vn1 ) that will converge to v. Corresponding to this sequence, we will
get a bunch of fn := F (x, y) + vn (we also get a sequence gn that converges to policy rule g).
Note that we can start our sequence with v2 , which is the value function for the 2 period optimization
problem. Note that T v2 = v3 is the solution to the 3 period finite horizon problem. Then the sequence {vn }
must converge in particular to V . So we get another nice property with this starting value (the dynamics
of the long term finite horizon problem are going to be similar to those of the infinite horizon problem).
Definition. Let (S, ) metric space. We say T : S S is a contraction mapping with modulus if scalar
(0, 1) s.t. x, y S,
(T x, T y) (x, y)
1

Theorem. If (S, ) is a complete metric space and T : S S is contraction mapping with modulus , then:
1. T has exactly one fixed point v S
n

2. v0 S the sequence {vn }

n=0 defined by vn := T v0 converges to v .

Note: completeness of space gives that fixed point will always be converged to (and exists) but uniqueness
of fixed point doesnt require completeness (so if we get first property some other way without completeness,
we can use second property).
Theorem (Blackwells Sufficient Conditions - SLP 3.3). Take set X and let B(X) be space of bounded
functions f : X R with sup norm. Let T operator on space with two properties:
monotonicity f, g B(x), then f g T f T g
discounting (0, 1) : f B(X), a R, then T (f + a)leqT f + a
Then T is a contraction map with modulus .
Proposition (Corollary to SLP 3.2). Let (S, ) complete metric space, T contraction mapping on it, v is
unique fixed point. If S 0 is closed subset of S and T (S 0 ) S 00 S 0 , then v S 00 .
Proof. Pick arbitrary v0 S 0 and construct sequence {T n v0 }. By assumption, T (S 0 ) S 0 , this whole
sequence contained in S 0 . By assumption that S 0 closed, if sequence converges, limit must be in S 0 . By
contraction mapping, sequence must converge and limit is v . Hence v S 0 .

Bounded Returns

Now we relax conditions and not assume max attained, so we switch from max notation to sup notation.
Assumption (SLP 4.3 and 4.4). The set X is a convex subset of Rn ; the correspondence is non-empty,
compact-valued, and continuous; and the payoff map F is bounded and continuous
Under these assumptions, use contract mapping theorem to establish that unique solution to Bellman
equation, and that the sup is attained (hence we can replace it with a max), then use this to establish that
we have a max in the sequence problem (which is different!).
Proposition. ! bounded function V that solves the Bellman equation (FE).
Proof. Let B(X) set of bounded functions on X, endow this set with the sup norm, and consider the operator
T : B(X) B(X) defined by
T v(x) := sup {F (x, y) + v(y)}
y(x)

Since F is bounded, T v is bounded whenever v is bounded; hence, T indeed maps B(X) into itself.
Moreover, B(X) with sup norm gives Banach space (complete, normed). Prove this by establishes that
T is a contraction mapping (use Blackwell sufficient conditions).
monotonicity Take f, g B(X) and suppose f g. Then take any x X and some y 0 arg maxy(x) [F (x, y) +
g(y)]. Then
T f (x) F (x, y 0 ) + f (y 0 ) F (x, y 0 ) + g(y 0 ) = T g(x)
thus we have T f T g.

discounting Pick any a 0 and any g B(X) and let f = g + a, then

T f (x) = max {F (x, y) + [g(y) + a]}
y(x)

** TODO: FINISH PROOF FROM NOTES ** there is additional stuff here.

Thus we have that T is a contraction mapping, hence the Bellman equation admits a unique solution (at
least inside the set of bounded functions).
Proposition. V is given by the unique bounded solution to (FE).
Proof. This is trivial. We know that V is a solution to FE, and we proved that FE has a unique solution,
hence it must be V . Hence, V = V .
Note that bounded functions trivially satisfy the first property in Theorem 4.3 in SLP.
Proposition. V is continuous and the supremum in (SP) is attained.
Proof. Let C(X) be set of continuous and bounded functions and consider the operator T :
T v(x) := sup [F (x, y) + v(y)]
y(x)

Whenever v C(X), the function F (x, y) + v(y) is also continuous (since F is continuous by hypothesis).
Together with assumption that (X) is non-empty, compact-valued, and continuous, so we can apply the
Theorem of the Maximum to guarantee that the sup above is attained that that T v is itself continuous (i.e.
if we start with continuous v, then the ToM gives us that sequence T n v are also all continuous functions).
Hence T maps C(X) into itself, i.e. T : C(X) C(X). Since C(X) is closed subset of B(X) and T is a
contraction on B(X), we get that the fixed point of T is in C(X), and therefore that V itself is continuous.
This proves the first part of the result.
Next, since V is continuous, we can replace the sup in (FE) with a max (i.e. we actually attain it, since
the space is complete); i.e. x X, y (x) | V (x) = F (x, y) + V (y). Iterating this, we have plan
x = {xt }
t=0 s.t. finite ,

V (x0 ) =
t F (xt , xt+1 ) + V (x +1 )
t=0

Since V is bounded, lim V (x +1 ) = 0. Thus in the limit

V (x0 ) = lim

t F (xt , xt+1 )

t=0

hence the plan x attains the supremum in (SP), establishing the second part of the proposition.

2.1

Monotonicity and Concavity

Assumption (SLP 4.5 and 4.6). F is strictly increasing in x (but not necessarily in y) and is nondecreasing in the sense that (x0 ) (x00 ) whenever x0 x00 .
Proposition. V is strictly increasing.
Proof. Pick any f C(X) ** TODO: finish proof from notes **
Assumption (SLP 4.7 and 4.8). F is strictly concave and is convex in the sense that
y 0 (x0 ) and y 00 (x00 ) = y 0 + (1 )y 00 (x0 + (1 )x00 ) [0, 1]
Proposition. The value function V is strictly concave and the policy correspondence G is a continuous,
single-valued function.
3

2.2

Convergence for Policy Functions

Proposition. Let v0 be bounded, continuous, and concave function on X and construct the sequences {vn }
and {gn } as follows:
vn+1 = T vn := arg max [F (x, y) + vn (y)]
y(x)

gn := arg max [F (x, y) + vn (y)]

y(x)

Then gn converges to g pointwise where g(x) = arg max[F (x, y) + V (y)]. If in addition X is conpact, then
the convergence is uniform.
Proof. We have already shown that T maps any concave function to a (strictly) concave function. Let then
fn = F + vn , f = F + V , and apply Theorem SLP 3.8. (Recall that this is the result we discussed right
after the Theorem of the Maximum in Lecture Notes #3).

2.3

Differentiability

To describe the optimum, it is often useful to use FOC of the FE problem:

y F (x, y) + V (y) = 0
for this, though, we need F differentiable in y and also V differentiable. We can establish this under only
certain conditions (note: F differentiable is not sufficent).

Proposition. Suppose F (x, y) is differentiable in x and the value function V (x) is concave. Take an x0 X

and suppose that y0 G(x0 ) (x0 ). Then V is differentiable at x0 with

V (x0 ) = x F (x0 , y0 )
Note that this is very similar to the statement of the envelope theorem.

Neoclassical Growth Model (NGM) with Flexible Labor

Properties of the economy:

populated by a single representative agent
single good, produced by combination of labor and capital, which can neither be consumed nor invested
into the next period capital
P
preferences given by t=0 t U (ct , lt )
ct consumption
lt leisure
(0, 1) discount rate
U : R2+ R+ per-period utility function is continuous, strictly increasing, strictly concave
leisure bounded above wlog by 1
aggregate output in period t is given by yt = F (kt , nt )
kt capital
nt labor

 F : R2+ R+ aggregate production function is continuous, strictly increasing in (n, k) as long as

(n, k) > (0, 0), and concave (but not necc strictly concave, since we could have constant returns
to scale which is only weakly concave)
average product of

F (k,n)
k

decreasing in k and vanishes as k , n

resource constraint
** TODO write down additional constraints from notes **
Note: with sufficiently high starting capital, since returns are diminishing, the level of capital cannot
possibly be sustained for future periods (production to that level is not feasible). Thus wlog we can restrict
where k is the threshold feasible/sustainable level of capital as described
the state space for k to be k [k, k]
above.
Proposition. Consider the NGM (with flexible labor supply). The maximum is attained and the value function V is obstained from the unique bounded solution to the Bellman equation. Moreover, V is continuous,
strictly increasing, strictly concave, and the policy function g(k) is continuous.

3.1

NGM with Fixed Labor (Ramset-Cass-Koopmans Optimal Growth Model)

Suppose labor supply exogenous fixed at n = n

(0, 1), and let f (k) := F (k, n
) + (1 )k (note f has same
properties as ).
** TODO write additional conditions from notes **
Note: in all of these, we are concerned with the planners problem
Proposition. The policy function g(k) is continuous and non-decreasing. The consumption function c(k) :=
f (k) g(k) is continuous and strictly increasing.
Proof. Continuity of g already established, and continuity of c follows immediateley from continuity of g and
f.
To prove monotonicity, let u be correspondence that gives all subgradients of u; let V correspondence
that gives all subgradients of V .

3.2

Steady State and Stability

Study of dynamics of the optimal path, using properties weve established for policy function g. We first
characterize the steady state(s) of the conomy, then study the transitional dynamics of the economy toward
(or away) from its steady state(s).
Definition. A steady state is a point k such that k = g(k ).
Proposition. There exists a unique non-zero steady state and it is globally stable (expectif k0 = 0). Furthermore, transition is monotonic in the sense that both capital and consumption increase over time if k0 < k ,
and the both decrease over time if k0 > k .