1

Differentiation

Chapter 2

Differentiation

Now that we have a good understanding of the Euclidean space Rn , we are ready to discuss
the concept of differentiation in multivariable calculus. We are going to deal with functions
defined on one Euclidean space with values in another Euclidean space. We shall use the
shorthand notation
f

Rn Rm
to describe such a situation. This means that f can be written in the form
f (x) = (f1 (x), f2 (x), . . . , fm (x)) ,
where x = (x1 , x2 , . . . , xn ) and each coordinate function fi is a real-valued function on Rn . In
these situations it may be that f is not defined on all of Rn , but well continue with the above
shorthand.
There are two important special cases: n = 1 and m = 1, respectively. We shall quickly
see that the case n = 1 is much, much simpler than all other cases. We shall also learn that
the case m = 1 already contains almost all the interesting mathematics that we investigate
the generalization to m > 1 will prove to be very easy indeed.
A. Functions of one real variable (n = 1)
In the situation
f

R Rm
we shall typically denote the real numbers in the domain of f by the letter t, and the points
in Rm in the usual manner by x = (x1 , x2 , . . . , xm ). As we mentioned above, we can represent
f in terms of its coordinate functions:
f (t) = (f1 (t), f2 (t), . . . , fm (t)) .
This formula displays the vector f (t) in terms of its coordinates, so that the function f can
be regarded as comprised of m real-valued functions f1 , f2 , . . . , fm . We often like to think of
real-valued functions in terms of their graphs, but when m > 1 this viewpoint seems somewhat
cumbersome. A more useful way to think of f in these higher dimensions is to imagine the
points f (t) plotted in Rm with regard to the independent variable t. In case f (t) depends
continuously on t, the points f (t) then form some sort of continuous curve in Rm :

Chapter 2

f (t )

We have placed arrows on our picture to indicate the direction of increasing t. Thus the
points f (t) form a sort of curve (whatever that may mean) in Rm .
We need to understand well the definition of limit as t t0 and/or continuity at t0 . As
we are somehow interested in the size of f (t) f (t0 ), we can merely use the definition of
continuity in the case of real-valued functions, modified so that instead of absolute value we
use the norm. Thus we have the
f

DEFINITION. Let R Rm . Then f is continuous at t0 if for each > 0 there exists > 0
such that
|t t0 | < kf (t) f (t0 )k < .
f

PROBLEM 21. Let R Rm and let L Rm . Write out the correct definition of
lim

tt0 ,t6=t0

f (t) = L.

Then prove that f is continuous at t0

lim

tt0 ,t6=t0

f (t) = f (t0 ).

In preparation for the important characterization of continuity by means of the coordinate

functions, work the following
PROBLEM 22. Prove that for all x Rm
max |xi | kxk

1im

m max |xi |.
1im

THEOREM. Let R Rm . Then f is continuous at t0 all the coordinate functions f1 ,

f2 , . . . , fm are continuous at t0 .

Differentiation

PROOF. =: Let > 0. Then the continuity of f guarantees that there exists > 0 such
that
|t t0 | < = kf (t) f (t0 )k < .
By Problem 22, conclude that
|t t0 | < = |fi (t) fi (t0 )| <
for each i. Thus each fi is continuous at t0 .
PROBLEM 23.
theorem.

Write out in careful detail the proof of the other half (=) of the

QED
Though this result reduces continuity to that of real-valued functions, we prefer the original
definition in terms of the norm of f (t) f (t0 ). For that definition is more geometric and
does not involve the coordinates of Rm at all.
f
We shall always assume that R Rm is at least continuous and defined on an interval in
R, which could be all of R itself.
DEFINITION. A curve in Rm is a continuous function f from an interval [a, b] into Rm .
The independent variable a t b is sometimes called a parameter for the curve.
Here are some specific examples:
Straight line: f (t) = x + tv, where x, v Rn , v 6= 0.
Unit circle in R2 : f (t) = (cos t, sin t).
Unit circle in R2 : f (t) = (cos t, sin t).
Helix in R3 : f (t) = (cos t, sin t, t).
A curve in R2 : f (t) = (t2 , t3 ).
REMARK. Our definition of curve is perhaps somewhat unusual. Normally we think of
a curve in Rm as some sort of subset of Rm which has a continuous one-dimensional shape.
This would correspond to the set which is the image of f in our actual definition,
{f (t) | a t b}.
However, it seems best to keep our definition, which provides the extra information of a
parameter t for the curve. It might be better to call the function f a parametrized curve, but
that just seems too cumbersome.

Chapter 2

Now we turn to the basic definition which introduces calculus in this context.
f

DEFINITION. Let R Rm . Then f is differentiable at t0 if the following limit exists:

f 0 (t0 ) =

f (t) f (t0 )
df
(t0 ) = lim
.
tt0 ,t6=t0
dt
t t0

In terms of coordinates for Rm the result is like that for continuity, namely, f is differentiable
all the coordinate functions are differentiable. Moreover,

d
df1
dfm
(f1 , . . . , fm ) =
,...,
.
dt
dt
dt
We also say f is differentiable if it is differentiable at t0 for every t0 . This coordinatewise
calculation of the derivative f 0 (t0 ) is valid because of the corresponding theorem we proved
above.
A helpful way to visualize f 0 (t0 ) is to draw the arrow from 0 to f 0 (t0 ). In drawing this
picture we like to position the vector f 0 (t0 ) so that its tail is at f (t0 ):

f ( t 0 )

f ( t0 )
O

We then employ the phrase tangent vector at t = t0 . This nice geometrical picture is
connected with the finite picture of the secant vector
f (t) f (t0 )
:
t t0

f (t )
f ( t0 )

Differentiation

Thus the tangent vector at t = t0 is the limit of the secant vector as t t0 .

EXAMPLE. f (t) = cos 2t , sin 2t . Here f 0 (0) = (0, 12 ).

(1, 0)

PROBLEM 24. Consider the circle in R2 described as

f (t) = (a1 + r cos t, a2 + r sin t) .
This is a parametrization of what we have denoted by S(a, r) in Section 1F. Here 6= 0
is a real constant. Prove that
f 0 (t) (f (t) a) = 0.
What is the geometrical interpretation of this result?
EXAMPLE. f (t) = (t2 , t3 ). Here f 0 (1) = (2, 3).

(1, 1)

Notice that in this example we have f 0 (0) = (0, 0). In a sense this explains why the image
in R2 has a nonsmooth appearance at the origin though the curve f is differentiable.
PROBLEM 25. Let f (t) = (t2 , t|t|) for < t < . Show that f is differentiable
and sketch its image in R2 .

Chapter 2

PROBLEM 26. Consider the figure 8 curve given as the set of points (x, y) R2
which satisfy the equation
(x2 + y 2 )2 = x2 y 2 .
a. Sketch this set reasonably accurately (you might use the corresponding polar coordinate equation r2 = cos 2).
b. Show that the curve

f (t) =

cos t
sin t cos t
2 ,
1 + sin t 1 + sin2 t

0 t < 2,

is a parametrization with the feature that f (s) = f (t) s = t or s = 2 , t =

s = 3
, t = 2 .
2


= (0, 0). Show that
c. From (b) we have f 2 = f 3
2

3
2

or

1 1
f
=
,
,
2
2 2

3
1 1
0
f
=
,
.
2
2 2
0

d. Conclude that f is differentiable at every t, and that it provides two distinct tangent
vectors at the geometric point (0, 0) on the original figure 8 curve.

PROBLEM 27.
Sketch the set of points in R2 described by the equation
y 2 = x2 (x + 1). Show that the curve
f (t) = (t2 1, t3 t),

< t < ,

gives all points on the given set. Indicate on your sketch the three tangent vectors f 0 (0),
f 0 (1), and f 0 (1).
We close this section with the following useful observation. There is a useful theorem in
single-variable calculus which asserts that differentiability = continuity. The same result is
valid in the present context, and the same proof applies:

Differentiation
f

THEOREM. If R Rm is differentiable at t0 , then f is continuous at t0 .

PROOF. We use the fact that the limit of a product equals the product of the limits. Thus
f (t) f (t0 )
tt0
t t0
f (t) f (t0 )
= lim (t t0 ) lim
tt0
tt0
t t0
0
= 0f (t0 )
= 0.

lim (f (t) f (t0 )) = lim (t t0 )

tt0

QED
B. Lengths of curves
f

We continue with our discussion of the special case R Rm . We assume that f is

differentiable.
KINEMATIC TERMINOLOGY. We often think of the independent variable t as time.
Then the derivative gives the important quantities
f 0 (t) = velocity (vector) of the curve at time t,
kf 0 (t)k = speed of the curve at time t.
In terms of the coordinate functions, the speed is
p
0 )2 .
(f10 )2 + (f20 )2 + + (fm
Taking our cue from the basic fact that distance = speed time, we next define the length
f
of the curve [a, b] Rm to be the following:
f

DEFINITION. Assume that the curve [a, b] Rm is differentiable. Then its length is the
definite integral
Z b
kf 0 (t)kdt,
a

provided that this integral exists.

We are not going to discuss in any detail the issue of the existence of this integral. The
quantity kf 0 (t)k might not be a continuous function of t, and in fact it might happen that the

Chapter 2

integral assigns the value to the length. Examples of this behavior are easily found. Here
is one:
PROBLEM 28. Let f be the curve in R2 defined by
(
(t2 cos t2 , t2 sin t2 ) for 0 < t 1,
f (t) =
(0, 0)
for t = 0.
a. Prove that f is differentiable (even at t = 0).
b. Prove that

kf 0 (t)k = 2 t2 + t2 .

c. Prove that

kf 0 (t)kdt = .

(HINT: use a lower bound for the integrand.)

(Incidentally, the existence of the integral has nothing to do with whether we can evaluate
it in closed form. In fact, lengths of curves are usually very difficult to calculate, because of
the square root involved in computing kf 0 (t)k.)
REMARKS. Most curves that actually arise in calculus are piecewise continuously differentiable. This means that there is a partition of the parameter interval a t0 < t1 < < tk = b
such that f is differentiable on each closed interval [ti1 , ti ] and f 0 is a continuous function
there. Strictly speaking, f itself is not necessarily differentiable at the points ti , but this
causes no problem with computing the above integral. Thus we think of curves which may
have corners, but for which the tangent vectors have limits as we approach the corners (but
the limits may be different from one another as t ti from t < ti or from t > ti ). The formula
for length in this case is then given as the sum of the lengths of the pieces,
k Z
X
i=1

ti

kf 0 (t)kdt.

ti1

This is really the same as the original definition in view of the fact that the integral from a
to b is independent of the values the integrand takes (or does not have) at the finitely many
points t1 , . . . , tk1 .

Differentiation

EXAMPLE. A circle of radius R is described parametrically by

f (t) = (R cos t, R sin t),

0 t 2.

The velocity is f 0 (t) = (R sin t, R cos t) and the speed is therefore kf 0 (t)k = R. Thus the
length of this circle is
Z 2
Rdt = 2R.
0

There is a related way to view the definition of length of a curve. Namely, think of a
polygon inscribed in the given curve. Such a polygon may be defined by choosing a
partition a = t0 < t1 < < tk = b of the parameter interval and using the line segments
[f (ti1 ), f (ti )], 1 i k, to approximate the arc. Then the length of this polygon is
k
X

kf (ti ) f (ti1 )k.

i=1

By the way, the polygon is an example of the piecewise continuously differentiable curves we
discussed above, and Problem 210 below shows that the sum given here is indeed its length.

f (b )

f (a )

Since f is differentiable, we know that the norm

kf (ti ) f (ti1 )k
ti ti1
is as close as we please to kf 0 (ti1 )k, provided ti ti1 is sufficiently small. Thus the length
kf (ti )f (ti1 )k is very well approximated by kf 0 (ti1 )k(ti ti1 ). Thus we expect the Riemann
sum
k
X
kf 0 (ti1 )k(ti ti1 )
i=1

10

Chapter 2

to be a good approximation to the length of the polygon. On the other hand, the length of
the polygon should be a good approximation of the length of the curve, so that the Riemann
sum should be a good approximation of the length.
All of the above can be made rigorous, if needed, with some moderate hypothesis on f .
However, as we are giving a definition of length, we choose not to pause to give the proof.
In fact, the idea of looking at inscribed polygons can be made into a definition of length
which doesnt even mention the derivative at all. Suppose f : [a, b] Rm is any curve (still
required to be continuous). Then we can define
length of f = sup{L},
where {L} stands for the set of numbers formed by all possible lengths L of polygons inscribed
in f . It could happen that the length of f is ; in case it is finite we say that f is rectifiable.
It is then a theorem that if f is piecewise continuously differentiable, then f is rectifiable and
the two definitions of length produce the same number.
PROBLEM 29. Find the length of the curve f (t) = (t2 , t3 ) for 1 t 0.
3/2
[Answer: 13 27 8 ]
PROBLEM 210.
The curve f (t) = x + t(y x), 0 t 1, represents the line
segment from x to y. Check that its length is ky xk.

PROBLEM 211. Find the length of the helix in R3 given by

f (t) = (R cos t, R sin t, at),

0 t 2.

PROBLEM 212. Find the length of the parabolic arch in R2 described by f (t) =
(t, a2 t2 ), a t a.

PROBLEM 213. Find the length of the exponential curve in R2 given as f (t) = (t, et ),
0 t 1.

Differentiation

PROBLEM 214.
(log t, t), 1 t e.

11
Find the length of the logarithmic curve in R2 given as f (t) =

PROBLEM 215. A curve in R2 called a cycloid is described by f (t) = (t sin t, 1

cos t), 0 t 2. Find its length.

PROBLEM 216.
A hypocycloid in R2 is described as the set of all points (x, y)
2/3
2/3
satisfying x + y
= 1. Draw a sketch of this set. Define the associated curve by
3
3
f (t) = (cos t, sin t) for 0 t 2, and compute its length.

PROBLEM 217. A curve in R3 is described by f (t) = (t sin t, 1 cos t, 4 sin 21 t).

Show that its speed is 2.

PROBLEM 218. The preceding problem is an example of a curve invented just so

its length can be calculated easily. Choose the constant a just right to render the length
of the curve f (t) = (t, t2 , at3 ) easily computable.

PROBLEM 219.
a t a.

Find the length of the catenary described as f (t) = (t, cosh t),

It is quite important to realize and exploit the fact that the length of a curve is unchanged
if a reasonable change of the independent variable is made. We now explain and prove this
f
feature. We suppose that [a, b] Rm is the curve, so a t b. We also suppose that another
parameter s is to be used, c s d. And we suppose these are related by a differentiable
function , so that t = (s). We further suppose that is increasing:

12

Chapter 2

s
c

Then we have, strictly speaking, a different curve g given by the composition

g = f ; that is , g(s) = f ((s)).
Now we begin to compute the length of g. We first notice the consequence of the chain rule,
g 0 (s) = f 0 ((s)) 0 (s).

vector

vector

scalar

This is a consequence of the chain rule of single-variable calculus, and is proved by simply
writing down the corresponding equation for each coordinate function of g. (Notice that we
have written vector times scalar on the right side its the same product as the usual scalar
times vector.) Since 0 0, the norms are related by
kg 0 (s)k = kf 0 ((s))k0 (s).
Thus we have
Z

length of g =
Z

kg 0 (s)kds

c
d

kf 0 ((s))k0 (s)ds

t=(s)

kf 0 (t)kdt

= length of f.

13

Differentiation

In case the orientation is reversed, so that 0 0, we get the same result:

b

kf 0 ((s))k(0 (s))ds
c
Z a
Z b
0
=
kf (t)k(dt) =
kf 0 (t)kdt.

kg (s)kds =
c

Because of this invariance, if we deal with a set of points in Rm that is clearly equal
to the image of some curve in a one-to-one fashion, we say that its length is the length of
the corresponding curve. Thus, we have no qualms about saying the length of the circle
S(a, r) R2 is 2r, even though we havent displayed a parametrization of S(a, r).
For instance, the length of a graph y = F (x), a x b, in R2 is given by the usual
formula
y

1+
a

dF
dx

2
dx.

x
a

This is seen by using the parametrization

x (x, F (x)) .
There are various calculations we need to perform with this derivative of functions of one
real variable. The chain rule, which we have already seen, is quite important. Other important

14

Chapter 2

ones are versions of the product rule:

Scalar Times Vector : (hf )0 = hf 0 + h0 f,
Vector Dot Vector : (f g)0 = f g 0 + f 0 g.

PROBLEM 220. Prove the two versions of the product rule.

PROBLEM 221. Suppose a curve in Rm lies on a sphere. That is, f (t) S(a, r) for
all t. Prove that the velocity f 0 (t) is tangent to the sphere and the acceleration vector
satisfies (f (t) a) f 00 (t) 0. What is the kinematic interpretation of that inequality?
A special case of the latter version of the product rule is frequently of great use: for a
curve in Rm
d
kf k2 = 2f f 0 .
dt
f

A nice kinematic fact follows from this. If a curve R Rm has an acceleration f 00 which
exists, and if it has constant speed , then its acceleration is orthogonal to the curve. The proof
is easy: we apply the above formula to f 0 rather than f and use the fact that kf 0 k2 is constant.
Thus
d 0 2
kf k
dt
= 2f 0 f 00 .

0=

That is, f 00 is orthogonal to the tangent vector f 0 .

f

PROBLEM 222. Let R Rm be a differentiable curve and assume x Rm is a

point which is not on the curve. Suppose f (t0 ) is a point on the curve which is closest
to x: that is,
kf (t0 ) xk kf (t) xk
for all t.
f ( t0 )

Prove that

f (t0 ) x is orthogonal to f 0 (t0 ).

15

Differentiation

PROBLEM 223. Consider the parabola y = x2 in R2 . Let < a < and find
the point(s) on the parabola which is (are) closest to (0, a).
[Careful: you should discover two cases.]
PROBLEM 224. For any number 0 a < 2 define the curve fa in R4 by
fa (t) = (cos t, sin t, cos(t + a), sin(t + a)),

0 t 2.

a. Show that for each fixed a the image {fa (t)|0 t < 2} is a circle Ca which lies in
a certain two-dimensional plane in R4 .

b. Show that each Ca has center 0 and radius 2.

c. Show that if a 6= b, then Ca Cb = .

C. Directional derivatives
As we have just observed, it is very easy to develop calculus for vector-valued functions of
one real variable. We now turn to the much more intriguing situation of functions of several
f
variables, Rn Rm with n > 1. For our first look at this situation we shall set things up to
use the n = 1 case in a significant way.
Though we are facing a situation here that we may never have seen, something significant
comes to mind. Namely, we could view the function values f (x1 , x2 , . . . , xn ) as depending on
the single real variable xi if we just regard all the other independent variables x1 , . . . , xi1 ,
xi+1 , . . . , xn as fixed. Then we can perform ordinary differentiation with respect to xi . The
result of this differentiation could be denoted in the usual way as
df
,
dxi
but the universally accepted and time-honored notation is instead
f
.
xi
Here then is the actual
f

DEFINITION. Let Rn Rm , and let 1 i n. Then the partial derivative of f with

16

Chapter 2

respect to xi is

f
f (x1 , . . . , xi1 , xi + t, xi+1 , . . . , xn ) f (x)
= lim
t0
xi
t
(provided the limit exists).
f

Notice that if Rn Rm , then f /xi is a vector in Rm .

EXAMPLES.
x1 x2
(e
)
x1

(y sin x)
x

(y sin x)
y
y
(x )
x
y
(x )
y

= x2 ex1 x2 ;
= y cos x;
= sin x;
= yxy1 ;
= xy log x.

So we now have the concept of partial differentiation as being ordinary differentiation

in coordinate directions. So far, so good, but we can do much better. After all, why be
restricted to coordinate directions only? Why not investigate all directions in Rn ? We now
explore this vast generalization, which will indeed free us from a coordinate system entirely.
Assume x Rn is fixed, and assume f is defined at least in a neighborhood of x, say a
small closed ball B(x, r).
We then consider a vector h Rn which will serve as a direction. This means we look at
the line through x in that direction, parametrized as the set of points x + th, < t < .
We then restrict attention to the behavior of f on this line. That is, we consider the function
of t given as f (x + th). This function is defined at least for all sufficiently small |t|. We then
compute the t-derivative of this function at t = 0, if it exists.
DEFINITION. The directional derivative of f at x in the direction h is

f (x + th) f (x)
d

= lim
f (x + th)
.
t0
dt
t
t=0
We shall use the notation
Df (x; h)

17

Differentiation
f

for this limit. Notice that if Rn Rm , then the directional derivative Df (x; h) Rm .
We stress that we have restricted our attention to an arbitrary straight line through x and
have thus been enabled to use differentiation for a function of a single real variable. Notice
that if h = 0 then we are not dealing with a straight line at all, but instead f (x + t0) = f (x)
is constant with respect to t and our definition yields

Df (x; 0) = 0.

PROBLEM 225. Prove that for any scalar a,

Df (x; ah) = aDf (x; h).
(Notice that this result implies Df (x; 0) = 0.)
Directional derivatives are not very interesting for functions of a single real variable, as
all the directions just lie on R. The directional derivative is just a scalar multiple of the
ordinary derivative f 0 :
f

PROBLEM 226. In the special case of R Rn , show that for any h R

Df (x; h) = hf 0 (x).

PROBLEM 227. Let R2 R be given by

f (x1 , x2 ) = (x1 + x2 )ex1 x2 .
Calculate the directional derivatives
Df ((1, 1); h) = 3h1 h2 ,
Df ((1, 0); h) = 2eh1 .

18

Chapter 2

PROBLEM 228. Let R2 R2 be given by

f (r, ) = (r cos , r sin ).
Calculate
Df ((1, ); h) = (h1 cos h2 sin , h1 sin + h2 cos ).

PROBLEM 229. Let R2 R2 be given by

p
y
2
2
f (x, y) =
x + y , arctan
.
x
Calculate
Df ((1, 0); h) = h.

PROBLEM 230. Let Rn R be given by

f (x) = u x + a,
where u Rn and a R are constants. Calculate
Df (x; h) = u h.

The next two problems give directional derivative versions of the product rule.
f

PROBLEM 231. Suppose Rn Rm and Rn R. Prove that

D(gf )(x; h) = g(x)Df (x; h) + Dg(x; h)f (x).

19

Differentiation

PROBLEM 232. Suppose Rn Rm and Rn Rm . Prove that

D(g f )(x; h) = g(x) Df (x; h) + Dg(x; h) f (x).

There is also a version of the chain rule:

PROBLEM 233. Suppose that Rn R R and that g is a differentiable function.

The composite function g f is defined by the equation g f (x) = g(f (x)). Prove that
D(g f )(x; h) = g 0 (f (x))Df (x; h).

PROBLEM 234. Let f (x) = kxk2 . Calculate

Df (x; h) = 2x h.

PROBLEM 235. Combine the two preceding problems to show that for any real
number and any x Rn with x 6= 0,
D(kxk )(x; h) = kxk2 x h.
(In particular,
D(kxk)(x; h) =

xh
.)
kxk

20

Chapter 2

PROBLEM 236. Let Rn Rn be given by

x
.
kxk2

f (x) =
Calculate
Df (x; h) =

h
2x h

x.
2
kxk
kxk4

PARTIAL DERIVATIVES. Still working in the context of directional derivatives, we frequently pay special attention to the unit coordinate directions e1 , e2 , . . . , en . Here each vector
ei Rn is given by
ei = (0, . . . , 0, 1, 0, . . . , 0),
where the single 1 appears in the ith position.
PROBLEM 237. Show that
Df (x; ei ) =

f
.
xi

We also frequently use a special notation:

Di f (x) = Df (x; ei ) =

f
.
xi

The notation Di f (x) has the advantage over f /xi of not having to name the coordinates a
special way. We just have to keep track of the order in which they are written. For instance,
if f (m, p, a) = am2 p3 , then D2 f = 3am2 p2 .
Still another useful special notation represents partial derivatives with subscripts, so that
fxi =

f
.
xi

PROBLEM 238. For the function of Problem 229 show that

!
f
y
x
= p
.
, p
y
x2 + y 2
x2 + y 2

21

Differentiation

Incidentally, in physics and engineering a special notation for the unit coordinate vectors
in R3 is in vogue:
i = (1, 0, 0),
= (0, 1, 0),
k = (0, 0, 1).
In fact, in physics special attention is paid to unit vectors (vectors with norm 1), in
that each such vector is dubbed with a circumflex ( ). Thus it is correct in R4 to write
whereas it is incorrect in R2 to write h = (1, 1) as h.
We shall often
h = ( 12 , 12 , 12 , 12 ) as h,
you are assured that the norm
employ this notation. Thus if you see a symbol Df (x; h),
= 1.
khk
D. Pathology
It is good for me that I have been afflicted, that I might learn thy statutes
Psalm 11971
The purpose of this entire section is to present some examples of functions which have
directional derivatives with certain strange properties. (Such examples are often called counterexamples.) The reason for doing this is not my own love for the perverse, but rather
to make sure we fully appreciate the tremendous usefulness of the concept of differentiability which will be discussed in the next section. These examples also serve to illustrate the
inadequacy of the concept of directional differentiation, however appealing and useful it is.
All our examples are going to require that n > 1, and it so happens that n = 2 gives us
enough room for the strange behavior we want to illustrate. Therefore, in this entire section
we deal with
f
R2 R,
and we denote points in R2 with the notation (x, y). We still denote the directions as h =
(h1 , h2 ). We shall also arrange things so that the pathology occurs at the origin in all cases.
Before going on, notice that all the examples and problems in Section C had the feature
that the directional derivatives were linear functions of h. Well have much more to say about
this in the next section, but for now we just note that a linear function of h is a function of
the form c1 h1 + c2 h2 , where c1 and c2 are constants.
QUESTION 1. Is it possible that a continuous function have a directional derivative which is a nonlinear function of the direction?
ANSWER. Yes!

22

Chapter 2

EXAMPLE.

3
f (x, y) = x1/3 + y 1/3 .

Continuity is clear. And it is easily seen that

3
1/3
1/3
Df (0; h) = h1 + h2
. . . a nonlinear function of h.
Incidentally, notice for example that
9
Df ((1, 8); h) = 9h1 + h2 is a linear function of h.
4
Heres another:

(
f (x, y) =

x2 y
x2 +y 2

if (x, y) 6= (0, 0),

if (x, y) = (0, 0).

Continuity is clear except at the origin. But notice that

|f (x, y)| =

x2 |y|
|y|,
x2 + y 2

so f is continuous at 0. (Also, |f (x, y)| 12 |x|.)

Now we compute
f (th) f (0)
=
t

t3 h21 h2
t2 h21 +t2 h22

t
h21 h2
.
= 2
h1 + h22

This doesnt even depend on t, so certainly for h 6= 0

Df (0; h) =

h21 h2
. . . a nonlinear function of h.
h21 + h22

QUESTION 2. Is it possible that a function have directional derivatives in every

direction and not be continous?
ANSWER. Yes!
EXAMPLE.

(
f (x, y) =

x2 y
x4 +y 2

for (x, y) 6= (0, 0),

for (x, y) = (0, 0).

23

Differentiation

PROBLEM 239. Show that f is discontinuous at the origin. Show that Df (0; h)
exists for all h and is given by
(
h21 /h2 if h2 6= 0,
Df (0; h) =
0
if h2 = 0.
One might think that perhaps the trouble with the preceding example is that the directional
derivative is nonlinear. Heres a somewhat more sophisticated counterexample.
QUESTION 3. Is it possible that a function have directional derivative equal to
zero in every direction and not be continous?
ANSWER. Yes!
EXAMPLE.
Let

(
f (x, y) =

x5 y
x8 +y 4

for (x, y) 6= 0,
for (x, y) = 0.

PROBLEM 240. Verify that f satisfies the conditions we have asserted for it.
MORAL. Unlike single-variable calculus, existence of directional derivatives does not imply
continuity of the function. More subtly, the directional derivative is not necessarily a linear
function of the direction. We shall soon discover how wonderful it is for the directional
derivative to depend linearly on the direction, so we shall incorporate this property into the
definition in the following section.
E. Differentiability of real-valued functions
At last we turn to the actual definition we shall employ. First, we need the important
definition of linearity. We shall discuss this thoroughly in Section I, but for now we give a
PROVISIONAL DEFINITION. A linear function from Rn to R is a function L of the
form
L(h) = c1 h1 + + cn hn ,
where the numbers c1 , . . . , cn are constants. Assembling the coefficients c1 , . . . , cn as the coordinates of a vector c Rn , we can use our scalar product notation to write L in the form
L(h) = c h.

24

Chapter 2

This probably doesnt quite agree with your usual terminology for linear functions. Here
are two graphs of functions from R to R:

not linear

linear

A function from R to R of the form f (x) = ax + b is said to be affine, and is therefore

linear b = 0. More generally, a function from Rn to R of the form f (h) = c h + d
is said to be affine. Thus every linear function is affine, and an affine function f is linear
f (0) = 0.
The next thing is to remind ourselves of the definition of differentiability of a function from
R to R. It is that the limit
f (x + y) f (x)
c = lim
y0
y
exists. Rewrite: there exists a number c such that
f (x + y) f (x) cy
= 0.
y0
y
lim

Heres a slight modification. Replace the denominator by |y| (since the limit is 0, this doesnt
change the definition):

(To repeat: replacing the denominator y by its absolute value does not change the value 0
of the limit.)
This is perfect! Notice the linear function in the numerator! Now that we have transformed
the definition cleverly, we can immediately generalize to functions on Rn , as follows.

25

Differentiation
f

F DEFINITION. F Let x Rn be a fixed point. Assume Rn R is defined at

least in a neighborhood of x (a ball B(x, r)). Then f is differentiable at x if there
exists a vector c Rn such that
f (x + y) f (x) c y
= 0.
y0
kyk
lim

REMARK. This definition is absolutely crucial. Notice the vast difference between it and
the concept of directional derivative. There is nothing directional in this definition the
variable point y tends to 0 in norm (which is true all coordinates of y tend to 0) with no
restriction on its direction.
REMARK. There is a nice geometric interpretation of this definition. In the case of a
function from R to R, the existence of f 0 (x) has the familiar tangent line interpretation,
namely that the graph of f near the point x looks affine on the microscopic scale. That is,
f (x) + f 0 (x)y is a very good approximation to f (x + y) for small y. For instance, here are
three sketches of the graph of x x2 near x = 0:

1/2

1/4

1/16

The same sort of thing is true in our case: the affine function of y given by f (x) + c y is a
very good approximation to the function f (x + y) for small kyk. The difference between the
given function and the affine function tends to zero as kyk 0, and it does so at a faster rate
than kyk itself: the quotient of the two even tends to zero.
Heres an easy fact:
if f is differentiable at x, then the directional derivatives exist at x.
Let y be restricted to have the form th in order to prove this (assuming (h 6= 0). Then the
differentiability of f at x implies that
f (x + th) f (x) c th
.
t0
|t| khk

0 = lim

26

Chapter 2

Multiply by the number khk and multiply by |t|/t:

f (x + th) f (x) tc h
t0
t
f (x + th) f (x)
= lim
c h.
t0
t

0 = lim

Thus, Df (x; h) = c h. In particular, if h = ei , then the ith component of c is

ci =

f
(x).
xi

(Thus c is uniquely determined by f .)

DEFINITION. The vector c is called the gradient of f at x and is written with two different
notations:

f
f
(gradf )(x) = (f )(x) =
,...,
.
x1
xn
Thus we have shown that if f is differentiable at x, then
Df (x; h) = f (x) h.
In particular, notice the very pleasant situation that Df (x; h) is a linear function of h.
The notations we have chosen for the gradient are quite standard. The symbol , an
upside down delta, is called del. It also has the rather obsolete name nabla, and its properties
are still sometimes called the nabla calculus.
We can now easily state some expected calculus formulas for the gradient. We assume that
f and g are real-valued functions defined on subsets of Rn . Then we have
(f + g) = f + g;
(af ) = af if a R is constant;
f = 0 if f is constant;
(f g) = f g + gf.
We do not treat the chain rule at the present time, as we shall discuss it thoroughly in
Section K.
PROBLEM 241. Prove the above formulas. In addition, state and prove the corresponding quotient rule.

27

Differentiation

PROBLEM 242.
a. Let f be an affine function: f (x) = c x + d. Show that f is differentiable at any
x, and (f )(x) = c.
b. Let f be the quadratic function: f (x) = kxk2 . Show that f is differentiable at any
x, and (f )(x) = 2x. Thus
(c x + d) = c,
(kxk2 ) = 2x.
(Solution: (a) f (x + y) f (x) c y = c (x + y) + d c x d c y = 0. Thus
lim

y0

f (x + y) f (x) c y
= 0.
kyk

(b) f (x + y) = kx + yk2 = kxk2 + 2x y + kyk2 = f (x) + 2x y + kyk2 . Thus

f (x + y) f (x) 2x y
= kyk 0 as y 0.
kyk
Thus (f )(x) = 2x.)
f

We now quickly show that just as in the special case of Section B (R Rm ), differentiability = continuity:
THEOREM. If f is differentiable at x, then f is continuous at x.
PROOF. This is tres simple: consider the two limits,
f (x + y) f (x) c y
= 0,
y0
kyk
lim kyk = 0.
lim

y0

Multiply them, using the fact that the limit of a product is the product of the limits:

lim f (x + y) f (x) c y = 0.
y0

But of course c y has limit zero, so we conclude

lim f (x + y) f (x) = 0;
y0

28

Chapter 2

i.e.,
lim f (x + y) = f (x).

y0

QED
We are surely elated that in the case of differentiability, Df (x; h) is indeed a linear function
of h. However, the converse is not valid, as the counterexample for Question 3 had Df (0; h) = 0
for all h (thus linear) and yet f was not even continuous at 0; and we now know that f could
certainly therefore not be differentiable at 0.
You may say, Aha! Suppose we assume that Df (x; h) exists and is a linear function of h
and f is continuous at x. Then perhaps f is differentiable at x:
QUESTION 4. Is it possible that a continuous function with Df (0; h) = 0 for all h
not be differentiable at 0?
ANSWER. Yes!
Heres an example:
PROBLEM 243. Let
(
f (x, y) =

x3 y
x4 +y 2

for (x, y) 6= (0, 0),

for (x, y) = (0, 0).

a. Show that f is continuous at (0, 0).

b. Show that Df (0; h) = 0 for all h.
c. Show that f is not differentiable at 0.
(The hard part is c. Heres a proof by contradiction. If f were differentiable at 0, show
that it would follow that f (0) = 0. Conclude that
f (x, y)
lim p
= 0.
(x,y)0
x2 + y 2
Now show that this is not true.)
Therefore our necessary conditions that f be differentiable at x: (1) f is continuous at
x and (2) Df (x; h) is a linear function of h, turn out not to be sufficient to ensure the
differentiability.

29

Differentiation

PROBLEM 244. As might be expected, the fourth calculus formula given above, the
product rule, is more complicated to prove than the other easy calculus rules. Here is a
lemma which will be useful in giving a proof: show that if f is differentiable at x, then
there exists a constant C such that
|f (x + y) f (x)| Ckyk
for all sufficiently small y Rn .

The above inequality is called a Lipschitz condition for f at the point x. It is named for
the German mathematician Rudolf Otto Sigismund Lipschitz. Notice that the inequality gives
another proof of the continuity of a differentiable function.

Now it is easy to prove the product rule:

PROBLEM 245. Prove that if f and g are differentiable at x, then the product f g
is differentiable at x, and
(f g)(x) = f (x)(g)(x) + g(x)(f )(x).
(HINT: write f = f1 + f (x) and g = g1 + g(x) (x is a fixed point). Express f g as a sum
of four products and use the preceding problem to show that (f1 g1 )(x) = 0.)

30

Chapter 2

PROBLEM 246. What is wrong with this proof for the preceding problem?
a. We know
f (x) = (f /x1 , . . . , f /xn ),
g(x) = (g/x1 , . . . , g/xn ).
b. We also know that

g
f
(f g) = f
+g
.
xi
xi
xi

c. Therefore,
(f g) = ((f g)/x1 , . . . , (f g)/xn )
= (f g/x1 + gf /x1 , . . . )
= f (g/x1 , . . . ) + g(f /x1 , . . . )
= f g + gf.
QED?
We have now made significant progress in our understanding of multivariable calculus,
thanks to the all-important definition of differentiability. In the next section we shall learn
how it can be quite useful.
F. Sufficient condition for differentiability
The property of differentiability is absolutely crucial in multivariable calculus. The definition is so technically involved that it is hard to verify directly. Happily, there is a sufficient
condition that is extremely useful, and handles almost all cases that we ever need. We now
state and prove it.
f

F THEOREM. F Assume Rn R is defined in a neighborhood of x, and assume

the partial derivatives f /x1 , . . . , f /xn all exist in this neighborhood. Furthermore, assume these partial derivatives are all continuous at x. Then f is differentiable at x.
PROOF. The main tool we shall employ in this proof is the famous mean value theorem of
single-variable calculus. This theorem asserts (under the proper hypothesis) that for <
a<b<
g(b) g(a) = g 0 (c)(b a),
where c is some point in the interval a < c < b.

31

Differentiation

To verify the differentiability of f at x, we need to compare f (x + y) and f (x). We do this

by moving from x to x + y along n steps taken one coordinate at a time (a taxicab trip).
Since there is nothing essentially different between the cases Rn and R2 , I am content for the
sake of brevity to perform the proof for the case n = 2 only.
We use the notation c = f (x) = (f /x1 , f /x2 ). (Of course, the hypothesis guarantees
the existence of these partial derivatives.) Then for any > 0 there exists > 0 such that

f
kyk < =
(x + y) ci
xi

< /2;

this is where we use the hypothesis of continuity of f /xi at x.

Now we write in two steps
f (x + y) f (x) = [f (x1 + y1 , x2 + y2 ) f (x1 , x2 + y2 )] + [f (x1 , x2 + y2 ) f (x1 , x2 )] .

This is perfectly arranged to use the mean value theorem on each term to produce for any
kyk < ,
f (x + y) f (x) =

f
f
(x1 + z, x2 + y2 ) y1 +
(x1 , x2 + w) y2 ,
x1
x2

32

Chapter 2

where z is between 0 and y1 and w is between 0 and y2 . Therefore,

f
f

(x1 + z, x2 + y2 ) c1 y1 +
(x1 , x2 + w) c2 y2
|f (x + y) f (x) c y| =
x1
x2

1
1
|y1 | + |y2 |
2
2
1
1
kyk + kyk
2
2
= kyk.
Therefore,
0 < kyk < =

|f (x + y) f (x) c y|
.
kyk

Thus we have proved that

f (x + y) f (x) c y
= 0.
y0
kyk
lim

Therefore, f is differentiable at x (and c = f (x)).

QED
The theorem we have just proved is quite wonderful, as is its proof. This is just the
sort of mathematical proof that essentially works itself. Once we decided to use the partial
derivatives, then the diagram in the body of the proof suggests itself, as does the use of the
mean value theorem.
Whereas the next problem holds no interest for calculus that I am aware of, working
through it may enhance your understanding of the above important proof. It is an improvement of the theorem to be sure, as it provides the same conclusion with weaker hypotheses.
f

PROBLEM 247. Assume Rn R is defined in a neighborhood of x, and assume the

partial derivatives f /x1 , . . . , f /xn1 all exist in this neighborhood and are continuous
at x. Assume also that f /xn exists at x. Prove that f is differentiable at x.
(HINT: limit attention to n = 2 and apply the mean value theorem only to the term
f (x1 + y1 , x2 + y2 ) f (x1 , x2 + y2 ) in the above proof.)
It so happens that in practice an even weaker theorem than the one we have proved is
definitive in an amazing variety of situations:
f

COROLLARY. Assume Rn R is defined in an open ball B(x, r), and that

the partial derivatives f /x1 , . . . , f /xn all exist and are continuous functions in
B(x, r). Then f is differentiable at every point of B(x, r).

33

Differentiation

This result is an immediate corollary of the theorem. The reason it is so useful is that we
can very often tell at a glance that a function has these continuous partial derivatives, and
thus it must be differentiable. For instance,

f (x1 , x2 , x3 , x4 ) = log x21 + x42 + x63 + ex1 x4 + sin x3 cos(sin x1 )

gives a function f that clearly satisfies the conditions on all of R4 ; for all we need do is the
mental exercise of thinking about how to compute the four partial derivatives f /xi . For
that we use the full power of one-variable calculus the chain rule, the product rule etc.
and realize that the formulas we thereby obtain give continuous functions on R4 . The
only possible trouble that could arise in this particular example would occur because the
argument of log might be 0. As this argument is clearly always positive, that cannot happen.
We conclude that each f /xi is continuous and thus that f is differentiable at every point
of R4 .
EXAMPLE. Heres a reworking of Problem 227:
f (x1 , x2 ) = (x1 + x2 )ex1 x2
We see at a glance that f satisfies the conditions of the corollary. If we want the directional
derivative at any x R2 , we can use the formula on p. 226:
f
= (1 + x1 + x2 )ex1 x2 ,
x1
f
= (1 x1 x2 )ex1 x2 .
x2
Thus
Df (x; h) = [(1 + x1 + x2 ) h1 + (1 x1 x2 ) h2 ] ex1 x2 .
No difference quotients needed! No need to substitute x + th and compute the t-derivative
and substitute t = 0. Its just algebra!
DEFINITION. A function which satisfies the hypothesis of the above corollary is said to be
continuously differentiable. It is also said to be of class C 1 . (A continuous function is of class
C 0 .)
Here we give an example which illustrates the power of this corollary. Consider the function
given as the norm, f (x) = kxk for x Rn , x 6= 0. Since f (x) is the square root of the positive
quantity x21 + + x2n , the chain rule of single-variable calculus makes it clear that all the

34

Chapter 2

partial derivatives f /xi are continuous for x 6= 0; thus f is of class C 1 for x 6= 0. We already
have from Problem 235 the directional derivative
xh
Df (x; h) =
.
kxk
Since f is differentiable (by the corollary),
f (x) h = Df (x; h).
Therefore,
f (x) h =

xh
kxk

for all h Rn .

We can now conclude that f (x) = x/kxk. (See Problem 1-11.) We record this result in the
form
x
kxk =
.
kxk
PROBLEM 248. Show that for any real number
kxk = kxk2 x

for x 6= 0.

For which is this equation also valid for x = 0?

There are two common misconceptions concerning differentiability. One is the idea that
our sufficient condition is also necessary. The following example shows this not to be the case
even for single-variable calculus:
f

PROBLEM 249. Define R R by

(
x2
f (x) =
0

if x is irrational,
if x is rational.

Prove the following:

a. f is continuous only at 0.
b. f is differentiable at 0, and f 0 (0) = 0.
c. f is differentiable only at 0.

35

Differentiation

The second misconception is that the mere existence of the partial derivatives in a ball
automatically implies their continuity. This again is not true even for single-variable calculus.
Here is an example found in almost all calculus texts:
f

PROBLEM 250. Define R R by

(
x2 sin x1
f (x) =
0

for x 6= 0,
for x = 0.

Prove that
a. f is differentiable on all of R;
b. f 0 (0) = 0;
c. f 0 (x) is not a continuous function of x at x = 0.

G. A first look at critical points

Well eventually present critical points in great depth, and in fact wont finish the discussion
until Chapter 4, but already we are able to discuss the concept to some extent.
f

DEFINITION. Let Rn R be differentiable at x. Then x is called a critical point of f if

f (x) = 0.
This terminology should already be familiar to you from single-variable calculus. Notice
that the condition for x to be a critical point can be expressed in terms of partial derivatives:
f
(x) = 0 for 1 i n.
xi
Another important concept:
f

DEFINITION. Let Rn R be defined on a set A Rn and let x0 A. We say that f has

a global maximum at x0 if
f (x) f (x0 ) for all x A.
We say that f has a local maximum at x0 if there exists r > 0 such that
f (x) f (x0 ) for all x A B(x0 , r).

36

Chapter 2

(Notice that a global maximum is certainly a local maximum, but not conversely.) We define
similarly global minimum and local minimum. We say that f has a local extreme value at x0
if it has a local maximum or a local minimum at x0 .
As in single-variable calculus, we have the easy
f

THEOREM. Let Rn R be differentiable at x0 and also have a local extreme value at x0 .

Then x0 is a critical point of f . The converse statement does not necessarily hold, even if
n = 1.
PROOF. We are content to handle the case of a local minimum. Then for any h Rn ,
f (x0 + th) f (x0 ) for all sufficiently small |t|.
Therefore for some > 0
f (x0 + th) f (x0 )
is
t

0 for 0 < t < ,

0 for < t < 0.

Now let t 0 to achieve both the inequalities

Df (x0 ; h) 0 and

0.

Therefore, Df (x0 ; h) = 0 for all directions h Rn . In particular, f /xi = 0 at x0 for

1 i n. Thus x0 is a critical point. For the statement about a converse, most peoples
favorite example is f (x) = x3 for x R.
QED
f

EXAMPLE. If R2 R is given by f (x, y) = xy 2 , then f = (y 2 , 2xy). Thus every point of

the form (x, 0) is a critical point. Notice that critical points do not have to be isolated.

EXAMPLE. Suppose we want to find all the critical points of the function given as f (x, y) =
5x2 y +xy 2 +15xy. Then we compute the two partial derivatives to get the two scalar equations
(
10xy + y 2 + 15y = 0,
5x2 + 2xy + 15x = 0.
There are two things to notice before proceeding. Namely, we have two equations for two
unknowns (x and y) and they are nonlinear. This is the usual situation, and usually it will be

37

Differentiation

difficult or impossible to solve the equations explicitly. However, in this particular example
its pretty simple. The equations can be rewritten
(
y(10x + y + 15) = 0,
x(5x + 2y + 15) = 0.
The first equation asserts that
y = 0 or 10x + y = 15,
and the second that
x = 0 or 5x + 2y = 15.
There are four possibilities. The least obvious one is the one in which the two affine equations
are satisfied, and elimination gives the solution x = 1, y = 5. Thus the four critical points
of f are
(0, 0), (0, 15), (3, 0), (1, 5).
The next eleven problems give functions defined on (a subset of) Rn . All of them are of
class C 1 , and thus are differentiable. You are to find all the critical points of each.
PROBLEM 251.
f (x, y) = 3x2 2xy + 3y 2 x2 y 2 .

PROBLEM 252.
f (x, y) =

3x
3y
2+
xy.
y
x

Here f is defined only for x 6= 0, y 6= 0.

PROBLEM 253.
f (x, y) = x2 y 3 (2x + 3y 6).

PROBLEM 254.
f (x, y) =
Here a, b are nonzero constants.

a
b
1
2 2 + 17.
xy x y xy

38

Chapter 2

PROBLEM 255.
f (x, y, z) =

x3 + y 3 + z 3
.
xyz

PROBLEM 256.
f (x, y) = xy + x1 + y 1 .

PROBLEM 257.
f (x, y, z) = xyz + ax1 + by 1 + cz 1 .
Here a, b, c are nonzero constants. There are two qualitatively different cases, depending
on the sign of abc.
PROBLEM 258.
f (x, y, z, w) = xyzw + ax1 + by 1 + cz 1 + dw1 .
Here a, b, c, d are nonzero constants.
PROBLEM 259.
f (x, y) = (y x2 )(y 2x2 ).

PROBLEM 260.

f (x) = (a1 x21 + + an x2n )ekxk .

Here kxk is the norm of x, and the constants satisfy a1 > a2 > > an > 0.
(There are 2n+1 critical points.)
PROBLEM 261.
(There are two critical points.)

2
2
f (x, y) = (x + y)e x +y .

39

Differentiation

PROBLEM 262. The preceding problem brings up an issue.

f

a. Prove that the function Rn R given as the norm, f (x) = kxk, is differentiable at
every x 6= 0, and is not differentiable at x = 0.
f

b. Let Rn R be defined as
f (x) = x1 ekxk .
Prove that f is differentiable even at x = 0, and calculate (f )(0).

PROBLEM 263. Let a be a fixed real number and find the critical points of the
function on Rn defined by f (x) = kxk2 + x1 + akxk. (There will be three cases depending
on what a is.)

EXAMPLE. The critical point structure of this function will prove to be very enlightening.
f
Define R2 R by
f (x, y) = (x2 1)2 + (x2 ey )2 .
As f is given as a sum of two squares, it is clear that f (x, y) 0. Moreover, f (x, y) = 0
x2 = 1 and x2 = ey . Thus f attains its global minimum value precisely at the two points
(1, 0) and (1, 0).
We know these are critical points. Let us see if there are others:
f /x = 4x(x2 1) + 4x(x2 ey ) = 0,
f /y = 2ey (x2 ey ) = 0.
The second of these equations of course requires x2 = ey , and then the first requires 4x(x2 1) =
0. Thus x = 0, 1, or 1. The value x = 0 is excluded by x2 = ey . We then have 1 = ey , so
y = 0. Thus the two points we found by inspection are the only critical points.
MORAL. Situations can be much more complicated in two variables than in one. A differentiable function defined on an interval in R cannot have two global minima unless it has at
least one more critical point (a local maximum), as indicated in the following sketch. But
with two independent variables there is more room to maneuver. We might say that there
doesnt need to be a mountain peak between two lakes.

40

Chapter 2

PROBLEM 264. Define R2 R by

f (x, y) = 3xey x3 e3y .
a. Show that (1, 0) is the only critical point.
b. Show that (1, 0) is a local maximum. (You may want to show first that
f (x, y) 2x3/2 x3 for all x > 0, < y < .)
c. Show that (1, 0) is not a global maximum.
d. Give the moral of this example.

H. Geometric significance of the gradient

In this section we return to the general situation
f

Rn R
in which f is differentiable at the point x. We have defined the gradient f (x) and we know
algebraically how to compute it in terms of partial derivatives,

f
f
,...,
,
f (x) =
x1
xn
but we now want to explore the geometry which is contained in this vector.

41

Differentiation

First, it could be that x is a critical point of f . By definition this means that f (x) = 0
and the geometrical situation is quite clear.
Thus we assume from now on that x is not a critical point; that is,

f (x) 6= 0.

We want to examine thoroughly the equation (p. 226) which relates the directional derivative
and the gradient,
Df (x; h) = f (x) h.

To recapitulate, the definition of Df (x; h) is very geometrical in nature, and so is that of

f (x). We also have a way of computing f almost algebraically, using the coordinates of
Rn and the partial derivatives of f at x. And then computing the directional derivative really
does become a matter of linear algebra:

n
X
f
Df (x; h) =
(x)hi .
xj
i=1

What we are going to do now is add to this situation our geometrical understanding of the
dot product so that we emerge with a geometrical understanding of the gradient.
Therefore we now consider the behavior of f (x) h as the direction of h varies. In order
(see p. 221 for
to make this significant, we shall now work exclusively with unit vectors h
this notation).
in Rn , emanating
Geometrically, this means that we are actually looking at directions h
is a measure of the rate of increase of f at x
from x, and the directional derivative Df (x; h)

in the direction h.
We then recall
Let denote the angle between the gradient f (x) and the direction h.
from p. 117 that

42

Chapter 2

f ( x)
h

Df (x; h) = kf (x)k cos .

= 1.) This equation is very revealing. It shows that the maximum value of
(Remember: khk
is the norm of the gradient kf (x)k, and this maximum rate of increase is realized
Df (x; h)

h is the unit vector in the same direction as f (x).

is kf (x)k, and this occurs h
is the unit vector in
(Also, the minimum of Df (x; h)
the same direction as f (x).)
Not only is this useful geometric information about the directional derivative, but also it
gives us a way to describe f (x) in purely geometric terms, with no reference whatsoever to
a coordinate system! Namely, still assuming x is not a critical point,
f (x) is the unique vector at x in Rn determined as follows:
its direction is the direction of maximal increase of f at x,
its magnitude or norm is the rate of this maximal increase.
This is indeed a wonderful situation, one that frequently happens in mathematics. We
have an important quantity (in this case, the gradient of a function) which, on the one hand,
has a completely geometric description, and which, on the other hand, can be computed in a
coordinate system in a very routine and useful manner (in this case, as (f /x1 , . . . , f /xn )).
Truly a double-edged sword!
f

Level sets. As you may realize, the graph of a function Rn R seems to be less visually
helpful for n > 1 than for n = 1. However, the idea of level sets of f seems quite useful. By
definition, these are sets of the form
{x Rn | f (x) = a},

43

Differentiation

where a is any fixed real number. These sets are obviously disjoint (as a varies) and fill up all
of the domain of f . Sketching them is clearly a different matter from sketching the graph of
f (a subset of Rn+1 ). They seem especially convenient when n = 2.
2

EXAMPLE. Let f (x, y) = x4 + y 2 . The level sets of f are of course ellipses with center (0, 0)
and with the same shape. (Unless a < 0, in which case the level set is empty; or a = 0, in
which case the level set is just (0, 0).) Here are rough sketches of a few level sets, where the
numbers refer to the value f attains along that particular level set.

4 5
3
2
1

I have sketched f (x, y) at three points of the level set f (x, y) = 1.

Notice that f is always orthogonal to the level sets in the above sketch. This feature is
always true, but we need to wait for the discussion of the chain rule in Section K to see this.
Also notice that the more tightly spaced the level sets are, the larger the gradient must be.
This is of course because the norm of the gradient measures the rate of increase of f in that

44

Chapter 2

direction, and when the level sets are close together, f must be changing rapidly. When you
hike in Colorado with a survey map that shows curves of constant altitude, its when they are
close together that your hike is strenuous.
PROBLEM 265.
f
R2 R:

Here is a rough sketch of some level sets of a certain function

Suppose that f is differentiable at the indicated point x0 . Prove that x0 is a critical point
for f .

PROBLEM 266. Let R2 R be differentiable at x0 . Suppose that

1 1
Df x0 ; ,
=1
2 2

!!
1 3
,
= 1.
Df x0 ;
2 2
Calculate (f )(x0 ).

45

Differentiation

PROBLEM 267. Let R4 R be differentiable at x0 . Suppose that

1 1 1 1
Df x0 ;
, , ,
= 0,
2 2 2 2

1 1 1 1
Df x0 ;
, , ,
= 1,
2 2 2 2

1 1 1 1
= 2,
Df x0 ;
, , ,
2 2 2 2

1 1 1 1
Df x0 ;
= 3.
, , ,
2 2 2 2
Calculate (f )(x0 ).
PROBLEM 268. Sketch a few level sets of the function defined on R2 by f (x, y) =
x2 y 2 + 3. Also give accurate sketches of f at several points in R2 .
PROBLEM 269. Repeat Problem 268 but with f (x, y) = y 2 x3 x2 .
I. A little matrix algebra
In the next section we are going to discuss the concept of differentiation for a function
from Rn to Rm . The key concept we shall have to understand is the idea of a linear function
from Rn to Rm . The reason is that the derivative in this general case is intimately connected
to a type of affine approximation to the given function. This is yet another instance of the
intimate connection that calculus provides between algebraic and geometric concepts. In the
present section we want to provide the necessary elementary algebraic structure. We first need
to modify the provisional definition we gave in Section E:
F

DEFINITION. A linear function from Rn to Rm is a function Rn Rm which is compatible

with vector addition and scalar multiplication, in the sense that
F (x + y) = F (x) + F (y),
F (ax) = aF (x),
for all x, y Rn and all a R.

46

Chapter 2

REMARK. Often linear functions are also called linear transformations or linear operators.
PROBLEM 270. Here are some immediate consequences of the definition, showing
further how a linear function respects the algebraic properties of Rn and Rm : prove that
if F is linear, then
F (0) = 0,
F (x) = F (x),
(1)

F a1 x + a2 x(2) + + ak x(k) = a1 F x(1) + a2 F x(2) + + ak F x(k) .

PROBLEM 271. Prove that we could have defined linear function by requiring F to
preserve linear combinations, in the sense that
F (ax + by) = aF (x) + bF (y)
for all x, y Rn and all a, b R.

PROBLEM 272. Show that our provisional definition in the case Rn R, namely
F
that F (x) = c x, gives linear functions. Moreover, prove that if Rn R is linear, then
there exists a unique c Rn such that F (x) = c x.

PROBLEM 273. Just as on p. 220, we need to understand the difference between

F
affine and linear functions. We define Rn Rm to be affine if
F (ax + (1 a)y) = aF (x) + (1 a)F (y)
for all vectors x and y and all scalars a. Prove that F is affine there exists a linear
function F0 and a fixed vector w Rm such that
F (x) = w + F0 (x) for all x.
Prove that F0 and w are uniquely determined by F .

47

Differentiation

PROBLEM 274. Prove that if F is affine, then

F a1 x(1) + a2 x(2) + + ak x(k) = a1 F x(1) + a2 F x(2) + + ak F x(k)

whenever a1 + a2 + + ak = 1.
It is virtually impossible to provide meaningful pictorial descriptions of linear functions.
However, there is a classic bookkeeping device for handling them, namely, the algebra of
matrices.
DEFINITION. An m n matrix of real
and n columns. A standard notation for A

a11
a21

A = ..
.
am1

numbers is a rectangular array A having m rows

is

a12 . . . a1n
a22 . . . a2n

am2 . . . amn

We say that the real number aij is the entry of A in row i and column j. We also say that
the ith row of A is (ai1 ai2 .
. . ain )
(a 1 n matrix)
a1j
a2j

and the j th column of A is ..

(an m 1 matrix).
.
amj
We say that the matrix A has shape m n. If we are in a context in which the shape of
A is known, we may abbreviate
A = (aij ).
Here are some basic algebra definitions:
two matrices A and B are equal they have the same entries at each position (A and
B must therefore have the same shape).
A + B is defined entry-wise (A and B must therefore have the same shape), so that
(aij ) + (bij ) = (aij + bij );
notice that A + B = B + A.

48

Chapter 2

0 is the m n matrix whose entries are all 0; thus

A + 0 = 0 + A = A.

If c R, cA is defined entry-wise by
cA = (caij ).

Notice that the matrix A defined as (1)A is the unique matrix which when added to
A gives the matrix 0.
It is nice to observe that the set of all m n matrices is now additively just like Rmn . Namely,
each m n matrix is specified by its mn real entries, arranged in a certain pattern. The
same is true of each vector in Rmn . Not only that, but also addition of matrices and scalar
multiplication are just like they are for Rmn .
However, matrices enjoy another algebraic property that far outweighs the above in importance. That is, we can multiply them in certain situations. The precise definition is this:

if A = (aij ) is an m p matrix and B = (bij ) is a p n matrix, then the product AB is

the m n matrix whose ij entry is
p
X

aik bkj .

k=1

In other words, the entry of AB in row i and column j is obtained from the ith row of
A and the j th column of B by a kind of dot product of vectors in Rp :
(ai1 , ai2 , . . . , aip ) (b1j , b2j , . . . , bpj ) .
(Notice the commas!).

49

Differentiation

Here are a few numerical examples:

1

1 2 3 2 = 6 ;
3

1
1
2
3

2 1 2 3 = 2
4 6 ;
3
3
6 9

1 1 1

1 0 2
3
1
1
2 0 0 =
;
0 1 3
1 0 3
1 0 1

1 0
0 0
0 0
=
.
0 0
2 0
5 9
To repeat, we can multiply matrices only in this case:
A times B

mp

pn

AB.

mn

There is an identity for this matrix multiplication. Well say that I is the m m (notice
its shape is square) matrix

1 0 ... 0
0 1 . . . 0

I = ..
.
.

0 0 ... 1
(The context will always determine the size of the identity matrix I.) Then
A times I

mn

nn

I times A

mm

mn

A,

mn

A.

mn

It is sometimes useful to employ the Kronecker delta function to work with I: by definition
(
1 if i = j,
ij =
0 if i 6= j,

50

Chapter 2

Thus,
I = (ij ).
Here are some properties of multiplication:
it is associative:
(AB)C = A(BC).
it is distributive:
(A + B)C = AC + BC,
A(B + C) = AB + AC.
0A = A0 = 0 (the three zeros may all be different!).
it is not commutative:
AB 6= BA in general.
The last situation is quite interesting. In the first place, AB is defined only if # of columns
of A = # of rows of B; and BA is defined only if # of columns of B = # of rows of A. Thus
AB and BA are both defined only if A is m n and B is n m; then AB is m m and
BA is n n. Thus AB = BA is possible only if m = n. Thus the only possible situation for
AB = BA is that A and B are both square matrices of the same size. But even then they
might not commute:

1 0
0 1
0 1
=
,
0 0
0 0
0 0

0 1
1 0
0 0
=
.
0 0
0 0
0 0
The latter situation also shows that we cannot in general cancel matrices from an equation:
AB = 0 6 A = 0 or B = 0.

PROBLEM 275. Show that matrix multiplication does not in general have multiplicative inverses, by showing that there is no 2 2 matrix A such that

1 0
1 0
A
=
,
2 0
0 1

1 0
even though
is not zero.
2 0

51

Differentiation

PROBLEM 276. Suppose A is an n n matrix which commutes with every n n

matrix. That is, AB = BA for every n n matrix B. Prove that A is a scalar multiple
of I.
Column vectors. In presenting the general definition of derivative it is helpful to introduce a
convention for writing linear functions from Rn to Rm . The convention is described as follows:
in these situations we modify the notational scheme for points in our basic vector space Rn
by expressing them as columns rather than rows, so that we write

y1
y2

y = .. , an n 1 matrix.
.
yn
The reason for doing this is simple. We want to have a compact expression for a linear function
from Rn to Rm . If A is an m n matrix, then the matrix product Ay is a well-defined m 1
matrix. Thus matrix multiplication produces the desired linear function y 7 Ay.
(If we used the row vector notation we have been following, then the corresponding linear
function would have to send 1 n matrices to 1 m matrices, so it would need to be written
in the form y 7 yA, where A is an n m matrix. As we usually prefer thinking of functions as
operating on the left, this would go against our custom. Hence the sudden change to column
vectors.)
Thus if y is an n 1 column vector, and A is an m n matrix, then Ay is the column
vector in Rm whose ith entry is
n
X
aik yk .
k=1

As a special case, note the interesting result involving ej , the j th coordinate vector for Rn :
A
ej = the j th column of A.
Here of course we are writing ej as a column vector:

0
..
.

ej = 1 (n 1 matrix, 1 in position j).
.
..
0

52

Chapter 2

PROBLEM 277. Notice that in the above situation, Ay is certainly a linear function
of y, in the sense of the definition at the beginning of this section. This exercise establishes
F
the converse. Namely, show that if Rn Rm is linear, then there is a unique mn matrix
A such that
F (y) = Ay for all y Rn .

I want to stress that the matrix A is not the linear function here its not a function at
all. It is just that when A multiplies vectors as above, the resulting function F is linear.
PROBLEM 278. Consider this situation:
G

Rn Rp Rm ,
where F is linear and G is linear. Let the corresponding matrices be A and B, so that
F (x) = Ax for all x Rp ,
G(y) = By for all y Rn .
Prove that the composite function F G is linear, and that its corresponding matrix is
the product AB.
REMARK. Problem 278 gives the actual reason for the strange-looking definition of matrix
multiplication.
J. Derivatives for functions Rn Rm
In this section we are going to attain our goal of defining derivatives in general. Recall that
in Section A we very easily took care of the case n = 1, which essentially was like one-variable
calculus; the generalization to values of functions in Rm rather than R was quite simple.
However, the case m = 1 and general n was quite another matter. We devoted Sections C,
D, and E just to getting the definition of differentiability correct.
Based on the above two paragraphs, you might expect that our current generalization from
m = 1 to general m will be completely straightforward. This is indeed the case. We can even
take our cue from the great definition on p. 224, as we recognize that the crucial form of the
numerator,
f (x + y) f (x) c y,
involves the linear function of y represented there by the scalar product. Literally, all we need

53

Differentiation

to do now is insert the correct form of linear function of y, namely one that maps Rn into Rm .
We know from Section I that such linear functions can be realized as products of the form
Ay, where A is an m n matrix and y Rn is written as an n 1 matrix (a column vector).
Here then is what we require:
f

F DEFINITION. F Let x Rn be a fixed point. Assume Rn Rm is defined at

least in a neighborhood of x (a ball B(x, r)). Then f is differentiable at x if there
exists an m n matrix A such that
lim

y0

f (x + y) f (x) Ay
= 0.
kyk

DISCUSSION
1. Notice that the numerator in this definition makes good sense, as f (x + y), f (x), and
Ay all belong to Rm .
2. This isnt quite like p. 224 in case m = 1. This is only because in the former definition
we wrote our linear function in the form of the scalar product c y. The corresponding
form we are now using in this case

y1
y2

Ay =(c1 c2 . . . cn ) .. .
.
yn
1n

matrix

n1

matrix

This is of course equal to the number

c1 y1 + c2 y2 + + cn yn ,
and in our old notation this is indeed the scalar product
(c1 , c2 , . . . , cn ) (y1 , y2 , . . . , yn ).
3. The remark on p. 225 still holds in this more general case, and asserts that the affine
function of y given by f (x) + Ay is a very good approximation to the function f (x + y)
for small kyk.

54

Chapter 2

4. The proof of p. 227 still applies to show that if f is differentiable at x, then f is

continuous at x.
5. Directional derivatives work the same way as before. If we restrict y to have the form
th for a fixed h Rn and let t 0, we obtain the formula
Df (x; h) = Ah,

all h Rn .

6. TERMINOLOGY. The matrix A is called the Jacobian matrix of f at x. This is in

honor of the great mathematician,
Carl Gustav Jakob Jacobi, 18041851.
We shall use either notation for this matrix,
(Df )(x) or f 0 (x).
7. In particular, let h = ej = the j th coordinate vector. Then we have
f
(x) = Df (x)
ej = j th column of Df (x)
xj
(see p. 251). That is, the n columns of Df (x) are given respectively by the n partial
derivatives f /x1 , . . . , f /xn . We can embellish this observation as follows. Write
the function f in terms of its coordinate functions (of course, arranged in a column!):

f1 (x)
f2 (x)

f (x) = .. .
.
fm (x)
Then the j th column of Df (x) is

f1 /xj
f2 /xj

.
..

.
fm /xj

This gives the all-important formula for the

derivatives,

f1 /x1
f2 /x1

(Df )(x) =
..

.
fm /x1

Jacobian matrix of f in terms of partial

...
...

f1 /xn
f2 /xn

,
..

...

fm /xn

55

Differentiation

where of course all the partial derivatives are evaluated at x.

8. The wonderfully useful sufficient condition for differentiability as given on p. 230 is
still in force. This means that if f1 , . . . , fm all have continuous partial derivatives with
respect to x1 , . . . , xn , then we have differentiability of f and we can use the formula for
the Jacobian matrix as given above.
9. In particular, if all the coordinate functions fi are of class C 1 in B(x, r), then f is
differentiable at every point of B(x, r).
f

10. In the special case of a real-valued function Rn R, we now have two distinct notions
for the derivative. One is the gradient f and the other is the Jacobian matrix Df . It
is perhaps unfortunate that these are different, but they are very similar. The Jacobian
matrix is by defintion the 1 n matrix
Df (x) = (fx1 fx2 . . . fxn ).
(No commas!) On the other hand, since we are thinking of vectors in Rn as column
vectors, we should probably now write

fx1
fx
2
f (x) = .. .
.
fxn
There is very little danger of confusion, for the distinction in these two concepts is that
between algebra and geometry. The directional derivative is given either way as
Df (x; h) = Df (x)h

(matrix product)

Df (x; h) = f (x) h

(dot product).

and

EXAMPLE. Here we work out the Jacobian matrices associated with polar coordinates for
R2 . Denoting R2 as the x y plane, the formulas are
(
x = r cos ,
y = r sin .

56

Chapter 2

We insist that 0 < r < , so we are leaving out the origin in R2 . Of course, < < ,
though a point (x, y) R2 determines only up to the addition of integer multiples of 2.
In order to conform with our column vector notation, we need to write our formulas in
f
column form in terms of a function R2 R2 as

f (r, ) =

r cos
.
r sin

Then

Df (r, ) =

cos r sin
.
sin r cos

(A small but important point: we have arbitrarily chosen to write the independent variables
r, in the order displayed. If we had written them in the order , r, then the columns of our
matrix would be interchanged.)
The two columns of this Jacobian matrix have tremendous geometric significance. Notice
first that they are orthogonal. We can think of f as depicting points in the x y plane as
functions of r, . Thus f /r describes how the point f (r, ) varies with increasing r for fixed
:

(Notice that

f
r

= 1.)

On the other hand, f / describes the motion of f (r, ) with increasing for fixed r:

57

Differentiation

(Notice that

= r.)

The function f assigns Cartesian coordinates if the polar coordinates are given. Conversely,
we now consider the function g which assigns polar coordinates to a point given in Cartesian
coordinates:

r
,
g(x, y) =

where the formulas x = r cos , y = r sin need to be solved for r, . Of course,

p
r = x2 + y 2 ;
as for , it is only determined modulo 2. If we use a formula like = arctan(y/x), we can
readily compute the partial derivatives as

y
= 2
,
x
x + y2

x
= 2
.
y
x + y2

(Theres another way to find these partial derivatives. Namely, start with x = r cos , y =
r sin , and compute directly. For instance, keeping y fixed and using the subscript notation
for partial derivatives,
(
1 = rx cos r sin x ,
.
0 = rx sin + r cos x
Now eliminate the unknown rx from this pair of equations:
( sin )1 + (cos )0 = r sin2 x + r cos2 x .
Thus
sin = rx ,

58

Chapter 2

so that
x =

sin
r sin
y
=
=
.
r
r2
x2 + y 2

Likewise for y . This method has the advantage of avoiding any problems with arctangent
when x = 0.)
Thus we find the Jacobian matrix
!
x

y
2
2
2
2
x +y
x +y
.
Dg(x, y) =
y
x
x2 +y 2

x2 +y 2

In terms of the polar coordinates themselves this may be written

Dg(x, y) =

cos sin
sin
r

cos
r

Notice that Df (r, ) and Dg(x, y) are inverse matrices:

cos sin
cos r sin
Df (r, )Dg(x, y) =
cos
sin
sin r cos
r
r

1 0
=
0 1
= I.
(We shall see in Section K that this is an illustration of the chain rule.)
f

PROBLEM 279. Let Rn Rn be the function given in Problem 236:

f (x) =

x
.
kxk2

Calculate the Jacobian matrix Df (

e1 ).
PROBLEM 280. For the function of the preceding problem show that
Df (x) = kxk2 I 2kxk4 (xi xj ).

59

Differentiation

PROBLEM 281.
v Rn

For the function of the preceding problem show that for any u,
Df (x)u Df (x)v = kxk4 u v.

(Because of this result, the function f is said to be conformal.)

Suppose Rn Rm is itself a linear function: f (x) = Ax. Show

PROBLEM 282.
that for any x Rn

Df (x) = A.
f

(In particular, if Rn Rn is the identity function, f (x) = x for all x, then Df (x) = I.)

PROBLEM 283.
the n m matrix

For any m n matrix A = (aij ), let At be its transpose, namely

At = (aji ).

Show that for all x, y in the appropriate spaces

Ax y = x At y
(what are the appropriate spaces?).

PROBLEM 284. Let A be an n n matrix and let Rn R be the corresponding

quadratic form on Rn ,
f (x) = Ax x.
Show that
f (x) = (A + At )x.
(In particular, if A is symmetric, meaning At = A, then
f (x) = 2Ax.)

60

Chapter 2

PROBLEM 285. Suppose Rn Rm and Rn Rm are both differentiable at x.

Show that f + g is also differentiable at x, and
D(f + g)(x) = Df (x) + Dg(x).

PROBLEM 286. Heres another product rule. Suppose Rn R and Rn Rm are

both differentiable at x. Show that f g is also differentiable at x, and
D(f g)(x) = f (x)Dg(x) + g(x)Df (x).

PROBLEM 287. Another product rule: if Rn Rm and Rn Rm , then show that

D(f g) = f t Dg + g t Df.
(Here f t and g t refer to the transposed values, so that f t (x) and g t (x) are 1 m matrices
(row vectors)).
K. The chain rule
The material in this section is very, very important in calculus and its applications, as the
results are used constantly in both theory and practice. It all has to do with the basic concept
of composition of two functions. In general, whenever two functions f and g are given and it
makes sense that we are able to define g(f (x)), we write the resulting function g f :
(g f )(x) = g(f (x)).
You are surely used to thinking this way, though perhaps not with this notation, from singlevariable calculus.
Our abstract framework will involve this situation:
f

Rn Rm R` ,
where as usual we do not require f and g to be defined everywhere. If n = m = ` = 1, then
we are in the familiar single-variable calculus situation and we certainly recognize the chain

61

Differentiation

rule in the form

d
(g(f (x))) = g 0 (f (x))f 0 (x).
dx
Another notation you are probably familiar with is something like
dz dy
dz
=
.
dx
dy dx
This result emphasizes the expected result that the derivative of the composition is the product
of the derivatives of the two involved functions.
There is a more geometric way of thinking of this in terms of affine approximations. In
the more general situation we are investigating, we think of the function y f (x) + Df (x)y
as the best affine approximation of f (x + y) near y = 0. Let us temporarily express this
approximation in the following notation:
.
f (x + y) = f (x) + Ay,
where A = Df (x). Likewise, if we denote B = Dg(f (x)), then near z = 0 we have the affine
approximation
.
g(f (x) + z) = g(f (x)) + Bz.
Therefore we definitely anticipate that we have the approximation near y = 0,
(g f )(x + y) = g(f (x + y))
.
= g(f (x) + Ay)
.
= g(f (x)) + BAy.
The last expression is an affine function of y and indicates that
D(g f )(x) = BA
= Dg(f (x))Df (x).
This is indeed what we shall prove. Theres a wonderful moral here: while the composition
g f may be very difficult to compute, involving all sorts of complicated operations, the
affine approximation of g f is very easy to compute, involving only the very basic algebra of
multiplying matrices!
We now state and prove what is also sometimes more accurately termed the composite
function theorem. Our proof does not rely on the single-variable calculus result, but actually
handles that as a special case. Though its a special case, all the essential ingredients are
present even there.

62

Chapter 2

THE CHAIN RULE. Assume

f

Rn Rm R` .
Let x be a fixed point in Rn . Assume
f is differentiable at x;
g is differentiable at f (x).
Then
g f is differentiable at x
and
D(g f )(x) = Dg(f (x))Df (x) .
This formula relating the derivatives is really beautiful, containing not only the three
Jacobian matrices, but also expressing the result in terms of the nice definition of matrix
multiplication:
D(g f ) = Dg
`n

`m

Df.
mn

The proof is not hard at all, essentially merely using the definition of differentiability.
PROOF. As above, we denote A = Df (x) (mn matrix) and B = Dg(f (x)) (`m matrix).
The first step in the proof is going to express g as the sum of two functions, one of which is
linear and the other of which varies so little near f (x) that it contributes zero to the derivative
at f (x). Namely, we write g = g1 + g2 , where
g1 (w) = Bw (a linear function),
g2 (w) = g(w) Bw.
Notice that
Dg1 (f (x)) = B
(Problem 282),
Dg2 (f (x)) = Dg(f (x)) B
(Problem 285)
=BB
= 0.

63

Differentiation

Now g f = g1 f + g2 f , so again we can use Problem 285 once we prove that g1 f and
g2 f are differentiable at x. First we have a straightforward algebra calculation
lim

y0

(g1 f )(x + y) (g1 f )(x) BAy

kyk
Bf (x + y) Bf (x) BAy
= lim
y0
kyk
B (f (x + y) f (x) Ay)
= lim
y0
kyk
f (x + y) f (x) Ay
= B lim
y0
kyk
= B0
= 0.

This proves that g1 f is differentiable at x and

D(g1 f )(x) = BA.
Now we are going to show that g2 f is differentiable at x and
D(g2 f )(x) = 0.
(This will finish the proof, thanks to Problem 285.) That is, we are going to prove that
g2 (f (x + y)) g2 (f (x))
= 0.
y0
kyk
lim

This is completely expected, thanks to the fact that Dg2 (f (x)) = 0. We just need to check
that the presence of f doesnt cause any unwelcome surprises.
We first claim that f satisfies a Lipschitz condition at the point x. Namely (see Problem 2
44), there exists a constant C and a positive number such that
kf (x + y) f (x)k Ckyk
for all kyk < . (Well prove this at the end.)
Now for kyk < we consider the quotient
kg2 (f (x + y)) g2 (f (x)) k
kyk

()

64

Chapter 2

If it happens to equal zero for a particular y, then we are of course quite pleased, as we
want to show () has limit zero. So we need only be concerned with those y for which kyk <
and () 6= 0. In particular, f (x + y) f (x) 6= 0. Let us call this latter difference w. Of course,
w depends on y and in fact kwk Ckyk. In this situation
kg2 (f (x) + w) g2 (f (x)) k kwk

kwk
kyk
kg2 (f (x) + w) g2 (f (x)) k

C.
kwk

() =

This last quantity has limit zero as y 0, since also w 0 and we are given that
Dg2 (f (x)) = 0.
Finally we establish the Lipschitz condition for f . We first notice that the entries of the
fixed matrix A are just fixed numbers, and thus there is a number C0 such that we have the
inequality for norms,
kAyk C0 kyk
for all y Rn .
Next, the triangle inequality implies
kf (x + y) f (x)k
kf (x + y) f (x) Ayk
kAyk

+
kyk
kyk
kyk
kf (x + y) f (x) Ayk

+ C0 ;
kyk
this sum has limit C0 as y 0, because A = Df (x). Thus it is no larger than, say, 1 + C0 = C
for all sufficiently small y (say, kyk < ).
QED
f

PROBLEM 288. In a general situation Rn Rm in which f is differentiable at a

fixed point x, define the affine approximation to be the function Aff(f, x), where
Aff(f, x)(u) = f (x) + Df (x)(u x)

for all u Rn .

(Why?) Then prove as a result of the chain rule

Aff(g f, x) = Aff(g, f (x)) Aff(f, x).

ILLUSTRATIONS.

65

Differentiation

1. Look again at the polar coordinate example on pp. 255 through 258. There we have a
situation where f and g are inverses of one another, so that f g = the identity function
from R2 to R2 . Thus D(f g)(x, y) = I by Problem 282. The chain rule thus gives
Df (g(x, y)) Dg(x, y) = I,
just as we observed by explicit calculation on p. 258.
f

2. More generally, any time we have a situation Rn Rn Rn in which f and g are

inverses in the sense that g f = the identity function, then
Dg (f (x)) Df (x) = I.
(We say that the corresponding Jacobian matrices are inverses of one another.) This
comes as no surprise. After all, if f and g are inverses then their affine approximations
should also be inverses, thanks to Problem 288.
3. The most important abstract situation to understand is the case n = 1, ` = 1:
f

R Rm R.
We shall in fact show in the two subsequent items that the generalization to arbitrary
n and ` is then immediate. For this illustration we shall write generically f = f (t) and
g = g(u) = g(u1 , . . . , um ). The chain rule then tells us immediately
D(g f )(t) = Dg (f (t)) Df (t).
That is,

df1 /dt
d
g
g

g (f (t)) =
(f (t)) . . .
(f (t)) ...
dt
u1
um
dfm /dt
m
X g
dfk
=
(f (t))
.
uk
dt
k=1

It seems to help in remembering this formula to abuse the notation by writing f (t) as
u(t). The idea is that the independent variables uk have been replaced by functions
uk (t) and the resulting calculus formula is
m
X
g duk
d
g (u1 (t), . . . , um (t)) =
.
dt
uk dt
k=1

66

Chapter 2

So you can really just use the single-variable chain rule as a pattern and just keep
differentiating as long as you see a t.
4. The generalization
f

Rn Rm R
is immediate, as the act of computing /xi is a matter of letting the other coordinates
be fixed and applying number 3:
m
X

g uk
g (u1 (x), . . . , um (x)) =
,
xi
u
k xi
k=1

an equation that is valid for each i between 1 and n.

5. The full generalization
f

Rn Rm R`
now follows simply by applying number 4 to each component of g, one at a time:
m
X

gj uk
gj (u1 (x), . . . , um (x)) =
xi
uk xi
k=1

valid for 1 i n and 1 j `.

6. The special case
f

Rn R R
is often quite useful. We have

f
g (f (x)) = g 0 (f (x))
.
xi
xi
In terms of the gradient notation,
(g f ) = g 0 (f (x)) f (x).
For instance,
(ef ) = ef f,
(g(kxk)) = g 0 (kxk)

x
.
kxk

67

Differentiation

PROBLEM 289. Here is another proof that the pathological function of Problem 2
43 is indeed not differentiable at the origin. Calculate for that function that

d
1
2
f (t, t ) = .
dt
2
t=0
Why does this show that f is not differentiable at 0?

PROBLEM 290. Define R2 R by

( 2 5/4
f (x, y) =

x |y|
x4 +y 2

for (x, y) 6= (0, 0),

for (x, y) = (0, 0).

Show that f is continuous on R2 and that all directional derivatives Df (0; h) = 0. Then
prove that f is not differentiable at the origin by consideration of f (t, t2 ).

PROBLEM 291. This is a rather standard situation that frequently arises in therF
modynamics and other applications. Suppose R3 R is a differentiable function whose
first order partial derivatives are never zero. Furthermore, suppose that the equation
F (x, y, z) = 0
can be solved for each of the three unknowns as functions of the other two variables.
For example, in this way we can regard x as a function of y and z, so with abuse of
notation
F (x, y, z) = 0 produces x = x(y, z).
It then makes sense to define x/y,
Prove that
x
y

the partial derivative of x(y, z) with z held fixed.

y z

= 1.
z x

68

Chapter 2

PROBLEM 292. Let R2 R2 be defined by

f (x1 , x2 ) = (x21 x22 , 2x1 x2 ).
Calculate the 2 2 Jacobian matrix Df (x).

PROBLEM 293. The function of the preceding problem actually comes from the
corresponding complex function (x1 + ix2 )2 . Since every complex number other than 0
has two square roots, the equation f (x) = y should have two distinct solutions for each
y 6= 0 in R2 . Find them explicitly. (Part of the answer is
r
kyk + y1
x1 =
.)
2

PROBLEM 294. Let R2 R2 be defined by

f (x1 , x2 ) = (x31 3x1 x22 , 3x21 x2 x32 ),
and calculate Df (x). What complex function does this resemble?

PROBLEM 295.
The complex exponential function ez produces through Eulers
f
formula the function R2 R2 given as
f (x1 , x2 ) = (ex1 cos x2 , ex1 sin x2 ).
Calculate the Jacobian matrix Df (x1 , x2 ).

69

Differentiation

PROBLEM 296. Given y 6= 0 in R2 one can define a complex logarithm of y1 + iy2

to be any complex number x1 + ix2 such that ex1 +ix2 = y1 + iy2 . In terms of R2 and
the function of the preceding exercise, this means that f (x) = y. Find all such points x.
Roughly speaking, the answer is

y2
.
x = log kyk, arctan
y1

PROBLEM 297. Pretend that the formula of Problem 296 gives a (single-valued)
function x = g(y). Verify directly from that and Problem 295 the relation
Dg(f (x))Df (x) = I.

PROBLEM 298. Let R2 R4 be defined by

f (, ) = (cos , sin , cos , sin ).
Calculate the 4 2 Jacobian matrix Df (, ). Show that the two columns of this matrix
are unit vectors orthogonal to one another.
L. Confession
Confess your faults to one another
James 516
Although we have given a correct definition of the differentiability at x of a function
R Rm , together with the Jacobian matrix Df (x), that is not quite the whole truth. The
reason is that we have relied on the identification of linear functions with their corresponding
matrices.
The better definition ignores matrices entirely and just mentions the linear functions. Thus
L
we can equivalently define f to be differentiable at x if there exists a linear function Rn Rm
such that
f (x + y) f (x) L(y)
lim
= 0.
y0
kyk
n f

70

Chapter 2

The corresponding terminology might be this: the linear function L is called the differential
of f at x, and is denoted
(df )(x) = L.
The correspondence with the Jacobian matrix Df (x) is that
(df )(x)(y) = Df (x) y
m1

mn

for all y Rn .

n1

Thus for example we have

(df )(x)(
ej ) =

f
(x).
xj

PROBLEM 299.
f
a. If Rn Rm is linear, show that df (x) = f .
f

b. If Rn Rm is affine, show that df (x) = f f (0).

PROBLEM 2100.
Section K. Show that

Use the notation of the statement of the chain rule as given in

d(g f )(x) = dg(f (x)) df (x).

What is the point of this shift in emphasis? It is that we gain in geometric insight by
stressing the geometric object df (x) instead of the algebraic object Df (x). But not only that.
We often have situations in which the ej s are not the natural basic vectors to be using and
the given coordinates are somehow unnatural. Then the actual m n matrix Df (x) may be
of little interest, and we might prefer using a different m n matrix.
Notice also that the statement of the chain rule is more elegant in the new formulation,
as both sides of the formula involve composition of functions. The algebra involved in matrix
multiplication does not appear.
In summary, we might say that the linear function df (x) is represented by the Jacobian
matrix Df (x) in the usual coordinate systems we are using.
M. Homogeneous functions and Eulers formula
f

In this section we are concerned with functions Rn R which are defined on all of Rn
except the origin. Let a be a fixed real number.

71

Differentiation

DEFINITION. The function is homogeneous of degree a if

f (tx) = ta f (x) for all 0 < t < and all x Rn {0}.

()

Assume from now on that f is of class C 1 on the set Rn {0}.

PROBLEM 2101.
Prove that if f is homogeneous of degree a, then the partial
derivatives f /xj are homogeneous of degree a 1.
PROBLEM 2102. For fixed x 6= 0 differentiate the equation () with respect to t.
Then set t = 1 and conclude that the Euler equation is satisfied:
n
X
j=1

xj

f
= af.
xj

()

PROBLEM 2103. Conversely, assume that the Euler equation () is satisfied and
then prove that f is homogeneous of degree a. (HINT: dtd (ta f (tx)).)
PROBLEM 2104.
Assume f is a polynomial which is homogeneous of degree a,
where a is of course a nonnegative integer. Establish the Euler equation for f by explicitly
calculating what happens for the individual monomials
xa11 xa22 . . . xann ,

a1 + a2 + + an = a.

PROBLEM 2105.
Suppose f is homogeneous of degree 1, and define f (0) = 0.
Assume that f is differentiable at 0. Prove that
f (x) = f (0) x.

PROBLEM 2106. Give an example of a function which is homogeneous of degree 1

and continuous on Rn and not differentiable at 0.