AMATH/MATH 516

FIRST HOMEWORK SET

Problems are due by class time Wednesday 04/09/14. Problems (2)[B] and 3 are significantly more challenging. The
purpose of this problem set is to have you brush up and further develop your linear algebra skills. The problem set
will be difficult for some and straightforward for others. If you are having any difficulty, please feel free to discuss
the problems with me at any time. I am very open with giving hints. Don’t delay in starting work on these problems!

(1) Let Q be an n × n symmetric positive definite matrix. The following fact for symmetric matrices can be used
to answer the questions in this problem.
Fact: If M is a real symmetric n × n matrix, then there is a real orthogonal n × n matrix U
(U T U = I) and a real diagonal matrix Λ = diag(λ1 , λ2 , . . . , λn ) such that M = U ΛU T .
(a) Show that the eigenvalues of Q2 are the square of the eigenvalues of Q.
(b) If λ1 ≥ λ2 ≥ · · · ≥ λn are the eigen values of Q, show that
λn kuk22 ≤ uT Qu ≤ λ1 kuk22 ∀ u ∈ IRn .
(c) If 0 < λ < λ̄ are such that
λkuk22 ≤ uT Qu ≤ λ̄kuk22 ∀ u ∈ IRn ,
then all of the eigenvalues of Q must lie in the interval [λ, λ̄].
(d) Let λ and λ̄ be as in Part (c) above. Show that
λkuk2 ≤ kQuk2 ≤ λ̄kuk2 ∀ u ∈ IRn .
Hint: kQuk22 = uT Q2 u.
(2) Do one of the following two problems: [A] or [B]. If you are already familiar with the general theory of inner
products as well as elliptic inner products, then do the second problem [B]; otherwise do the first [A]. Of
course, you are welcome to do both, but I will only grade one of them. Also note that problem [A] is used
in problem 3.
[A ] A mapping h·, ·i : IRn 7→ IRn is said to be an inner product on IRn is for all x, y, z ∈ IRn
(a) hx, xi ≥ 0 Non-Negative
(b) hx, xi = 0 ⇔ x = 0 Positive
(c) hx + y, zi = hx, zi + hy, zi Additive
(d) hαx, yi = α hx, yi ∀ α ∈ IR Homogeneous
(e) hx, yi = hy, xi Symmetric
Two vectors x, y ∈ IRn are said to be orthogonal in the inner product h·, ·i if hx, yi = 0
Unless otherwise specified, we use the notation hx, yi to designate the usual Euclidean inner product:
n
X
hx, yi = xi yi .
i=1

(i) Let hx, yi be the Euclidean inner product on IRn . Given A ∈ IRn×n , show that A = 0 if and only
if
hx, Ayi = 0 ∀ x, y ∈ IRn .
(ii) Let H ∈ IRn×n be symmetric and positive definite (i.e. H = H T and xT Hx > 0 ∀ x ∈ IRn \ {0}).
Show that the bi-linear form given by
hx, yiH = xT Hy ∀ x, y ∈ IRn
defines an inner product on IRn .
(iii) Every inner product defines a transformation on the space of linear operators called the adjoint.
For the Euclidean inner product on IRn , this is just the usual transpose. Given a linear transfor-
mation M : IRn 7→ IRn , the adjoint is defined by the relation
hy, M xi = hM ∗ y, xi , for all x, y, ∈ IRn .
1
2

The inner product given above, h·, ·iH , also defines an adjoint mapping which we can denote by
M TH . Show that
M TH = H −1 M T H .
(iv) The matrix P ∈ IRn×n is said to a projection if P 2 = P . Clearly, if P is a projection, then so is
I − P . The subspace P IRn = Ran (P ) is called the subspace that P projects onto. A projection
is said to be orthogonal with respect to a given inner product h·, ·i on IRn if and only if
h(I − P )x, P yi = 0 ∀ x, y ∈ IRn ,
that is, the subspaces Ran (P ) and Ran (I − P ) are orthogonal in the inner product h·, ·i. Show
that the projection P is orthogonal with respect to the inner product h·, ·iH (defined above), where
H ∈ IRn×n is symmetric and positive definite, if and only if
P = H −1 P T H .
[B ] Let H ∈ IRsn×n , u ∈ IRn , and α ∈ IR where IRsn×n is the linear space of all real symmetric n × n
matrices. Recall that H is said to be positive definite if xT Hx > 0 for all x ∈ IRn with x 6= 0. Moreover,
H is said to be positive semi–definite if xT Hx ≥ 0 for all x ∈ IRn . We consider the block matrix
 
H u
Ĥ := .
uT α
(i) Show that Ĥ is positive semi–definite if and only if H is positive semi–definite and there exists a
vector z ∈ IRn such that u = Hz and α ≥ z T Hz.
(ii) Show that Ĥ is positive definite if and only if H is positive definite and α > uT H −1 u.
(iii) Let A ∈ IRm×n , b ∈ IRm , c ∈ IRn , and δ ∈ IR. Use either Part (i) or Part (ii) to show that x ∈ IRn
is a solution to the quadratic inequality
(Ax + b)T (Ax + b) ≤ cT x + δ
if and only if the block matrix
 
I (Ax + b)
(Ax + b)T (cT x + δ)
is positive semi–definite.
(iv) Suppose H is positive definite. Show that
    
H u I 0 H u
= .
0 (α − uT H −1 u) (−H −1 u)T 1 uT α
(v) Recall that the kth principal minor of a matrix B ∈ IRn×n is the determinant of the upper left–
hand corner k×k–submatrix of B for 1 ≤ k ≤ n. Use an induction argument and Parts (ii) and
(iv) above to show that H is positive definite if and only if every principal minor of H is positive.
Note: Your argument must use either Part (i) or Part (ii) above.
Hint: det(AB)=det(A)det(B), and the determinant of an upper or lower block triangular matrix
is the product of the determinants of the diagonal blocks.
(3) Let A ∈ IRm×n , c ∈ IRn , a ∈ IRm , δ > 0, and H ∈ IRn×n with H symmetric positive definite. Consider the
problem
P minx∈IRn cT x
subject to Ax = a
kx0 − xkH ≤ δ
where x0 ∈ IRn satisfies Ax0 = a,
kzkH = (z T Hz)1/2 = [hz, ziH ]1/2 ,
and the inner product h·, ·iH is defined in part (b) of problem 2 above.
(a) Suppose H = LLT for some non–singular matrix L ∈ IRn×n , e.g. L = H 1/2 . If Q is the orthogonal
projector onto the null–space of AL−T in the usual (or Euclidean) inner product, show that the operator
P given by
P = L−T QLT
is the orthogonal projector onto the null–space of A with respect to the inner product h·, ·iH .
 3

(b) Show that

x̄ = x0 − δkP H −1 ck−1
H PH
−1
c
solves P where P is as given in part (a) above.
Hint: It may be helpful to first reduce the problem to one of the form
min ĉT w
subject to Âw = 0
kwk22 ≤ δ 2 .
It is also helpful to apply results relating least–squares to orthogonal projection.