Introduction to MATLAB and Linear Algebra

Ralph C. Smith
Department of Mathematics
North Carolina State University

Overview:
1. MATLAB examples throughout tutorial
2. Elements of linear algebra
• Fundamental properties of vectors and matrices
• Eigenvalues, eigenvectors and singular values

Linear Algebra and Numerical Matrix Theory:

• Vector properties including orthogonality
• Matrix analysis, inversion and solving Ax = b for very large systems
• Eigen- and singular value decompositions
 Linear Algebra and Numerical Matrix Theory
Topics: Illustrate with MATLAB as topics are introduced
• Basic concepts
• Linear transformations
• Linear independence, basis vectors, and span of a vector space
• Fundamental Theorem of Linear Algebra
• Determinants and matrix rank
• Eigenvalues and eigenvectors
• Solving Ax = b and condition numbers
• Singular value decomposition (SVD)
• Cholesky and QR decompositions

Linear Algebra has become as basic and as applicable

as calculus, and fortunately it is easier.
--Gilbert Strang, MIT
 Vectors and Matrices
Note:

• Vector in Rn is an ordered æ1ö

set of n real numbers. ç ÷
ç6÷
– e.g. v = (1,6,3,4) is in R4 ç 3÷
– “(1,6,3,4)” is a column ç ÷
ç 4÷
vector: è ø
– as opposed to a row
vector: (1 6 3 4)
• m-by-n matrix is an object
with m rows and n columns,
æ1 2 8ö
each entry fill with a real ç ÷
number: ç 4 78 6 ÷
ç9 3 2÷
è ø
 Vector Norms
Vector Norms: The norm ||x|| quantifies the ``length’’

Common Norms:

Euclidean: Vector length

 Vector and Matrix Products
Vector products:
æ v1 ö
– Dot product: u • v = u v = (u1
T
u2 )çç ÷÷ = u1v1 + u2 v2
è v2 ø
Note: A × B = A B cos(q )

If u•v=0, ||u||2 != 0, ||v||2 != 0 à u and v are orthogonal

If u•v=0, ||u||2 = 1, ||v||2 = 1 à u and v are orthonormal

æ u1 ö æ u1v1 u1v2 ö
– Outer product: uv = çç ÷÷(v1 v2 ) =
T
çç ÷÷
è u2 ø è u2 v1 u2 v2 ø
Matrix Product:

æ a11 a12 ö æ b11 b12 ö

A = çç ÷÷, B = çç ÷÷
è a21 a22 ø è b21 b22 ø
æ a11b11 + a12b21 a11b12 + a12b22 ö
AB = çç ÷÷
è a21b11 + a22b21 a21b12 + a22b22 ø
 Special Matrices

æ a 0 0ö æa b cö
ç ÷ ç ÷
ç 0 b 0÷ diagonal ç0 d e÷ upper-triangular
ç0 0 c÷ ç0 0
è ø è f ÷ø

æa b 0 0ö æa 0 0ö
ç ÷ ç ÷
çc d e 0÷
çb c 0 ÷ lower-triangular
ç0 ÷ tri-diagonal
ç
f g h
÷ çd e f ÷ø
ç0 è
è 0 i j ÷ø

æ1 0 0ö
ç ÷
ç0 1 0÷ I (identity matrix)
ç0 0 1÷
è ø
 Special Matrices
• Matrix A is symmetric if A = AT
• A is positive definite if xTAx>0 for all non-zero x (positive semi-definite if
inequality is not strict)

æ 1 0 0 öæ a ö æ 1 0 0 öæ a ö
ç ÷ç ÷ ç ÷ç ÷
(a b c )ç 0 1 0 ÷ç b ÷ = a 2 + b 2 + c 2 (a b c )ç 0 - 1 0 ÷ç b ÷ = a 2 - b 2 + c 2
ç 0 0 1 ÷ç c ÷ ç 0 0 1 ÷ç c ÷
è øè ø è øè ø

• Useful fact: Any matrix of form ATA is positive semi-definite.

To see this, xT(ATA)x = (xTAT)(Ax) = (Ax)T(Ax) ≥ 0

Recall: Covariance matrix

2
V = (X T X )-1
 Matrices as Linear Transformations

æ5 0 öæ1ö æ 5ö
çç ÷÷çç ÷÷ = çç ÷÷ (stretching)
è0 5 øè1ø è 5ø

æ0 - 1öæ1ö æ - 1ö
çç ÷÷çç ÷÷ = çç ÷÷ (rotation)
è1 0 øè1ø è 1 ø

æ0 1 öæ 1 ö æ 0 ö
çç ÷÷çç ÷÷ = çç ÷÷
è1 0 øè 0 ø è 1 ø (reflection)
æ1 0 öæ1ö æ 1 ö
çç ÷÷çç ÷÷ = çç ÷÷
è0 0 øè1ø è 0 ø (projection)
 Linear Independence

• A set of vectors is linearly independent if none of them

can be written as a linear combination of the others.
• Vectors v1,…,vk are linearly independent if c1v1+…+ckvk
= 0 implies c1=…=ck=0 æ | | | öæ c1 ö æ 0 ö
ç ÷ç ÷ ç ÷
ç v1 v2 v3 ÷ç c2 ÷ = ç 0 ÷
ç| | | ÷ç c ÷ ç 0 ÷
è øè 3 ø è ø
e.g. æ1 0ö æ0ö
ç ÷æ u ö ç ÷ (u,v)=(0,0), i.e. the columns
ç 2 3 ÷çç ÷÷ = ç 0 ÷
ç 1 3 ÷è v ø ç 0 ÷ are linearly independent.
è ø è ø

x3 = −2x1 + x2
 Basis Vectors

• If all vectors in a vector space may be expressed as linear

combinations of a set of vectors v1,…,vk, then v1,…,vk spans
the space.
• The cardinality of this set is the dimension of the vector
space.
æ 2ö æ1ö æ0ö æ0ö
e.g. ç ÷ ç ÷
ç 2 ÷ = 2ç 0 ÷ +
ç ÷
2ç 1 ÷ +
ç ÷
2ç 0 ÷
ç 2÷ ç0÷ ç0÷ ç1÷
(0,0,1) è ø è ø è ø è ø

(0,1,0)

(1,0,0)

• A basis is a maximal set of linearly independent vectors and

a minimal set of spanning vectors of a vector space
 Basis Vectors

Note:
• An orthonormal basis consists of orthogonal vectors of unit length.

æ 2ö æ1ö æ0ö æ0ö æ 2ö æ .9 ö æ .3 ö æ .1 ö

ç ÷ ç ÷ ç ÷ ç ÷ ç ÷ ç ÷ ç ÷ ç ÷
2
ç ÷ = 2ç0÷ + 2ç 1 ÷ + 2ç 0 ÷ ç 2 ÷ = 1.57ç .2 ÷ + 1.29ç 1 ÷ + 2ç .2 ÷
ç 2÷ ç0÷ ç0÷ ç1÷ ç 2÷ ç0÷ ç0÷ ç1÷
è ø è ø è ø è ø è ø è ø è ø è ø
(0,0,1) (.1,.2,1)

(0,1,0)

(.3,1,0)
(1,0,0) (.9,.2,0)
 Rank of a Matrix

• The rank of A is the dimension of the column space of A.

• It also equals the dimension of the row space of A (the subspace of vectors
which may be written as linear combinations of the rows of A).

æ1 0ö (1,3) = (2,3) – (1,0)

ç ÷
ç 2 3÷
ç 1 3÷ Only 2 linearly independent rows, so
è ø rank = 2.

Fundamental Theorem of Linear Algebra:

If A is m x n with rank r,
Column space(A) has dimension r
Nullspace(A) has dimension n-r (= nullity of A)
Row space(A) = Column space(AT) has dimension r
Left nullspace(A) = Nullspace(AT) has dimension m - r
Rank-Nullity Theorem: rank + nullity = n
 Matrix Inverse
Note:
• To solve Ax=b, we can write a closed-form solution if we can find a matrix A-1
such that AA-1 =A-1A=I (Identity matrix)
• Then Ax=b iff x=A-1b:
x = Ix = A-1Ax = A-1b
• A is non-singular iff A-1 exists iff Ax=b has a unique solution.
• Note: If A-1,B-1 exist, then (AB)-1 = B-1A-1,
and (AT)-1 = (A-1)T

• For orthonormal matrices

A-1 = AT
 Matrix Determinants
Note:
• If det(A) = 0, then A is singular.
• If det(A) ≠ 0, then A is invertible.
• To compute:
– Simple example:
æa b ö
detçç ÷÷ = ad - bc
èc d ø

– Matlab: det(A)
 MATLAB Interlude
Special Variables:
• Special variables:
– ans : default variable name for the result
– pi: p = 3.1415926…………
– eps: Î = 2.2204e-016, smallest amount by which 2 numbers can differ.
– Inf or inf : ¥, infinity
– NaN or nan: not-a-number

Vectors: Example:
>> x = [ 0 0.25*pi 0.5*pi 0.75*pi pi ] x is a row vector.
x=
0 0.7854 1.5708 2.3562 3.1416
>> y = [ 0; 0.25*pi; 0.5*pi; 0.75*pi; pi ] y is a column vector.
y=
0
0.7854
1.5708
2.3562
3.1416
 Vectors

• Vector Addressing – A vector element is addressed in MATLAB with an integer

index enclosed in parentheses.
• Example:
>> x(3)
ans =
1.5708 ç 3rd element of vector x
• The colon notation may be used to address a block of elements.
(start : increment : end)
start is the starting index, increment is the amount to add to each successive index, and end
is the ending index. A shortened format (start : end) may be used if increment is 1.
• Example:
>> x(1:3)
ans =
0 0.7854 1.5708 ç 1st to 3rd elements of vector x

NOTE: MATLAB index starts at 1.

 Vectors

Some useful commands:

x = start:end create row vector x starting with start, counting by

one, ending at end

x = start:increment:end create row vector x starting with start, counting by

increment, ending at or before end
linspace(start,end,number) create row vector x starting with start, ending at end,
having number elements
length(x) returns the length of vector x

y = x’ transpose of vector x

dot (x, y) returns the scalar dot product of the vector x and y.
 Matrices
§ A Matrix array is two-dimensional, having both multiple rows and multiple columns,
similar to vector arrays:
§ it begins with [, and end with ]
§ spaces or commas are used to separate elements in a row
§ semicolon or enter is used to separate rows.
• Matrix Addressing: •Example:
-- matrixname(row, column) •>> f = [ 1 2 3; 4 5 6]
-- colon may be used in place of a row or column f=
reference to select the entire row or column. 1 2 3
4 5 6
Example:
n
recall:
>> f(2,3) f=
ans = 1 2 3
6 4 5 6
>> h(:,1) h=
ans = 2 4 6
2
1 3 5
1
 Matrices

more commands
Transpose B = A’
Identity Matrix eye(n) è returns an n x n identity matrix
eye(m,n) è returns an m x n matrix with ones on the main
diagonal and zeros elsewhere.
Addition and subtraction C=A+B
C=A–B
Scalar Multiplication B = aA, where a is a scalar.
Matrix Multiplication C = A*B
Matrix Inverse B = inv(A), A must be a square matrix in this case.
rank (A) è returns the rank of the matrix A.
Matrix Powers B = A.^2 è squares each element in the matrix
C = A * A è computes A*A, and A must be a square matrix.
Determinant det (A), and A must be a square matrix.
A, B, C are matrices, and m, n, a are scalars.
 Matrices

>> A = [1 2;3 4] >> A = [1 2 3;0 2 0]

A= A=
1 2
3 4

>> B = [2 0;2 1] >> B = [1;-1;0]

B= B=
2 0
2 1

>> A*B
ans = >> A*B
6 2 ans =
14 4

>> A.*B
ans =
2 0
6 4
 Eigenvalues and Eigenvectors
Note:
• How can we characterize matrices?
• The solutions to Ax = λx in the form of eigenpairs (λ,x) = (eigenvalue,eigenvector)
where x is non-zero
• To solve this, (A – λI)x = 0
• λ is an eigenvalue iff det(A – λI) = 0

Example:
æ1 4 5 ö
ç ÷
A = ç0 3 / 4 6 ÷
ç0 0 1 / 2 ÷ø
è

æ1 - l 4 5 ö
ç ÷
det( A - lI ) = ç 0 3/ 4 - l 6 ÷ = (1 - l )(3 / 4 - l )(1 / 2 - l )
ç 0 0 1 / 2 - l ÷
è ø

l = 1, l = 3 / 4, l = 1 / 2
 Eigenvalues and Eigenvectors
Example:

æ 2 0ö Eigenvalues λ = 2, 1 with
A = çç ÷÷
è 0 1 ø eigenvectors (1,0), (0,1)
Eigenvectors of a linear transformation A are not rotated (but will be
scaled by the corresponding eigenvalue) when A is applied.

(0,1)
Av
v

(1,0) (2,0)

Important Results:
• Eigenvalues of nxn symmetric matrix are real but may not be distinct. There
are n orthogonal eigenvectors.
 Eigenvalues and Eigenvectors
Important Results:
• Eigenvalues of nxn symmetric matrix are real but may not be distinct. There
are n orthogonal eigenvectors; e.g., Identity matrix
• Eigenvalues of a positive definite matrix are positive.
• MATLAB Command: [V,D] = eig(A)
• Suppose A has n linearly independent eigenvectors. Then

Q = [v1 v2 · · · vn ]
) AQ = [ 1 v1 2 v2 ··· n vn ]

) AQ = Q ⇤
) A = Q ⇤Q -1
) Q -1 AQ = ⇤
Note:
• The matrix Q thus provides a new basis for A.
 Eigenvalues and Eigenvectors
Example:
>>Q*D*inv(Q)
>> A = [2 1 3;-2 0 -2;5 5 2]
ans =
A=

>> Q’
>> X = A'*A
ans =
X=

>> inv(Q)
Q,D] = eig(X)
ans =
Q

D=
0
 Eigenvalues and Eigenvectors
Cholesky Decomposition: Matrix must be positive definite
>> A = [2 1 3;-2 0 -2;5 5 2]
A=

>> X = A'*A
X=

>> R = chol(X)
R=

>> R'*R
ans =
 Solving Ax = b
Notes:
• MATLAB: x = A\b;
• How stable is the solution?
• If A or b are changed slightly, how much does it effect x?
• The condition number c of A measures this:
c = λmax/ λmin
• Values of c near 1 are good.

>> A = [1 0;0 1e-10]

A=

>> cond(A)
ans =
e+10>>
 MATLAB

• Example: a system of 3 linear equations with 3 unknowns (x1, x2, x3):

3x1 + 2x2 – x3 = 10
-x1 + 3x2 + 2x3 = 5
x1 – x2 – x3 = -1
Let :
é3 2 1ù é x1 ù é10 ù
A = ê- 1 3 2 ú x = ê x2 ú b=ê 5 ú
ê ú ê ú ê ú
êë 1 - 1 - 1úû êë x3 úû êë- 1úû

Then, the system can be described as:

Ax = b
 MATLAB

• Solution by Matrix Inverse: • Solution by Matrix Division:

Ax = b The solution to the equation
A-1Ax = A-1b Ax = b
x = A-1b can be computed using left division.
• MATLAB: n MATLAB:
>> A = [ 3 2 -1; -1 3 2; 1 -1 -1]; >> A = [ 3 2 -1; -1 3 2; 1 -1 -1];
>> b = [ 10; 5; -1]; >> b = [ 10; 5; -1];
>> x = inv(A)*b >> x = A\b
x= x=
-2.0000 -2.0000
5.0000 5.0000
-6.0000 -6.0000
Answer: Answer:
x1 = -2, x2 = 5, x3 = -6 x1 = -2, x2 = 5, x3 = -6
 MATLAB: Flow Control
For Loops:
for j=1:5 % use for-loops to execute iterations / repetitions
for i=1:3
a(i, j) = i + j ;
end
end

If Conditional:
a = zeros(3); b = zeros(3);
for j=1:3
for i=1:3
a(i,j) = rand; % use rand to generate a random number
if a(i,j) > 0.5
b(i,j) = 1;
end
end
end
 MATLAB
Cell Arrays:

A cell array is a special array of arrays. Each element of the cell

array may point to a scalar, an array, or another cell array.
>> C = cell(2, 3); % create 2x3 empty cell array
>> M = magic(2);
>> a = 1:3; b = [4;5;6]; s = 'This is a string.';
>> C{1,1} = M; C{1,2} = a; C{2,1} = b; C{2,2} = s; C{1,3} = {1};
C=
[2x2 double] [1x3 double] {1x1 cell}
[2x1 double] ‘This is a string.‘ []
>> C{1,1} % prints contents of a specific cell element
ans =
1 3
4 2
>> C(1,:) % prints first row of cell array C; not its content
 MATLAB
Structures:
Ideal layout for grouping arrays that are related.
>> name(1).last = ‘Smith’; name(2).last = ‘Hess’;
>> name(1).first = ‘Mary’; name(2).first = ‘Robert’;
>> name(1).sex = ‘female’; name(2).sex = ‘male’;
>> name(1).age = 45; name(2).age = 50;
>> name(2)
ans =
last: 'Hess'
first: 'Robert'
sex: 'male'
age: 50
 MATLAB
Frequently Used Functions:

>> magic(n) % creates a special n x n matrix; handy for testing

>> zeros(n,m) % creates n x m matrix of zeroes (0)
>> ones(n,m) % creates n x m matrix of ones (1)
>> rand(n,m) % creates n x m matrix of random numbers
>> repmat(a,n,m) % replicates a by n rows and m columns
>> diag(M) % extracts the diagonals of a matrix M
>> help elmat % list all elementary matrix operations ( or elfun)
>> abs(x); % absolute value of x
>> exp(x); % e to the x-th power
>> fix(x); % rounds x to integer towards 0
>> log10(x); % common logarithm of x to the base 10
>> rem(x,y); % remainder of x/y
>> mod(x, y); % modulus after division – unsigned rem
>> sqrt(x); % square root of x
>> sin(x); % sine of x; x in radians
>> acoth(x) % inversion hyperbolic cotangent of x
 Plotting
Line Plot:

>> t = 0:pi/100:2*pi;
>> y = sin(t);
>> plot(t,y)
>> xlabel(‘t’);
>> ylabel(‘sin(t)’);
>> title(‘The plot of t vs sin(t)’);
 Plotting
Customizing Graphical Effects

Generally, MATLAB’s default graphical settings are adequate which make

plotting fairly effortless. For more customized effects, use the get and set
commands to change the behavior of specific rendering properties.
>> hp1 = plot(1:5) % returns the handle of this line plot
>> get(hp1) % to view line plot’s properties and their values
>> set(hp1, ‘lineWidth’) % show possible values for lineWidth
>> set(hp1, ‘lineWidth’, 2) % change line width of plot to 2
>> gcf % returns current figure handle
>> gca % returns current axes handle
>> get(gcf) % gets current figure’s property settings
>> set(gcf, ‘Name’, ‘My First Plot’) % Figure 1 => Figure 1: My First Plot
>> get(gca) % gets the current axes’ property settings
>> figure(1) % create/switch to Figure 1 or pop Figure 1 to the front
>> clf % clears current figure
>> close % close current figure; “close 3” closes Figure 3
>> close all % close all figures
 Plotting
Surface Plot

>> Z = peaks; % generate data for plot; peaks returns function values
>> surf(Z) % surface plot of Z

Try these commands also:

>> shading flat
>> shading interp
>> shading faceted
>> grid off
>> axis off
>> colorbar
>> colormap(‘winter’)
>> colormap(‘jet’)
 Plotting
Contour Plots
>> Z = peaks;
>> contour(Z, 20) % contour plot of Z with 20 contours

>> contourf(Z, 20); % with color fill

>> colormap('hot') % map option
>> colorbar % make color bar
 Singular Value Decomposition

For an m´ n matrix A of rank r there exists a factorization

(Singular Value Decomposition = SVD) as follows:
A = USV T

m´m m´n V is n´n

The columns of U are orthogonal eigenvectors of AAT.

The columns of V are orthogonal eigenvectors of ATA.
Eigenvalues l1 … lr of AAT are the eigenvalues of ATA.
s i = li
S = diag (s 1...s r ) Singular values.
 Singular Value Decomposition

n Illustration of SVD dimensions and sparseness

 Singular Value Decomposition

é1 - 1ù
Let A = êê 0 1 úú
êë 1 0 úû
Thus m=3, n=2. Its SVD is

é 0 2/ 6 1/ 3 ùé 1 0 ù
ê úê ú é1 / 2 1/ 2 ù
ê1 / 2 -1/ 6 1 / 3 ú ê0 3úê ú
ê1 / 2 ú ë1/ 2 -1/ 2 û
ë 1/ 6 - 1 / 3 û êë 0 0 úû

Typically, the singular values arranged in decreasing order.