SEAS 2025 Linear Algebra Instructor: Quynh Nguyen

Lecture 2: Vector spaces and Linear transformations

July 2025 TA: Nguyen Huynh

1 Vector spaces
Last time, we familiarized ourselves with the basics: scalars, vectors, matrices, and how to perform
some operations with them. We saw they’re not just abstract numbers but tools used in computer
graphics, machine learning, and much more. Today, we’re going to explore the place where vectors
live, called vector spaces, and then meet some very special and useful types of matrices.

1.1 Rn
 
2
You’ll remember from last time that a vector is an ordered list of numbers, like v = or
3
 
1
u =  0 . When we talk about Rn , we’re defining the set of all possible vectors that have n real
−5
number entries.
To get a sense of what it is, let’s consider simplest cases:
• R1 (or just R) is the space of all vectors with one component, like (x1 ). This is just our
familiar number line!
 
x1
• R is the space of all vectors with two components, like
2 . This corresponds to all the
x2
points on a 2D Cartesian plane (your standard x-y graph). Each vector can be thought of as
an arrow from the origin to the point (x1 , x2 ).
 
x1
• R3 is the space of all vectors with three components, x2 . This is the 3D space we live in,
x3
with x, y, and z axes.
 
x1
 x2 
So, Rn is the generalization of this idea to n dimensions. A vector in Rn looks like x =  . ,
 
 .. 
xn
where each xi is a real number.

1.2 Motivation for high dimensions

You might be thinking, “Okay, 1D, 2D, and 3D make sense, but why would we ever need R4 or R100 ?
We can’t even picture that!” And you’re right, visualizing it directly is tough, but high-dimensional
spaces are very useful. Even if we can’t “see” these high dimensions in the same way we see 3D,

SEAS 2025 Linear Algebra-1

thinking of data as vectors in Rn allows us to use the powerful tools of linear algebra to analyze
and understand complex information.

1.3 Into higher dimensions

So how do we even begin to conceptualize something beyond three dimensions? We can’t physically
point to a “fourth spatial dimension” in our everyday experience. But instead of trying to visualize
it, we can rely on analogy. Imagine a 2D being living on a flat plane (like a character in an old video
game). They can understand length and width, but height (the 3rd dimension) would be a strange,
abstract concept. They might see 3D objects passing through their plane as changing 2D shapes.
We are in a similar position with 4D or higher dimensions. We can see “slices” or “projections” of
these high-dimensional objects onto our 3D space (like a shadow of a 3D object is a 2D shape).
The rules of geometry and vector algebra that work in R2 and R3 (like how to add vectors,
find distances, calculate angles using dot products) can be extended to Rn . We trust the math
to guide our understanding. For example, two vectors in R57 are “orthogonal” (perpendicular) if
their dot product is zero, even if we can’t picture them. We focus on the relationships between
these high-dimensional vectors and how they transform, rather than trying to form a complete
mental image of the space itself. It’s more about using the algebraic framework to work with these
multi-component objects effectively.

2 The Structure of Vector Spaces: Dependence, Span, Basis, and

Inner Product
Now that we have a sense of Rn , let’s explore some fundamental concepts that describe the structure
within these spaces.

2.1 Linear combinations

A linear combination of vectors v1 , v2 , . . . , vk is any vector that can be formed by scaling them
by some scalars c1 , c2 , . . . , ck and adding them up:

w = c1 v1 + c2 v2 + · · · + ck vk

2.2 Linear Dependence and Independence

This is a crucial idea. A set of vectors is linearly dependent if at least one of the vectors in
the set can be written as a linear combination of the others. In simpler terms, one of them is
“redundant” because it doesn’t add any new direction or dimension that the others can’t already
achieve. If no vector in the set can be written as a linear combination of the others, then the set
is linearly independent.
Formally, a set of vectors {v1 , v2 , . . . , vk } is linearly dependent if there exist scalars c1 , c2 , . . . , ck ,
not all zero, such that:

c1 v1 + c2 v2 + · · · + ck vk = 0 (the zero vector)

If the only way to make this sum zero is by choosing c1 = c2 = · · · = ck = 0, then the vectors are
linearly independent.

SEAS 2025 Linear Algebra-2

    
1 2
Let’s consider an example in R2
Let v1 = , v2 = . These are linearly dependent because
2 4
 
1
v2 = 2v1 . We can write 2v1 − v2 = 0, so c1 = 2, c2 = −1 (not all zero). Now consider v1 = ,
0
         
0 1 0 c1 0
v2 = . These are linearly independent. The only way c1 + c2 = = is if
1 0 1 c2 0
c1 = 0 and c2 = 0.
Linearly independent vectors each contribute a unique “direction” to the space they can de-
scribe.

2.3 Span
The span of a set of vectors {v1 , v2 , . . . , vk } is the set of all possible linear combinations of these
vectors. It’s the entire region or space that you can “reach” or “generate” using just those vectors
and scalar multiplication/addition.

Example 1. • The span of a single non-zero vector v in R2 or R3 is a line through the origin
in the direction of v. (e.g., all vectors cv).

• The span of two linearly independent vectors in R3 is a plane passing through the origin.

• If a set of vectors spans the entire space Rn , it means any vector in Rn can be written as a
linear combination of those vectors.

2.4 Basis
Now we combine the ideas of linear independence and span. A basis for a vector space (like Rn )
is a set of vectors that satisfies two conditions:

1. The vectors are linearly independent. (No redundancy)

2. The vectors span the entire space. (They can reach everywhere)

A basis is like the most efficient set of building blocks for a space. You have just enough vectors,
and they’re all pointing in fundamentally different directions.
The standard basis for Rn consists of n vectors, whereeach vector
  has asingle
 1 in one position
1 0
and 0s everywhere else. For R2 , the standard basis is e1 = , e2 = . These are your
0 1
      
 1 0 0 
3
familiar x-axis and y-axis unit vectors. For R , it’s e1 = 0 , e2 = 1 , e3 = 0 . (x, y, z
    
0 0 1
 
unit vectors).
An important fact: every basis for a particular vector space has the same number of vectors.
This number is called the dimension of the vector space. So, the dimension of Rn is n.

2.5 Linear transformations

So far, we’ve been talking about vectors as objects that live in spaces like Rn , and we’ve explored
concepts like how to combine them (span) and how to find the most efficient set of building blocks for

SEAS 2025 Linear Algebra-3

a space (basis). Now, let’s think about how we can systematically change or transform these vectors
and, by extension, the spaces they inhabit. This is where the idea of a linear transformation
comes in.
Imagine you have a rule, let’s call it T , that takes any vector from one vector space (say,
R ) and gives you a new vector, possibly in a different vector space (say, Rm ). This rule T is a
n

“transformation.” But what makes it a “linear” transformation? A transformation T is linear if

it fits with the two main operations we have with vectors: addition and scalar multiplication. In
other words, it maps vectors to vectors. Specifically, two conditions must hold:
1. It preserves addition: If you add two vectors u and v first, and then apply the transfor-
mation T to their sum, you get the same result as if you applied T to u and v individually
and then added the resulting transformed vectors. Mathematically, T (u + v) = T (u) + T (v).
2. It preserves scalar multiplication: If you scale a vector u by a scalar c first, and then
apply the transformation T , you get the same result as if you applied T to u first and then
scaled the resulting transformed vector by c. Mathematically, T (cu) = cT (u).
Think of it like this: a linear transformation doesn’t “warp” space arbitrarily. If we visualize
the space as grid points in the direction of basis vectors, grid lines should remain parallel and
evenly spaced (though their spacing and orientation might change). The origin 0 always stays at
the origin (because T (0) = T (0 · u) = 0 · T (u) = 0). Common examples of linear transformations in
2D or 3D include rotations around the origin, reflections across a line or plane, scaling (stretching
or shrinking), and shearing (like pushing the top of a deck of cards).
Now, here’s the really powerful connection: every linear transformation T from Rn to Rm
can be represented by an m × n matrix! Let’s call this matrix A. Applying the transformation
T to a vector x is then equivalent to simply multiplying the matrix A by the vector x:
T (x) = Ax
This means that the abstract idea of a linear rule can be made concrete through matrix multipli-
cation, which we already learned how to do!
How do we find this magic matrix A for a given linear transformation T ? It’s surprisingly
straightforward. We just need to see what the transformation T does to the standard basis
vectors of the input space Rn . Let e1 , e2 , . . . , en be the standard basis vectors for Rn . The
columns of our matrix A are precisely the transformed versions of these basis vectors:
 
| | |
A = T (e1 ) T (e2 ) . . . T (en )
| | |
So, the first column of A is T (e1 ), the second column is T (e2 ), and so on. Once you have this
matrix A, you can find where any vector x goes under the transformation T just by computing
Ax. This works because any vector x can be written as a linear combination of the basis vectors,
and T preserves these linear combinations.
Example 2. Suppose we have a linear transformation T : R2 → R2 that rotates every vector90◦
1
counterclockwise around the origin. Let’s see what it does to the standard basis vectors e1 =
0
 
0
and e2 = .
1

SEAS 2025 Linear Algebra-4

 • Rotating e1 (which points along the positive ◦
  x-axis) by 90 counterclockwise makes it point
0
along the positive y-axis. So, T (e1 ) = .
1
• Rotating e2 (which points along the positive ◦
  y-axis) by 90 counterclockwise makes it point
−1
along the negative x-axis. So, T (e2 ) = .
0
Therefore, the matrix A for this rotation is:
 
0 −1
A=
1 0
 
2
Now, if you want to rotate any vector, say x = , by 90◦ counterclockwise, you just compute:
3
      
0 −1 2 (0)(2) + (−1)(3) −3
T (x) = Ax = = =
1 0 3 (1)(2) + (0)(3) 2

You can verify that this is the correct result of the rotation.
This link between linear transformations and matrices is fundamental and allows us to use all
our matrix algebra tools to understand and manipulate geometric transformations and other linear
operations.

2.6 Inner Product

In Rn , the most common inner product
 (also
 often called
  the dot product which we might have
u1 v1
touched on) between two vectors u =  ...  and v =  ...  is defined as:
   

un vn
n
X
⟨u, v⟩ = u · v = u1 v1 + u2 v2 + · · · + un vn = ui vi
i=1

The inner product gives us a scalar. It’s useful because it helps us define geometric concepts:
• Length (or Norm): The length of a vector u is ∥u∥ = ⟨u, u⟩ = u21 + u22 + · · · + u2n .
p p

• Angle: The angle θ between two non-zero vectors u and v can be found using ⟨u, v⟩ =
∥u∥∥v∥ cos θ.

• Orthogonality: Two vectors u and v are orthogonal (perpendicular) if their inner product
is zero: ⟨u, v⟩ = 0.
The inner product has some key properties:
1. Symmetry: ⟨u, v⟩ = ⟨v, u⟩.

2. Linearity in the first argument: ⟨u + v, w⟩ = ⟨u, w⟩ + ⟨v, w⟩ and ⟨ku, v⟩ = k⟨u, v⟩ for
a scalar k.

SEAS 2025 Linear Algebra-5

Since it’s symmetric, linearity in the first argument implies linearity in the second argument as
well. This combined property is called bilinearity.
     
u1 v1 w1
Exercise 2.1. Let u = ,v= ,w= be vectors in R2 , and let k be a scalar. Show
u2 v2 w2
explicitly that:

1. ⟨u + v, w⟩ = ⟨u, w⟩ + ⟨v, w⟩

2. ⟨ku, v⟩ = k⟨u, v⟩

3 Some important square matrices

We’re going to narrow our focus now to square real matrices (matrices with the same number
of rows and columns, and all entries are real numbers). There are a few types that have special
structures and properties, making them extremely useful.

3.1 Diagonal Matrices

A diagonal matrix is a square matrix where all the entries off the main diagonal are zero. The
entries on the main diagonal can be anything (including zero).
 
d11 0 . . . 0
 0 d22 . . . 0 
D= .
 
.. .. .. 
 .. . . . 
0 0 . . . dnn
 
3 0 0
Example: 0 −2 0 is a 3 × 3 diagonal matrix.
0 0 5
Diagonal matrices are great because they simplify many operations:

• Multiplying diagonal matrices is just multiplying corresponding diagonal entries.

• The powers of a diagonal matrix Dk are found by simply raising each diagonal entry to the
power k. (show this!)

• Their eigenvalues are just the diagonal entries (we’ll learn about eigenvalues later).

3.2 Orthonormal Matrices

An orthonormal matrix Q is a square matrix with a useful property: its transpose is also its
inverse.
QT Q = QQT = I (where I is the identity matrix)
This has some important consequences. Firstly, the columns of an orthonormal matrix form an
orthonormal basis. This means each column vector has a length of 1 (they are ”normal”), and
they are all mutually orthogonal (their dot product is 0). The same is true for the rows. As

SEAS 2025 Linear Algebra-6

a result, geometric intuition tells us that when we multiply a vector by an orthogonal matrix, it
corresponds to a rotation around the origin or a reflection, or a combination of these.
Crucially, orthogonal matrices preserve lengths of vectors and angles between vectors.
If you transform vectors x and y by Q to get Qx and Qy, the length of Qx is the same as x, and
the angle between Qx and Qy is the same as the angle between x and y.
Because angles are preserved, the inner product itself is preserved.

Exercise 3.1 (Inner product preservation by orthogonal matrices). Let Q be an n × n orthogonal

matrix, and let x, y be vectors in Rn . Prove that ⟨Qx, Qy⟩ = ⟨x, y⟩.
Hint: Remember that the inner product ⟨u, v⟩ can be written as uT v. Also, recall from Lecture
1 the transpose property (AB)T = BT AT , and the definition QT Q = I.

3.3 Symmetric Matrices

A symmetric matrix is a square matrix A that is equal to its own transpose:

A = AT

This means that the entry in the i-th row and j-th column is the same as the entry in the j-th row
and i-th column (aij = aji ). The matrix is symmetric about its main diagonal.
 
1 7 −2
Example 3. A =  7 5 0 . If you flip this across its main diagonal, it stays the same.
−2 0 3
Symmetric matrices can be found in many areas of mathematics. For example, covariance ma-
trices in statistics are symmetric. They describe how different variables vary together. Symmetric
matrices also describe energy functions in physics.

SEAS 2025 Linear Algebra-7