Matrix Tricks

August 18, 2019

1 Introduction
This document is my informal reference sheet for linear algebra topics that I should stop forgetting.
Remember that the whole idea of this stuff is that Ax = b. Given linear equations of the form

ax + by = c (1)

and

dx + ey = f (2)

we construct matrices as a shorthand for their coefficients:

[ ][ ] [ ] [ ] [ ]
a b x a b c
=x +y = . (3)
d e y d e f

See [https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-
lectures/lecture-1-the-geometry-of-linear-equations/].

[1]: v = [i for i=1:3]

[1]: 3-element Array{Int64,1}:

1
2
3

[2]: m = [3(i-1) + j for i=1:3, j=1:3]

[2]: 3×3 Array{Int64,2}:

1 2 3
4 5 6
7 8 9

[3]: ones(Int,1,3)

[3]: 1×3 Array{Int64,2}:

1 1 1

1
 2 Rotate or transpose a matrix
According to [https://math.stackexchange.com/questions/1945329/can-you-transpose-a-matrix-
using-matrix-multiplication] and [https://math.stackexchange.com/questions/2816073/does-there-
exist-2-matricies-such-that-they-can-be-used-to-transpose-any-n-by-n] this cannot be done with a
cross product. For this, you need the help of your programming language.
[4]: transpose(m)

[4]: 3×3 LinearAlgebra.Transpose{Int64,Array{Int64,2}}:

1 4 7
2 5 8
3 6 9

[5]: rotl90(m)

[5]: 3×3 Array{Int64,2}:

3 6 9
2 5 8
1 4 7

[6]: rotl90(rotl90(m)) == rot180(m)

[6]: true

[7]: rotl90(rotl90(rotl90(m))) == rotr90(m)

[7]: true

3 Copy vectors
3.1 Duplicate a vector as columns
[8]: v * ones(Int,1,3)

[8]: 3×3 Array{Int64,2}:

1 1 1
2 2 2
3 3 3

3.2 Duplicate a vector as rows

[9]: ones(Int,3,1) * transpose(v)

[9]: 3×3 Array{Int64,2}:

1 2 3
1 2 3

2
 1 2 3

3.3 Extract column 1 from a matrix

[10]: m * [1, 0, 0]

[10]: 3-element Array{Int64,1}:

1
4
7

3.4 Extract row 2 from a matrix

[11]: [0 1 0;] * m

[11]: 1×3 Array{Int64,2}:

4 5 6

4 Substitutions
4.1 Swap x and y of a vector
[12]: [0 1 0; 1 0 0; 0 0 1] * v

[12]: 3-element Array{Int64,1}:

2
1
3

4.2 Swap rows 1 and 2 of a matrix

[13]: [0 1 0; 1 0 0; 0 0 1] * m

[13]: 3×3 Array{Int64,2}:

4 5 6
1 2 3
7 8 9

4.3 Reverse rows

[14]: r = [0 0 1; 0 1 0; 1 0 0]

[14]: 3×3 Array{Int64,2}:

0 0 1
0 1 0
1 0 0

3
[15]: r * m

[15]: 3×3 Array{Int64,2}:

7 8 9
4 5 6
1 2 3

4.4 Reverse columns

[16]: m * r

[16]: 3×3 Array{Int64,2}:

3 2 1
6 5 4
9 8 7

4.5 Swap columns 1 and 2 of a matrix

(Remember when they said matrix multiplciation is not commutative?)

[17]: m * [0 1 0; 1 0 0; 0 0 1]

[17]: 3×3 Array{Int64,2}:

2 1 3
5 4 6
8 7 9

5 Permutations
Generically, all of the above exchanges are called permutations. For a n × n matrix there are n!
such permutation (“P ”) matrices. For n = 3, these are:

 
1 0 0
0 1 0
0 0 1
 
0 1 0
1 0 0
0 0 1
 
0 0 1
0 1 0
1 0 0
 
1 0 0
0 0 1
0 1 0

4
  
0 0 1
1 0 0
0 1 0
 
0 1 0
0 0 1
1 0 0

For all permutation matrices,

P −1 = P T . (4)

Also notice that these matrices compose to each other.

6 Insertions
6.1 Append an empty row to the bottom
[18]: [1 0 0; 0 1 0; 0 0 1; 0 0 0] * m

[18]: 4×3 Array{Int64,2}:

1 2 3
4 5 6
7 8 9
0 0 0

6.2 Insert an empty row to the top

[19]: [0 0 0; 1 0 0; 0 1 0; 0 0 1] * m

[19]: 4×3 Array{Int64,2}:

0 0 0
1 2 3
4 5 6
7 8 9

6.3 Append an empty column to the right

[20]: m * [1 0 0 0; 0 1 0 0; 0 0 1 0]

[20]: 3×4 Array{Int64,2}:

1 2 3 0
4 5 6 0
7 8 9 0

5
 6.4 Insert an empty column to the left
[21]: m * [0 1 0 0; 0 0 1 0; 0 0 0 1]

[21]: 3×4 Array{Int64,2}:

0 1 2 3
0 4 5 6
0 7 8 9

7 Deletions
7.1 Clear rows 2 and 3 from a matrix
[22]: [1 0 0; 0 0 0; 0 0 0] * m

[22]: 3×3 Array{Int64,2}:

1 2 3
0 0 0
0 0 0

7.2 Clear columns 2 and 3 from a matrix

[23]: m * [1 0 0; 0 0 0; 0 0 0]

[23]: 3×3 Array{Int64,2}:

1 0 0
4 0 0
7 0 0

7.3 Drop the right column

[24]: m * [1 0; 0 1; 0 0]

[24]: 3×2 Array{Int64,2}:

1 2
4 5
7 8

7.4 Drop the center column

[25]: m * [1 0; 0 0; 0 1]

[25]: 3×2 Array{Int64,2}:

1 3
4 6
7 9

6
 7.5 Drop the left column
[26]: m * [0 0; 1 0; 0 1]

[26]: 3×2 Array{Int64,2}:

2 3
5 6
8 9

7.6 Drop the bottom row

[27]: [1 0 0; 0 1 0] * m

[27]: 2×3 Array{Int64,2}:

1 2 3
4 5 6

7.7 Drop the center row

[28]: [1 0 0; 0 0 1] * m

[28]: 2×3 Array{Int64,2}:

1 2 3
7 8 9

7.8 Drop the top row

[29]: [0 1 0; 0 0 1] * m

[29]: 2×3 Array{Int64,2}:

4 5 6
7 8 9

You can achieve the same results by composing transposition with a single drop function. For
example, transpose rows 2 and 3, then drop row 3. This has the same effect as dropping the center
row. Remember, order of operations matters.
[30]: [1 0 0; 0 1 0] * [1 0 0; 0 0 1; 0 1 0] * m

[30]: 2×3 Array{Int64,2}:

1 2 3
7 8 9

Another way to think about this is that the “drop row 3” and “transpose rows 2 and 3” matrices
compose, which is identical to the “drop row 2” matrix.
[31]: [1 0 0; 0 1 0] * [1 0 0; 0 0 1; 0 1 0]

7
[31]: 2×3 Array{Int64,2}:
1 0 0
0 0 1

8 Affine Transforms (2D)

Suppose you begin with Bv. Then an affine transform ABv occurs in global coordinate
space, and an affine transform BCv occurs in the object’s local coordinate space. See
[http://math.hws.edu/graphicsbook/c2/s3.html].

8.1 Scaling
 
a 0 0
Sa,b = 0 b 0 (5)
0 0 1

8.2 Rotation
We use Ptolemy’s Sum and Difference Formulae

cos α ± β = cos α cos β ∓ sin α sin β (6)

sin α ± β = cos α sin β ± sin α cos β (7)

in order to increase the rotation of a vector from some initial angle α by β:

       
k · cos α k · cos α cos α + β k · cos α + β
Rβ  k · sin α  = Rβ k  k · sin α  = k  sin α + β  =  k · sin α + β  . (8)
1 1/k 1/k 1

The matrix which delivers this behavior is

 
cos Θ − sin Θ 0
RΘ =  sin Θ cos Θ 0 . (9)
0 0 1

8.3 Translation
 
1 0 a
Ta,b = 0 1 b  (10)
0 0 1

9 Gaussian Elimination
Augment the coefficient matrix with the b column vector.

8
      
a11 a12 ... a1n b1 a11 a12 ... a1n b1
 a21 a22 ... a2n   b2   a21 a22 ... a2n b2 
     
Ax = b →  . .. .. ..  =  ..  →  .. .. .. .. ..  (11)
 .. . . .   .   . . . . .
am1 am2 . . . amn bm am1 am2 . . . amn bn

Subtract rows from one another, rearranging rows as needed, until the matrix is in upper triangular
form. See [http://mathworld.wolfram.com/GaussianElimination.html].

[32]: g = [9 3 4 7; 4 3 4 8; 1 1 1 3]

[32]: 3×4 Array{Int64,2}:

9 3 4 7
4 3 4 8
1 1 1 3

The bottom row is very convenient, so switch the first and third rows.
[33]: [0 0 1; 0 1 0; 1 0 0] * g

[33]: 3×4 Array{Int64,2}:

1 1 1 3
4 3 4 8
9 3 4 7

Subtract four times the first row from the second.

[34]: [1 0 0; -4 1 0; 0 0 1] * [0 0 1; 0 1 0; 1 0 0] * g

[34]: 3×4 Array{Int64,2}:

1 1 1 3
0 -1 0 -4
9 3 4 7

This gives a “nice” result of one variable that we could have used to reduce our problem immediately,
but we would still have to solve for x and z.
Subtract nine times the first row from the third.
[35]: [1 0 0; 0 1 0; -9 0 1] * [1 0 0; -4 1 0; 0 0 1] * [0 0 1; 0 1 0; 1 0 0] * g

[35]: 3×4 Array{Int64,2}:

1 1 1 3
0 -1 0 -4
0 -6 -5 -20

Subtract six times the second row from the third.

[36]: [1 0 0; 0 1 0; 0 -6 1] * [1 0 0; 0 1 0; -9 0 1] * [1 0 0; -4 1 0; 0 0 1] * [0 0␣
,→1; 0 1 0; 1 0 0] * g

9
[36]: 3×4 Array{Int64,2}:
1 1 1 3
0 -1 0 -4
0 0 -5 4

Now we see −5z = 4, −y + 0z = −4, and x + y + z = 3, which give z = −4/5, y = 4, and x = −1/5.

9.1 Left Division Operator

Julia contains a built-in “left division operator” (\) for this purpose.

[37]: round.([9 3 4; 4 3 4; 1 1 1] \ [7; 8; 3],digits=2)

[37]: 3-element Array{Float64,1}:

-0.2
4.0
-0.8

9.2 Units
This is a general reminder to be very cautious with units. Example TMP at
[http://linear.ups.edu/html/section-SSLE.html] gives a system of linear equations

 
7 kg/batch of raisins 6 kg/batch of raisins 2 kg/batch of raisins
 6 kg/batch of peanuts 4 kg/batch of peanuts 5 kg/batch of peanuts 
2 kg/batch of chocolate 5 kg/batch of chocolate 8 kg/batch of chocolate
   
b batches of bulk trail mix 380 kg of raisins
s batches of standard trail mix =  500 kg of peanuts  (12)
f batches of fancy trail mix 620 kg of chocolate

This is straightfoward to solve, but the question is looking for answers in kilograms.
[38]: round.([7 6 2; 6 4 5; 2 5 8] \ [380; 500; 620])

[38]: 3-element Array{Float64,1}:

20.0
20.0
60.0
 
b

To get to the answer, we needed to know 15 kg = 1 batch, so s  should actually be 15× the
f
amount calculated.
Again, what we solved was

[ ][ ] [ ]
kg/batch batches = kg (13)

10
 when the question really needs

[ ][ ] [ ]
units kg = kg . (14)

[39]: round.(([7 6 2; 6 4 5; 2 5 8]) / 15 \ [380; 500; 620])

[39]: 3-element Array{Float64,1}:

300.0
300.0
900.0

10 Inversions
10.1 Not all matrices are invertible
Let us −1
[ assume,
] for the sake[ of contradiction,
] [ ] [ ] that there exists some matrix [ ]A for the
[ ] matrix
1 3 1 3 3 0 3 0
A= . Observe that = . If A−1 exists, then A−1 A = A−1 , which
2 6 2 6 −1 0 −1 0
[ ] [ ]
3 0
implies = . This is a contradiction, therefore it must not be true that all matrices are
−1 0
invertible.
(The key to this proof is to find some x such that Ax = 0).
See [https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-
lectures/lecture-3-multiplication-and-inverse-matrices/].

10.2 Invert a Matrix

[ ] [ ]
1 3 a c
Let A = . We want to find A−1 = such that AA−1 = I. Then
2 7 b d

[ ][ ] [ ]
1 3 a c 1 0
= (15)
2 7 b d 0 1
[ ][ ] [ ]
1 3 a 1
= (16)
2 7 b 0
[ ][ ] [ ]
1 3 c 0
= (17)
2 7 d 1

which produces the linear equations

1a + 3b = 1 (18)
2a + 7b = 0 (19)
1c + 3d = 0 (20)
2c + 7d = 1 (21)

11
 which we can solve!
[40]: [1 3; 2 7] \ [1; 0]

[40]: 2-element Array{Float64,1}:

7.0
-2.0

[41]: [1 3; 2 7] \ [0; 1]

[41]: 2-element Array{Float64,1}:

-3.0
1.0

[42]: [1 3; 2 7] * hcat([7; -2], [-3, 1])

[42]: 2×2 Array{Int64,2}:

1 0
0 1

10.3 Gauss-Jordan
Gauss-Jordan lets you do this by hand by performing elimination “downwards” and then again
“upwards” with your coefficient matrix augumented with a complete identity. It looks like:

[ ]
a c 1 0 [ ] [ ]
E = E A I = I A−1 (22)
b d 0 1

12