Strassen’s Matrix
Multiplication
Matrix
Consider two n x n matrices A and B
Multiplication
Recall that the matrix product C = AB of two n x n matrices is
defined as the n x n matrix that has the coefficient
c kl = ∑m akm bml
in row k and column l, where the sum ranges over the
integers from 1 to n; the scalar product of the kth row of a
with the l t h column of B.
The straightforward algorithm uses O(n3) scalar
operations. Can we do better?
Basic Matrix Multiplication
Suppose we want to multiply two matrices of size N x N: for
example A x B = C.
C11 = a11b11 + a12b21
C12 = a11b12 + a12b22
C21 = a21b11 + a22b21
C22 = a21b12 + a22b22
2x2 matrix multiplication can be accomplished in
8 multiplication.(2log28 =23)
Basic Matrix Multiplication
void matrix_mult (){
algorithm
for (i = 1; i <= N; i++) {
for (j = 1; j <= N; j++) {
compute Ci,j;
}
N
}} Ci , j ai , k bk , j
Time analysis k 1
N N N
Thus T ( N ) c cN 3 3
O ( N )
i 1 j 1 k 1
Idea: Use Divide and
Conquer
The divide and conquer paradigm is important general
technique for designing algorithms. In general, it follows
the steps:
- divide the problem into subproblems
- recursively solve the subproblems
- combine solutions to subproblems to get solution to
original problem
Divide-and-Conquer
Let write the product A B = C as
follows:
A0 A1 B0 B1 A0B0+A1B2 A0B1+A1B3
=
A2 A3 B2 B3 A2B0+A3B2 A2B1+A3B3
• Divide matrices A and B into four submatrices each
• We have 8 smaller matrix multiplications and 4 additions. Is it
faster?
Divide-and-Conquer
Let us investigate this recursive version of the
matrix multiplication.
Since we divide A, B and C into 4 submatrices each,
we can compute the resulting matrix C by
• 8 matrix multiplications on the submatrices of A
and B,
• plus (n2) scalar operations
Divide-and-Conquer
• Running time of recursive version of straightfoward
algorithm is
T(n) = 8T(n/2) + (n2) and T(2) = (1)
where T(n) is running time on an n x n matrix
• Master theorem gives us:
T(n) = (n3)
• Can we do
fewer recursive
calls (fewer
Strassen’s Matrix
Multiplication
A B = C
.
A11 A12 B11 B12 C11 C12
A21 A22 B21 B22 C21 C22
=
P1 = (A11+ A22)(B11+B22)
P2 = (A21 + A22) * B11 C 11 = P1 + P4 - P5 + P7
P3 = A11 * (B12 - B22) C 12 = P3 + P5
P4 = A22 * (B21 - B11) C 21 = P2 + P4
P5 = (A11 + A12) * B22 C 22 = P1 + P3 - P2 + P6
P6 = (A21 - A11) * (B11 + B12)
P7 = (A12 - A22) * (B21 + B22)
Strassen's Matrix
Multiplication
• Strassen found a way to get all the required information
with only 7 matrix multiplications, instead of 8.
• Strassen showed that 2x2 matrix multiplication can be
accomplished in 7 multiplication and 18 additions or
subtractions. .(2log27 =22.807)
• Recurrence for new algorithm is
• T(n) = 7T(n/2) + (n2)
Solving the Recurrence Relation
Applying the Master Theorem to
T(n) = a T(n/b) + f(n)
with a=7, b=2, and f(n)=(n2).
Since f(n) = O(nlogb(a)-) =
O(nlog2(7)-), case a) applies and
we get
T(n)= (nlogb(a)) = (nlog2(7)) =
Discussion of Strassen's
Algorithm
• Not always practical
• constant factor is larger than for naïve method
• specially designed methods are better on sparse matrices
• issues of numerical (in)stability
• recursion uses lots of space
• Not the fastest known method
• Fastest known is O(n2.3727) [Winograd-Coppersmith algorithm
improved by V. Williams]
• Best known lower bound is (n2)
