Design and Analysis of Algorithm
Divide and Conquer strategy
(Matrix Multiplication
by Strassen’s Algorithm)
Lecture -16
Overview
• Learn the implementation techniques
of “divide and conquer” in the context
of the Strassen’s Matrix multiplication
with analysis.
• 𝐶𝑜𝑛𝑣𝑒𝑛𝑡𝑖𝑜𝑛𝑎𝑙 𝑠𝑡𝑟𝑎𝑡𝑒𝑔𝑦 ⇒ Ο(𝑛3 ).
• 𝐷𝑖𝑣𝑖𝑑𝑒 𝑎𝑛𝑑 𝐶𝑜𝑞𝑢𝑒𝑠 𝑠𝑡𝑟𝑎𝑡𝑒𝑔𝑦 ⇒ Ο(𝑛3 ).
• 𝑆𝑡𝑟𝑎𝑠𝑠𝑒𝑛′ 𝑠 𝑠𝑡𝑟𝑎𝑡𝑒𝑔𝑦 ⇒ Ο(𝑛2.81 ).
Matrix Multiplication
• Problem definition:
Matrix Multiplication
• Conventional strategy:
Matrix Multiplication
• Question is
𝐼𝑠 Θ(𝑛3 ) is the best or we can multiply
the matrix in 𝜊 𝑛3 time?
(i.e. can we solve it in < Θ(𝑛3 ) )
• Let’s see with Divide and Conquer
strategy………
Matrix Multiplication
• Divide-and-conquer strategy :
➢ As with the other divide-and-conquer algorithms, assume that n is
a power of 2(i.e. 2𝑛 ).
𝒏 𝒏
➢ Partition each of A,B, C into four 𝟐 × 𝟐 matrices:
𝐴11 𝐴12 𝐵11 𝐵12 𝐶 𝐶12
𝐴= ,𝐵 = , 𝐶 = 11
𝐴21 𝐴22 𝐵21 𝐵22 𝐶21 𝐶22
For multiplication we can write 𝐶 = 𝐴 . 𝐵 as
𝐶11 𝐶12 𝐴 𝐴12 𝐵11 𝐵12
𝐶21 𝐶22
= 11
𝐴21 𝐴22
. 𝐵21 𝐵22
Matrix Multiplication
• Divide-and-conquer strategy :
For multiplication we can write 𝐶 = 𝐴 . 𝐵 as
𝐶11 𝐶12 𝐴 𝐴12 𝐵11 𝐵12
𝐶21 𝐶22
= 11
𝐴21 𝐴22
. 𝐵21 𝐵22
Which create four equations. They are
𝐶11 = 𝐴11 . 𝐵11 + 𝐴12 . 𝐵21
𝐶12 = 𝐴11 . 𝐵12 + 𝐴12 . 𝐵22
𝐶21 = 𝐴21 . 𝐵11 + 𝐴22 . 𝐵21
𝐶22 = 𝐴21 . 𝐵12 + 𝐴22 . 𝐵22
𝒏 𝒏
Each of these equations multiplies two 𝟐 × 𝟐 matrices and then adds their
𝒏 𝒏
× products.
𝟐 𝟐
Matrix Multiplication
• Divide-and-conquer strategy :
By using the equations of previous slide we ca write the Divide and conquer
algorithm.
REC-MAT-MULT (A, B, n)
Let C be a n x n matrix
If n== 1
C11 = A11 x B11
𝒏 𝒏
else partition A,B, and C into × submatrices.
𝟐 𝟐
C11 = REC−MAT−MULT A11 , B11 , NΤ2 + REC−MAT−MULT A12 , B21 , NΤ2
C12 = REC−MAT−MULT A11 , B12 , NΤ2 + REC−MAT−MULT A12 , B22 , NΤ2
C21 = REC−MAT−MULT A21 , B11 , NΤ2 + REC−MAT−MULT A22 , B21 , NΤ2
C22 = REC−MAT−MULT A21 , B12 , NΤ2 + REC−MAT−MULT A22 , B22 , NΤ2
Return C
Matrix Multiplication
• Analysis of Divide-and-conquer strategy :
𝒏 𝒏
Let T(n) be the time to multiply two × matrices.
𝟐 𝟐
Base Case: n=1. Perform one scalar multiplication: Θ(1).
Recursive Case: n>1
• Dividing takes b Θ(1) time, using index calculations.
𝒏 𝒏
• Conquering makes 8 recursive calls, each multiplying × matrices.
𝟐 𝟐
(i.e. 8𝑇 𝑛Τ2 )
𝒏 𝒏
• Combining Takes Θ 𝑛2 time to add × matrices four items.
𝟐 𝟐
Hence the Recurrence is
Θ 1 𝑖𝑓 𝑛 = 1
𝑇 𝑛 =ቐ 𝑛
8𝑇 + Θ 𝑛2 𝑖𝑓𝑛 > 1
2
The Complexity is Θ(𝑛3 ) (Apply Master Method)
Matrix Multiplication
• Analysis of Divide-and-conquer strategy :
𝒏 𝒏
Let T(n) be the time to multiply two × matrices.
𝟐 𝟐
Base Case: n=1. Perform one scalar multiplication: Θ(1).
Recursive Case: n>1
• Dividing takes b Θ(1) time, using index calculations.
𝒏 𝒏
• Conquering makes 8 recursive calls, each multiplying × matrices.
𝟐 𝟐
(i.e. 8𝑇 𝑛Τ2 )
𝒏 𝒏
• Combining Takes Θ 𝑛2 time to add × matrices four items.
𝟐 𝟐
Hence the Recurrence is
Θ 1 𝑖𝑓 𝑛 = 1
𝑇 𝑛 =ቐ 𝑛
8𝑇 + Θ 𝑛2 𝑖𝑓𝑛 > 1
2
The Complexity is Θ(𝑛3 ) (Apply Master Method, or Recursive Method, or
Iterative Method)
Matrix Multiplication
• Analysis of Divide-and-conquer strategy :
𝒏 𝒏
Let T(n) be the time to multiply two × matrices.
𝟐 𝟐
Base Case: n=1. Perform one scalar multiplication: Θ(1).
Recursive Case: n>1
• Dividing takes b Θ(1) time, using index calculations.
𝒏 𝒏
• Conquering makes 8 recursive calls, each multiplying × matrices.
𝟐 𝟐
(i.e. 8𝑇 𝑛Τ2 )
𝒏 𝒏
• Combining Takes Θ 𝑛2 time to add × matrices four items.
𝟐 𝟐
Hence the Recurrence is
Θ 1 𝑖𝑓 𝑛 = 1
𝑇 𝑛 =ቐ 𝑛
8𝑇 + Θ 𝑛2 𝑖𝑓𝑛 > 1
2
The Complexity is Θ(𝑛3 ) (Apply Master Method, or Recursive Method, or
Iterative Method)
Matrix Multiplication
• Strassen’s strategy :
The Idea:
➢ Make the recursion tree less bushy.
➢ Perform only 7(seven) recursive multiplications
of n/2 x n/2 matrices, rather than 8(Eight).
Matrix Multiplication
• Strassen’s strategy :
The Algorithm:
1. As in the recursive method, partition each of the
𝒏 𝒏
matrices into four 𝟐 × 𝟐 submatrices. Time: Θ 1
2. Compute 7 matrix products 𝑃, 𝑄, 𝑅, 𝑆, 𝑇, 𝑈, 𝑉 for each
𝒏 𝒏
𝟐
× 𝟐
.
𝒏 𝒏
3. Compute 𝟐 × 𝟐 submatrices of C by adding and
subtracting various combinations of the 𝑃𝑖 . Time:
Θ(𝑛2 ) .
Matrix Multiplication
• Strassen’s strategy :
Details of Step 2:
Compute 7 matrix products:
P=(𝐀𝟏𝟏 + 𝐀𝟐𝟐 ). (𝑩𝟏𝟏 +𝑩𝟐𝟐 ) T=(𝑨𝟏𝟏 + 𝑨𝟏𝟐 ). 𝑩𝟐𝟐
Q=(𝑨𝟐𝟏 + 𝑨𝟐𝟐 ). 𝑩𝟏𝟏 U= (𝑨𝟐𝟏 − 𝑨𝟏𝟏 ).(𝑩𝟏𝟏 + 𝑩𝟏𝟐 )
R= 𝐀𝟏𝟏 .(𝑩𝟏𝟐 − 𝑩𝟐𝟐 ) V=(𝑨𝟏𝟐 − 𝑨𝟐𝟐 ).(𝑩𝟐𝟏 + 𝑩𝟐𝟐 )
S= 𝑨𝟐𝟐 .(𝑩𝟐𝟏 − 𝑩𝟏𝟏 )
Matrix Multiplication
• Strassen’s strategy :
Details of Step 3:
Compute C with 4 adding and subtracting :
𝑪𝟏𝟏 = 𝑷 + 𝑺 − 𝑻 + 𝑽
𝑪𝟏𝟐 = 𝑹 + 𝑻
𝑪𝟐𝟏 = 𝑸 + 𝑺
𝑪𝟐𝟐 = 𝑷 + 𝑹 − 𝑸 + 𝑼
Matrix Multiplication
• Strassen’s strategy :
Analysis:
The Recurrence is
Θ 1 𝑖𝑓 𝑛 = 1
𝑇 𝑛 =ቐ 𝑛
7𝑇 + Θ 𝑛2 𝑖𝑓𝑛 > 1
2
The Complexity is Θ(𝑛log2 7 ) = Θ(𝑛2.81 ) (By using
Master Method)
Matrix Multiplication
Example 1
• Compute Matrix multiplication of the following two
matrices with the help of Strassen’s strategy
1 2 5 6
𝐴= and 𝐵 =
3 4 7 8
Matrix Multiplication
Example 1
• Compute Matrix multiplication of the following two
matrices with the help of Strassen’s strategy
1 2 5 6
𝐴= and 𝐵 =
3 4 7 8
Ans:
𝐴11 =1, 𝐴12 =2, 𝐴21 =3, and 𝐴22 =4
𝐵11 =5, 𝐵12 =6, 𝐵21 =7, and 𝐵22 =8
Matrix Multiplication
Example 1
Calculate the value of P, Q, R, S, T, U and V
P=(A11 + A22 ). (𝐵11 +𝐵22 ) = (1+4)(5+8)=5 x 13=65
Q=(𝐴21 + 𝐴22 ). 𝐵11 = (3+4)5=7 x 5=35
R= A11 .(𝐵12 − 𝐵22 )=1(6-8)=1 x -2=-2
S= 𝐴22 .(𝐵21 − 𝐵11 ) = 4(7-5)=4 x 2=8
T=(𝐴11 + 𝐴12). 𝐵22 =(1+2)8=3 x 8=24
U= (𝐴21 − 𝐴11 ).(𝐵11 + 𝐵12 )=(3-1)(5+6)=2 x 11=22
V=(𝐴12 − 𝐴22 ).(𝐵21 + 𝐵22 )=(2-4)(7+8)=-2 x 15=-30
Matrix Multiplication
Example 1
Compute 𝑪𝟏𝟏 , 𝑪𝟏𝟐 , 𝑪𝟐𝟏 , 𝒂𝒏𝒅 𝑪𝟐𝟐 :
𝐶11 = 𝑃 + 𝑆 − 𝑇 + 𝑉= 65+8-24-30=19
𝐶12 = 𝑅 + 𝑇=-2+24=22
𝐶21 = 𝑄 + 𝑆=35+8=43
𝐶22 = 𝑃 + 𝑅 − 𝑄 + 𝑈=65-2-35+22=50
Hence,
19 22
𝐶=
43 50
Matrix Multiplication
Example 2
Compute Matrix multiplication of the following two matrices with
the help of Strassen’s strategy.
4 2 0 1 2 1 3 2
𝐴= 3 1 2 5 ,B= 5 4 2 3
3 2 1 4 1 4 0 2
5 2 6 7 3 2 4 1
Matrix Multiplication
Example 2
First we partition the input matrices into sub matrices as shown
below:
4 2 0 1 2 1 3 2
𝐴11 = , 𝐴12 = 𝐵11 = , 𝐵12 =
3 1 2 5 5 4 2 3
3 2 1 4 1 4 0 2
𝐴21 = , 𝐴22 = 𝐵21 = , 𝐵22 =
5 2 6 7 3 2 4 1
Matrix Multiplication
Example 2
Calculate the value of P, Q, R, S, T, U and V
𝑃 = (A11 + A22 ). (𝐵11 +𝐵22 )
5 6 2 3
=
9 8 9 5
Q=(𝐴21 + 𝐴22 ). 𝐵11
64 45
=
90 67 4 6 2 1
=
11 9 5 4
38 28
=
67 47
Matrix Multiplication
Example 2
R= A11 .(𝐵12 − 𝐵22 )
4 2 3 0
=
3 1 −2 2
8 4
=
7 2 S= 𝐴22 .(𝐵21 − 𝐵11 )
1 4 −1 3
=
6 7 −2 −2
−9 −5
=
−20 4
Matrix Multiplication
Example 2
T=(𝐴11 + 𝐴12 ). 𝐵22 U= (𝐴21 − 𝐴11 ).(𝐵11 + 𝐵12 )
4 3 0 2 −1 0 5 3
= =
5 6 4 1 2 1 7 7
12 11 −5 −3
= =
24 16 17 13
V=(𝐴12 − 𝐴22 ).(𝐵21 + 𝐵22 )
−1 −3 1 6
=
−4 −2 7 3
−22 −15
=
−18 −30
Matrix Multiplication
Example 2
Now, Compute 𝑪𝟏𝟏 , 𝑪𝟏𝟐 , 𝑪𝟐𝟏 , 𝒂𝒏𝒅 𝑪𝟐𝟐 :
𝐶11 = 𝑃 + 𝑆 − 𝑇 + 𝑉
64 45 −9 −5 12 11 −22 −15 21 14
𝐶11 = + - + =
90 67 −20 4 24 16 −18 −30 28 25
𝐶12 = 𝑅 + 𝑇
8 4 12 11 20 15
𝐶12 = + =
7 2 24 16 31 18
Matrix Multiplication
Example 2
Now, Compute 𝑪𝟏𝟏 , 𝑪𝟏𝟐 , 𝑪𝟐𝟏 , 𝒂𝒏𝒅 𝑪𝟐𝟐 :
𝐶21 = 𝑄 + 𝑆
38 28 −9 −5 29 23
𝐶21 = + =
67 47 −20 4 47 51
𝐶22 = 𝑃 + 𝑅 − 𝑄 + 𝑈
64 45 8 4 38 28 −5 −3 29 18
𝐶22 = + - + =
90 67 7 2 67 47 17 13 47 35
Matrix Multiplication
Example 2
So the values of 𝑪𝟏𝟏 , 𝑪𝟏𝟐 , 𝑪𝟐𝟏 , 𝒂𝒏𝒅 𝑪𝟐𝟐 are:
21 14 20 15 29 23 29 18
𝐶11 = , 𝐶12 = , 𝐶21 = and 𝐶22 =
28 25 31 18 47 51 47 35
Hence the resultant Matrix C is =
21 14 20 15
𝐶11 𝐶12
𝐶= = 28 25 31 18
𝐶21 𝐶22 29 23 29 18
47 51 47 35
