Design & Analysis of Algorithms

Dr Anwar Ghani

Semester: Fall 2017

Department of Computer Science & Software Engineering,

International Islamic University, Islamabad.
Quick Sort
 Quick Sort
Topics
ƒ Quick Sort Algorithm

- Pivot selection

- Partitioning

- Pseudo code for Partitioning and Sorting

- Visualization of Quick Sort Algorithm

ƒ Analysis of Quick Sort

- Best case

- Worst case

- Average case
 Quick Sort
Phases
The quick sort is a divide-and-conquer procedure. It works in two phases, which are
referred to as partitioning and recursive call
Phase #1: In this phase an array element is chosen as a pivot The array is partitioned into two
subarrays such that all elements in the left partition are less than or equal to the pivot, and all
elements in right partition are greater than or equal to the pivot. An illustration of
partitioning is as follows. The left-most key 51 is selected as the pivot.

51 67 11 78 90 13 66 80 92 30 40 59 68 71 65 23
pivot

23 11 13 30 40 51 67 78 90 66 80 92 59 68 71 65

Unsorted partition Unsorted partition

Phase #2: In this phase, the left and right subarrays are sorted recursively

11 13 23 30 40 51 59 65 66 67 68 71 78 80 90 92

Sorted partition Sorted partition

 Quick Sort Partitioning
Pivot Selection
ƒ Choice of pivot influences the performance of quick sort algorithm

ƒ Several strategies are used for the selection of pivot. Some common approaches are:
(a) Left-most key
(b) Right-most key
(c) Middle key
(d) Key in a random location
(e) Median of three randomly selected keys

ƒ Some of the choices lead to bad partitioning, in which case the left and the right partitioned
are not evenly balanced. Consequently, the performance of quick sort is degraded.
 Quick Sort Partitioning
Schemes
• As discussed, the partitioning is based on the choice of pivot. Once a pivot has been
selected, the array is scanned to partition it into two subarrays. There are several schemes for
scanning. Generally, two pointers are used for this purpose

• In one scheme, both the pointers are moved in the left-to-right or right-to-left direction to
examine the stored keys and compare them with the pivot. The keys are swapped depending
upon the outcome of comparison.

• Another approach moves the pointer, alternately, from right-to-left and left-to-right until a
key smaller than and a key larger than pivot are found. The keys are then swapped. This
method is referred to as Hoare Partitioning
 Hoare Partitioning
Algorithm
ƒ A common partitioning procedure, due originally to Hoare , is as follows

Step# 1: Select left-most element as pivot Define two variables i, j to scan the array
Use variable i as left pointer, and j as right pointer to scan the array

Step# 2: Scan array from right-to-left, by decreasing j, until a key smaller than
or equal to pivot is encountered

Step# 3: Scan array from left-to-right, by increasing i, until key larger than
or equal to pivot is encountered

Step# 4: Swap the keys identified by i and j

Step# 5: Move pointers to next cells ( increase i by 1, and decrease j by 1)

Step# 6: Repeat Step# 2 through Step# 5 until i < j

ƒ On termination of the procedure the index j partitions the array

 Hoare Partitioning
Example

• Consider sample array shown in the figure

i j

indexes 1 2 3 4 5 6 7 8 9 10
keys 34 61 30 65 33 59 18 10 15 45
Sample Array

• The array will be partitioned using Hoare algorithm. Two pointers are used to scan the array,
alternately, from right-to-left and left to-right. The pointers are integer variables i and j, which
initially refer to the indexes of the first and last keys in the array
 Hoare Partitioning
Example
The left-most key 34 is chosen as pivot. The variables i, j are used to scan the array . The
partitioning steps are illustrated in the following figures. The keys in the left partition are
shaded blue, and those in the right partition are shaded pink. The swapped keys are shaded
green
(1) The left pointer i is positioned at cell 1 of i j
Pivot=34
the array. The right pointer j is positioned at
cell 10. Scanning is started from right-to-left 1 2 3 4 5 6 7 8 9 10
34 61 30 65 33 59 18 10 15 45

(2) Since key 45 in cell 10 is larger than the

pivot 34, the pointer j is moved to the next left i j
cell 9. Since the key 15 in cell 9 is smaller than
the pivot, the right scanning is stopped .
1 2 3 4 5 6 7 8 9 10
34 61 30 65 33 59 18 10 15 45

(3) The scanning from left-to-right is started. i j

Since the key in cell 1 is equal to the pivot,
the left scanning is stopped. 1 2 3 4 5 6 7 8 9 10
34 61 30 65 33 59 18 10 15 45
swap
 Hoare Partitioning
Example
(4) The keys 34 and 15 in cells 1 and 9 are i j
swapped. The right pointer j is moved to next
left cell 10, and the left pointer i is moved to
1 2 3 4 5 6 7 8 9 10
next right cell 2, as shown in the next
diagram. 15 61 30 65 33 59 18 10 34 45

(5) The right-to-left scanning is started. i j

Since key 10 in cell 8 is smaller than the
pivot 34, the right-to-left scanning is stopped.
and the left-to-right scanning is started. 1 2 3 4 5 6 7 8 9 10
Since the key 61in cell 2 is larger than the 15 61 30 65 33 59 18 10 34 45
pivot, the left-to-right scanning is stopped swap

(6) The keys 61 and 10 are swapped. The left i j

pointer is moved to next right cell and right
pointer is moved to next left cell, as shown in 1 2 3 4 5 6 7 8 9 10
the next diagram..
15 10 30 65 33 59 18 61 34 45
 Hoare Partitioning
Example
(7) The right-to-left scanning is started. Since
key 18 is less than pivot 34, the right-to-left i j
scanning is stopped at cell 7. The key 61
belongs to the right partition. 1 2 3 4 5 6 7 8 9 10
15 10 30 65 33 59 18 61 34 45

(8) The left-to-right scanning is started. Since

key 30 in cell 3 is smaller than the pivot, the i j
left pointer is moved to the next cell 4, as
shown in the next diagram.
1 2 3 4 5 6 7 8 9 10
15 10 30 65 33 59 18 61 34 45

(9) Since key 65 in cell 4 is larger than the i j

pivot, the left-to-right scanning is stopped.
1 2 3 4 5 6 7 8 9 10
15 10 30 65 33 59 18 61 34 45
swap
 Hoare Partitioning
Example
(10) The keys 65 and18 are swapped. The i j
pointers i, j are moved to the next adjacent
cells
1 2 3 4 5 6 7 8 9 10
15 10 30 18 33 59 65 61 34 45

(11) The pointers i, j point to the same cell.

i j
The partitioning procedure is terminated

1 2 3 4 5 6 7 8 9 10
15 10 30 18 33 59 65 61 34 45

(12) The array is now partitioned. The right 1 2 3 4 5 6 7 8 9 10

partition begins with cell identified by j+1 15 10 30 18 33 59 65 61 34 45
i.e, cell 6. Thus, all keys in the left partition
are smaller than the pivot. Likewise, all keys
in the right partition are greater than or Left partition Right partition
equal to the pivot, shown in red color Pivot=34
 Quick Sort
Example
ƒ In order to sort the array, the Hoare’s partitioning is applied repeatedly to the left and
right sub partitions, until each subarray reduces to a single element. As a result of
comparison and swapping , the elements are arranged in the ascending order.

ƒ Consider, for example, the sample unsorted array shown below. The quick sort steps are
shown the next diagram

34 61 30 65 33 59 18 10 15 45
Sample unsorted array
 Quick Sort
Example
The following diagram illustrates the quick sort procedure using Hoare’s partitioning method. The sorted
elements are numbered and shown in bloe color

Unsorted Array 34 61 30 65 33 59 18 10 15 45
pivot =34 34 61 30 65 33 59 18 10 15 45

15 10 30 18 33 59 65 61 34 45
1

10 15 30 18 33 45 34 61 65 59
2 6 7 8

15 30 18 33 34 45 59 65 61

3 9 10

18 30 33 61 65
4 5

30 33

Sorted Array 10 15 18 30 33 34 45 59 61 65
 Quick Sort
Partitioning
ƒ Quick sorting is implemented using two procedures, namely, PARTITION and QUICKSORT

ƒ The PARTITION consists of two counters to scan the array from left-to-right and then from
right-to-left. (Source: T.Cormen et al) The first element in the array is chosen as the pivot

PARTITION( A, p, r)
x ← A[p] ► left-most element is chosen as pivot. It is stored in x
i ← p-1 ► Index i scans the array from left-to-right
j ←r+1 ►Index j scans the array from right-to-left
while TRUE do ►A loop is set up to scan, which terminates when pointers cross
repeat j ← j-1 ►right pointer is moved until a key smaller than or equal to the
until A[j] ≤ x pivot is found
repeat i ←i+1 ►Left pointer is moved until a key larger than or equal to the
until A[i] ≥ x pivot is found

if i < j ► If pointer are not equal or do not cross-over

then exchange A[i]↔A[j] ► Exchange elements identified by i and j
else
return j ► Return partitioning index when pointer are equal or crossover

Visualization
Visualization Example
 Quick Sort
Algorithm
The QUICKSORT procedure is recursive. It is used to sort left and right partitions .

QUICKSORT(A, p ,r)
if p < r then ► Recursion is terminated when boundaries cross-over

q ← PARTITION(A, p, r ) ► The PARTITION method returns index of pivot

QUICKSORT(A, p, q) ► Sort left partition recursively

QUICKSORT(A, q+1, r) ► Sort right partition recursively

Visualization
Visualization Example
Analysis of Quick Sort
 Quick Sort Analysis
Best Case Scenario
ƒ In best case, the array is partitioned in to two nearly equal subarrays

A[1] A[2] - - A[k-1] A[k] A[k+1] - - A[n-1] A[n]

pivot
Left partition Right partition
ƒThe recurrence relation for best running time is

T(n) = 2 T( n / 2) + n-1

Time to sort Time to sort

two subarrays Cost of partitioning
array of size n
each of size n/2 array of size n-1
 Quick Sort Analysis
Best Case Solution
ƒ The recurrence
T(n) = 2 T( n / 2) + n - 1
can be solved by iteration-substitution method

ƒ After kth iteration:

T(n) = 2k T( n / 2k ) + k.n - ( 20 + 21+ 22+…….. + 2k-1)

ƒ Summing the series

T(n) = 2k T( n / 2k ) + k.n -( 2k - 1)

ƒ Setting n/2k =1, gives k=lg n

T(n) = 2lg n. T(1 ) + n. lg n -( 2lg n - 1)

= n + n lg n – ( n - 1) (closed form )

T(n) = n lg n + 1= θ( n lg n) (asymptotic form )

ƒ The best case running time of quick sort is θ( n lg n)

 Quick Sort Partitions
Presorted Input
If the input array is presorted, the quick sort partitions degenerate into a single right partition.
This scenario is demonstrated by the following example. The input keys are {10, 20, 30, 40,
50, 60, 70, 80, 90}
(1) The left pointer i is positioned at cell 1 of j
i Pivot=10
the array. The right pointer j is positioned at
cell 9. Key 10 in the left-most cell is chosen
as pivot. Scanning is started from right-to- 1 2 3 4 5 6 7 8 9
left 10 20 30 40 50 60 70 80 90

(2) Since key 90 is larger than the pivot 10, i j

the pointer j is moved to the next left cell
1 2 3 4 5 6 7 8 9
10 20 30 40 50 60 70 80 90

(3) Since key 80 is larger than the pivot 10, j

i
the pointer j is moved to the next left cell

1 2 3 4 5 6 7 8 9
10 20 30 40 50 60 70 80 90
 Quick Sort Partitions
Presorted Input
(4) Since key 60 is larger than pivot 10, the j
i Pivot=10
right pointer is moved to the next left cell
1 2 3 4 5 6 7 8 9
10 20 30 40 50 60 70 80 90

(5) Since key 50 is larger than the pivot 10, i j

the pointer j is moved to the next left cell
1 2 3 4 5 6 7 8 9
10 20 30 40 50 60 70 80 90

(6) Since key 40 is larger than the pivot 10, i j

the pointer j is moved to the next left cell

1 2 3 4 5 6 7 8 9
10 20 30 40 50 60 70 80 90
 Quick Sort Partitions
Presorted Input
i j
(7) Since key 40 is larger than pivot 10, the
right pointer is moved to the next left cell
1 2 3 4 5 6 7 8 9
10 20 30 40 50 60 70 80 90

(8) Since key 30 is larger than the pivot 10, i j

the pointer j is moved to the next left cell
1 2 3 4 5 6 7 8 9
10 20 30 40 50 60 70 80 90

i j
(9) Since key 20 is larger than the pivot 10,
the pointer j is moved to the next left cell
1 2 3 4 5 6 7 8 9
10 20 30 40 50 60 70 80 90

i j
(10) Since key 20 is larger than the pivot 10,
the pointer j is moved to the next left cell.
The left and right pointers are now 1 2 3 4 5 6 7 8 9
positioned at the same cell. The array is 10 20 30 40 50 60 70 80 90
partition into a single partition to the right
 Quick Sort Partitions
Reverse Sorted Input
If the input array is reverse sorted, the quick sort partitions degenerate into a single left
partition. This scenario is demonstrated by the following example. The input keys are {90, 80,
70, 60, 50, 40, 30, 20, 10}
(1) The left pointer i is positioned at cell 1 of j
i Pivot=90
the array. The right pointer j is positioned at
cell 9. Key 90 in the left-most cell is chosen
as pivot. Scanning is started from right-to- 1 2 3 4 5 6 7 8 9
left 90 80 70 60 50 40 30 20 10

i j
(2) Since key 10 is smaller than the pivot
90, the right-to-left scan is stopped
1 2 3 4 5 6 7 8 9
90 80 70 60 50 40 30 20 10

(3 The left-to-right scan is started. Since key

10 is less than pivot, the scan is stopped. The i j
keys 10 and 90 are swapped, and pointers
are moved to next adjacent cells 1 2 3 4 5 6 7 8 9
10 80 70 60 50 40 30 20 90
 Quick Sort Partitions
Reverse Sorted Input
(4)The right-to-left scan is started. Since key
i j
20 is less than the pivot 90 the right scan is
stopped. The left scan is started. Since key 80
is smaller than the pivot, the left pointer is 1 2 3 4 5 6 7 8 9
moved to the next adjacent cell. 10 80 70 60 50 40 30 20 90

(5) Since key 70 is smaller than the pivot i j

90, the left pointer is moved to the next right
cell 1 2 3 4 5 6 7 8 9
10 80 70 60 50 40 30 20 90

(6) Since key 60 is smaller than the pivot

90, the left pointer is moved to the next right i j
cell
1 2 3 4 5 6 7 8 9
10 80 70 60 50 40 30 20 90
 Quick Sort Partitions
Reverse Sorted Input
(7) Since key 50 is smaller than the pivot i j
90, the left pointer is moved to the next right
cell 1 2 3 4 5 6 7 8 9
10 80 70 60 50 40 30 20 90

(8) Since key 40 is smaller than the pivot i j

90, the left pointer is moved to the next right
cell 1 2 3 4 5 6 7 8 9
10 80 70 60 50 40 30 20 90

i j
(9) Since key 30 is smaller than the pivot
90, the left pointer is moved to the next right 1 2 3 4 5 6 7 8 9
cell
10 80 70 60 50 40 30 20 90

1 2 3 4 5 6 7 8 9
(10) The left and right pointers are now
10 80 70 60 50 40 30 20 90
positioned at the same cell. The partitioning
procedure terminates. The array is
partitioned into a single left partition Left partition
 Quick Sort Analysis
Worst Case Scenario
Consider the case when left partition is empty

A[1] A[2] A[3] - - A[n-2] A[n-1] A[n]

pivot right partition of size n - 1

ƒ The recurrence relation for worst running time is
T(n) = T(0) + T( n -1) + n-1

Time to sort Time to sort Time to sort Cost of partitioning

array of size n left partition right partition subarray of size n-1
T(0)=0 of size n-1
ƒ Since left partition is empty, recurrence simplifies to:

T(n) = T( n -1) + n – 1, n>1

ƒ The same recurrence applies when right partition is empty.
 Quick Sort Analysis
Worst Case Solution
ƒ The recurrence

T(n) = T( n -1) + n – 1, n >1

T(1) =0

can be solved by iteration method

ƒ Iteration yields the solution:

T(n) = n-1 + (n-2) + (n-3) + +……..+ 2 + 1

T(n)= n (n -1) / 2 (closed form)

=θ(n2) (asypmtotic form )

ƒ The worst case running time of quick sort is θ(n2)

 Quick Sort Analysis
Average Case Scenario
In order to do the analysis we assume that the pivot is likely to be in any of the cells, Thus, the probability of pivot being in
any of the n cells is 1/n. The table summarizes the running time for different possibilities and associated costs.
pivot
Pivot cell Probability Sort time
T(0)+T(n-1) A[1] A[2] A[3] - - - - A[n-2] A[n-1] A[n]
ist cell 1/n
(n-1) keys

2nd cell 1/n T(1)+T(n-2) A[1] A[2] A[3] - - - - A[n-2] A[n-1] A[n]
1 key (n-2 )keys

3rd cell 1/n T(2)+T(n-3) A[1] A[2] A[3] - - - - A[n-2] A[n-1] A[n]
2 keys (n-3) keys

kth cell 1/n T(k-1)+T(n-k) A[1] A[2] A[3] A[4] A[k-1] A[k] - A[n-2] A[n-1] A[n]
k-1 keys (n-k) keys

n-1th cell 1/n T(n-2)+T(1) A[1] A[2] A[3] A[4] - - - A[n-2] A[n-1] A[n]
(n-2 )keys 1 key

nth cell 1/n T(n-1)+T(0) A[1] A[2] A[3] A[4] - - - A[n-2] A[n-1] A[n]
(n-1) keys
 Quick Sort Analysis
Average Case Recurrence
A[1] A[2] A[3] - A[k-1] A[k] A[k+1] - A[n-1] A[n]

pivot
Left partition of size k-1 right partition of size n- k

ƒ By summing over running times for all costs and associated probabilities
the expected running time, T(n), can be expressed as the recurrence:

∑
1
T(n) = n–1 + — [ T( k - 1) + T(n – k) ]
n
k =1

Expected time Cost of partitioning Time to sort

to sort array subarrays of Probability Time to sort subarray
of pivot subarray
of size n size n-1 of size n - k
being in of size k-1
cell k
 Quick Sort Analysis
Average Running Time
n
1
ƒ Consider the recurrence: T(n) = n – 1 + ∑k =1
—
n
[ T( k - 1) + T(n – k) ]
ƒ Expanding the summation
n

∑T(n – k) = T(n-1)+T(n-2)+…..+T(2) + T(1) + T(0)

k =1
= T(0)+T(1)+T(2)+…….. + T(n-2) + T(n-1) ( Rearranging in reverse order )
n
= ∑T(k-1)
k =1
……………(1)

ƒ The recurrence can be rewritten as:

n
T(n) = n – 1 +
2
—
n ∑T( k - 1)
k =1
……………(2)

ƒ Multiplying both sides of (2) with n

n
n.T(n) = n(n – 1) + 2 ∑ T( k - 1)
k =1
……………(3)

ƒ Substituting n-1 for n in the recurrence (3)

n −1

(n-1).T(n-1) = (n-1)(n – 2) + 2 ∑ T( k - 1)
k =1
……(4)
 Quick Sort Analysis Cont’d-1

Average Running Time

n
n.T(n) = n(n – 1) + 2 ∑ T( k - 1)
k =1
……………(5)

n −1
(n-1).T(n-1) = (n-1)(n – 2) + 2 ∑ T( k - 1)
k =1
……………(6)

Subtracting (6) from (5), and rearranging

nT(n) - (n-1)T(n-1) = 2(n-1) + 2T(n-1) …….…(7)

Simplifying and rearranging

T(n) T(n-1) 2(n-1)
———— = ——— + ———— ……………(8)
n+1 n n(n+1)

Defining S(n) as
T(n)
S(n) = ———
n+1

gives the new recurrence

2(n - 1) ……………(9)
S(n) = S(n-1)+ ——— , S(0)=0
n(n+1)
 Quick Sort Analysis Cont’d-2

Average Running Time

The following recurrence is solved by iteration method.
2(n - 1)
S(n) = S(n-1)+ ——— , S(0)=0 …………..(10)
n(n+1)
The solution is as follows
n n

∑ ∑
2( i - 1) 2(i+1)-4
S(n) = ——— =
i =1 i (i +1) i =1 i(i +1)
n n

=
i
∑ i =1
2
i(i+1)
- ∑ 4
n
…………..(11)
i =1
Performing summation of harmonic series ,
n
∑ —1i = ln( n )+ 0.386
i =1
…………..(12)
n
∑ i(i+1) = ∑ 1 n
1
Also,
i =1 i =1 i
- ∑
i =1
1
(i+1)
= 1- 1/(n+1)=n/(n+1) …………..(13)

Substituting results (12) and (13) into summation (11), we get

…………..(14)
S(n) = 2( ln (n) + 0.386 ) – 4 n / (n+1) Log2 e = 1.442695040888963387

T(n) = 1.386 n lg n – 2.846 n (converting ln n to lg n ) Replace s(n) with T(n)/n+1

The Average running time of quick sort is θ(n lg n)

 Quick Sort Analysis
Running Times

Tworst(n)= n (n -1) / 2

T(n)

Texpected(n) = 1.386 n lg n – 2.846 n

Tbest(n) = n lg n + 1