Data Structures Using C++ 2E

Chapter 10
Sorting Algorithms
 Objectives

• Learn the various sorting algorithms

• Explore how to implement
• Selection sort
• Insertion sort
• Quicksort
• Mergesort
• Heapsort
• Discover how the sorting algorithms discussed in
this chapter perform

Data Structures Using C++ 2E 2

 Sorting Algorithms

• Several types in the literature

– Discussion includes most common algorithms
• Analysis
– Provides a comparison of algorithm performance
• Functions implementing sorting algorithms
– Included as public members of related class

Data Structures Using C++ 2E 3

 Selection Sort: Array-Based Lists

• List sorted by selecting elements in the list

– Select elements one at a time
– Move elements to their proper positions
• Selection sort operation
– Find location of the smallest element in unsorted list
portion
• Move it to top of unsorted portion of the list
– First time: locate smallest item in the entire list
– Second time: locate smallest item in the list starting
from the second element in the list, and so on

Data Structures Using C++ 2E 4

 FIGURE 10-1 List of 8 elements

FIGURE 10-2 Elements of list during the first iteration

FIGURE 10-3 Elements of list during the second iteration

Data Structures Using C++ 2E 5
 Selection Sort: Array-Based Lists
(cont’d.)
• Selection sort steps
– In the unsorted portion of the list
• Find location of smallest element
• Move smallest element to beginning of the unsorted list
• Keep track of unsorted list portion with a for loop

Data Structures Using C++ 2E 6

 Selection Sort: Array-Based Lists
(cont’d.)
• Given: starting index, first, ending index, last
– C++ function returns index of the smallest element in
list[first]...list[last]

Data Structures Using C++ 2E 7

 Selection Sort: Array-Based Lists
(cont’d.)
• Function swap
• Definition of function selectionSort

Data Structures Using C++ 2E 8

 Selection Sort: Array-Based Lists
(cont’d.)
• Add functions to implement selection sort in the
definition of class arrayListType

Data Structures Using C++ 2E 9

 Analysis: Selection Sort

• Search algorithms
– Concerned with number of key (item) comparisons
• Sorting algorithms
– Concerned with number of key comparisons and
number of data movements
• Analysis of selection sort – O(n2)
– Function swap
• Number of item assignments: 3(n-1)
– Function minLocation
• Number of key comparisons of O(n2)

Data Structures Using C++ 2E 10

 Insertion Sort: Array-Based Lists

• Attempts to improve high selection sort key

comparisons
• Sorts list by moving each element to its proper place
• Given list of length eight

FIGURE 10-4 list

Data Structures Using C++ 2E 11

 Insertion Sort: Array-Based Lists
(cont’d.)
• Elements list[0], list[1], list[2], list[3]
in order
• Consider element list[4]
– First element of unsorted list

FIGURE 10-5 list elements while moving list[4] to its proper place

Data Structures Using C++ 2E 12

 Insertion Sort: Array-Based Lists
(cont’d.)
• Array containing list divided into two sublists
– Upper and lower
• Index firstOutOfOrder
– Points to first element in the lower sublist

Data Structures Using C++ 2E 13

 Insertion Sort: Array-Based Lists
(cont’d.)
• length = 8
• Initialize firstOutOfOrder to one

FIGURE 10-6 firstOutOfOrder = 1

Data Structures Using C++ 2E 14

 Insertion Sort: Array-Based Lists
(cont’d.)
• list[firstOutOfOrder] = 7
• list[firstOutOfOrder - 1] = 13
– 7 < 13
• Expression in if statement evaluates to true
– Execute body of if statement
• temp = list[firstOutOfOrder] = 7
• location = firstOutOfOrder = 1
– Execute the do...while loop
• list[1] = list[0] = 13
• location = 0

Data Structures Using C++ 2E 15

 Insertion Sort: Array-Based Lists
(cont’d.)
• do...while loop terminates
– Because location = 0
• Copy temp into list[location] (list[0])

FIGURE 10-7 list after the first iteration of insertion sort

Data Structures Using C++ 2E 16

 Insertion Sort: Array-Based Lists
(cont’d.)
• Suppose list given in Figure 10-8(a)
– Walk through code

FIGURE 10-8 list elements while moving list[4] to its proper place

Data Structures Using C++ 2E 17

 Insertion Sort: Array-Based Lists
(cont’d.)
• Suppose list given in Figure 10-9
– Walk through code

FIGURE 10-9 First out-of-order element is at position 5

Data Structures Using C++ 2E 18

 Insertion Sort: Array-Based Lists
(cont’d.)
• C++ function implementing previous algorithm

Data Structures Using C++ 2E 19

 Insertion Sort: Linked List-Based Lists
• If list stored in an array
– Traverse list in either direction using index variable
• If list stored in a linked list
– Traverse list in only one direction
• Starting at first node: links only in one direction

FIGURE 10-10 Linked list

Data Structures Using C++ 2E 20

 Insertion Sort: Linked List-Based Lists
(cont’d.)
• firstOutOfOrder
– Pointer to node to be moved to its proper location
• lastInOrder
– Pointer to last node of the sorted portion of the list

FIGURE 10-11 Linked list and pointers lastInOrder

and firstOutOfOrder

Data Structures Using C++ 2E 21

 Insertion Sort: Linked List-Based Lists
(cont’d.)
• Compare firstOutOfOrder info with first node
info
– If firstOutOfOrder info smaller than first node
info
• firstOutOfOrder moved before first node
– Otherwise, search list starting at second node to find
location where to move firstOutOfOrder
• Search list using two pointers
– current
– trailCurrent: points to node just before current
• Handle any special cases
Data Structures Using C++ 2E 22
 Insertion Sort: Linked List-Based Lists
(cont’d.)

proper

Data Structures Using C++ 2E 23

 Insertion Sort: Linked List-Based Lists
(cont’d.)
• Case 1: firstOutOfOrder->info less than first->info
• Node firstOutOfOrder moved before first
• Adjust necessary links

Data Structures Using C++ 2E 24

 Insertion Sort: Linked List-Based Lists
(cont’d.)
• Case 2: firstOutOfOrder at the proper place
• Adjust lastInOrder = lastInOrder->link

Data Structures Using C++ 2E 25

 Insertion Sort: Linked List-Based Lists
(cont’d.)
• Case 3: firstOutOfOrder->info greater than first->info
• Find place with current and trailcurrent pointers
• Adjust necessary links

Data Structures Using C++ 2E 26

Data Structures Using C++ 2E 27
Data Structures Using C++ 2E 28
 Analysis: Insertion Sort

TABLE 10-1 Average-case behavior of the selection sort and

insertion sort for a list of length n

• Proofs are given in the Appendix of your book.

Data Structures Using C++ 2E 29

 Lower Bound on Comparison-Based
Sort Algorithms (cont’d.)
• Theorem
– Let L be a list of n distinct elements. Any sorting
algorithm that sorts L by comparison of the keys only,
in its worst case, makes at least O(nlog2n) key
comparisons

• So far: O(n2)
• The rest: O(nlog2n)

Data Structures Using C++ 2E 30

 Quicksort: Array-Based Lists
• Uses the divide-and-conquer technique to sort a list
– List partitioned into two sublists
• Two sublists sorted and combined into one list
• Combined list then sorted using quicksort (recursion)
• Trivial to combine sorted lowerSublist and
upperSublist
• All sorting work done in partitioning the list

Data Structures Using C++ 2E 31

 Quicksort: Array-Based Lists (cont’d.)
• Pivot divides list into two sublists
– lowerSublist: elements smaller than pivot
– upperSublist: elements greater than pivot
• Choosing the pivot
– lowerSublist and upperSublist nearly equal

FIGURE 10-21 List before the partition

FIGURE 10-22 List after the partition

Data Structures Using C++ 2E 32
 Quicksort: Array-Based Lists (cont’d.)
• Partition algorithm
– Determine pivot; swap pivot with first list element
• Suppose index smallIndex points to last element
smaller than pivot. smallIndex initialized to first list
element
– For the remaining list elements (starting at second
element): If current element smaller than pivot
• Increment smallIndex
• Swap current element with array element pointed to by
smallIndex
– Swap first element (pivot) with array element pointed
to by smallIndex
Data Structures Using C++ 2E 33
 Quicksort: Array-Based Lists (cont’d.)

Data Structures Using C++ 2E 34

 Quicksort: Array-Based Lists (cont’d.)

Data Structures Using C++ 2E 35

 Quicksort: Array-Based Lists (cont’d.)

Data Structures Using C++ 2E 36

 Quicksort: Array-Based Lists (cont’d.)

Data Structures Using C++ 2E 37

 Quicksort: Array-Based Lists (cont’d.)

Data Structures Using C++ 2E 38

 Quicksort: Array-Based Lists (cont’d.)
• Function partition
– Passes starting and ending list indices
– Swaps certain elements of the list

Data Structures Using C++ 2E 39

 Quicksort: Array-Based Lists (cont’d.)
• Given starting and ending list indices
– Function recQuickSort implements the recursive
version of quicksort
• Function quickSort calls recQuickSort

Data Structures Using C++ 2E 40

 Analysis: Quicksort

TABLE 10-2 Analysis of quicksort for a list of length n

• Proofs are given in the Appendix of your book.

Data Structures Using C++ 2E 41

 Mergesort: Linked List-Based Lists

• Quicksort
– Average-case behavior: O(nlog2n)
– Worst-case behavior: O(n2)
• Mergesort behavior: always O(nlog2n)
– Uses divide-and-conquer technique to sort a list
– Partitions list into two sublists
• Sorts sublists
• Combines sorted sublists into one sorted list
• Difference between mergesort and quicksort
– How list is partitioned
Data Structures Using C++ 2E 42
 Mergesort: Linked List-Based Lists
(cont’d.)

FIGURE 10-32 Mergesort algorithm

Data Structures Using C++ 2E 43
 Mergesort: Linked List-Based Lists
(cont’d.)
• Most sorting work done in merging sorted sublists
• General algorithm for mergesort

Data Structures Using C++ 2E 44

 Divide

• To divide list into two sublists

– Need to find middle node
– Linked list implementation: Length not known
– Use two pointers: middle and current
• middle initialized to the first node
• current initialized to the third node

Data Structures Using C++ 2E 45

 Divide (cont’d.)
– Advance two pointers: middle and current
• Advance middle by one node, advance current by
one node
• If current not NULL, again advance current by one
node
• current becomes NULL; middle points to last node
of the first sublist

Data Structures Using C++ 2E 46

 Divide (cont’d.)

– Divide list into two sublists

• Using the link of middle: assign pointer to node
following middle
• Set link of middle to NULL

• See function divideList on page 561

Data Structures Using C++ 2E 47
 Merge

• Once sublists sorted

– Next step: merge the sorted sublists
• Merge process
– Compare elements of the sublists
– Adjust references of nodes with smaller info

Data Structures Using C++ 2E 48

 Merge (cont’d.)

• Set and advance pointers

– Set newHead to the first node of the new list
– lastMerged to keep track of the last node of the
merged list
– Pointer of the first node of the sublist with the smaller
node advances to the next node of that sublist

Data Structures Using C++ 2E 49

 Merge (cont’d.)

• Continue the process

• Eventually, either first1 or first2 becomes NULL

– If first1 is NULL: The first sublist exhausted first,
attach the remaining nodes of second sublist to new list
– If first2 is NULL: The second sublist exhausted first,
attach the remaining nodes of first sublist to new list
Data Structures Using C++ 2E 50
 Analysis: Mergesort

• Maximum number of comparisons made by

mergesort: O(n log2n)
• If W(n) denotes number of key comparisons
– Worst case to sort L: W(n) = O(n log2n)
• Let A(n) denote number of key comparisons in the
average case
– Average number of comparisons for mergesort
– If n is a power of 2
• A(n) = n log2n - 1.25n = O(n log2n)
• Mathematical proofs included in your book
Data Structures Using C++ 2E 51
 Heapsort: Array-Based Lists
• Overcomes the worst case of quicksort
• Heap: list in which each element contains a key
– Key in the element at position k in the list
• At least as large as the key in the element at position
2k + 1 (if it exists) and 2k + 2 (if it exists)

FIGURE 10-41 A heap

• Ex: list[3] = 50
– list[7] = 20 & list[8] = 10
– list[3] >= list[7] & list[3] >= list[8]
Data Structures Using C++ 2E 52
 Heapsort: Array-Based Lists (cont’d.)
• Data given in Figure 10-41
– Can be viewed in a complete binary tree

FIGURE 10-41 A heap

FIGURE 10-42 Complete binary tree

corresponding to the list in Figure 10-41
Data Structures Using C++ 2E 53
 Heapsort: Array-Based Lists (cont’d.)
• Heapsort
– First step: convert list into a heap
• Called buildHeap
• length denotes the length of the list
• index = length / 2 – 1
• Elements list[index+1] … list[length-1] are
leaves
• Convert subtrees to heap
– After converting the array into a heap
• Sorting phase begins

Data Structures Using C++ 2E 54

 Build Heap
• Algorithm with an example:

Data Structures Using C++ 2E 55

 Build Heap (cont’d.)

• length = 11, index = 11 / 2 - 1 = 4

• list[4] = 56, Children are list[9] and list[10] (both exist)
• Convert the tree with root list[4]:
1. Find the largest child of list[4]: list[10] > list[9]
2. If larger child (list[10]) > parent (list[4]): swap list[10] and list[4]
3. Check if list[10] has a subtree. If it has a subtree, repeat steps 1
and 2 to build heap from there.

Data Structures Using C++ 2E 56

 Build Heap (cont’d.)

• Figure 10-45(a) : build heap for list[4] = 56

• Figure 10-45(b) : build heap for list[3] = 70
• Figure 10-45(c) : build heap for list[2] = 72

Data Structures Using C++ 2E 57

 Build Heap (cont’d.)
• Next, consider list[1] = 60
– list[1] = 60 < list[3] = 92 (largest child) : swap list[1] and list[3]
• Step 3: Subtree with root node list[3] is no longer a heap.
Repeat steps 1 and 2 for that subtree.
– list[3] = 60 < list[7] = 70 (largest child) : swap list[3] and list[7]

Data Structures Using C++ 2E 58

 Build Heap (cont’d.)
• Finally, consider list[0] =15

• Consider subtree with root node list[1]

• Consider subtree with root node list[3]
• Resulting binary tree in Figure 10-47(c) is in a heap

Data Structures Using C++ 2E 59

 Build Heap (cont’d.)
• low: index of the root node of the subtree
• high: index of the last item in the list

Data Structures Using C++ 2E 60

 Build Heap (cont’d.)

• Function heapify
– Restores the heap in a subtree
– Implements the buildHeap function
• Converts list into a heap

Data Structures Using C++ 2E 61

Data Structures Using C++ 2E 62
 Build Heap (cont’d.)

• Function heapify
– Implements the buildHeap function
• Converts list into a heap

Data Structures Using C++ 2E 63

 Build Heap (cont’d.)

• The heapsort algorithm

– Swap root node with the last item (Figure 10-48(b))
– Consider the remaining elements: list[0]…list[9]
• No longer a heap: heapify(list, 0, 9)
– Repeat steps

Data Structures Using C++ 2E 64

 Build Heap (cont’d.)

Data Structures Using C++ 2E 65

 Analysis: Heapsort

• Given L a list of n elements where n > 0

• Worst case
– Number of key comparisons to sort L
• 2nlog2n + O(n)
– Number of item assignments to sort L
• nlog2n + O(n)
• Average number of comparisons to sort L
– O(nlog2n)
• Heapsort takes twice as long as quicksort generally
– Avoids the slight possibility of poor performance (worst
case of quicksort)
Data Structures Using C++ 2E 66
 Summary

• Search algorithms may require sorted data

• Several sorting algorithms available
– Selection sort, insertion sort, quicksort, mergesort,
and heapsort
• Can be applied to either array-based lists or linked lists
• Compare algorithm performance through analysis
– Number of key comparisons
– Number of data movements
• Functions implementing sorting algorithms
– Included as public members of the related class
Data Structures Using C++ 2E 67
 Programming Exercises

• Chapter 10 – Exercise 1
• Chapter 10 – Exercise 2

• Chapter 10 – Exercise 9
• Chapter 10 – Exercise 10
• Chapter 10 – Exercise 11

Data Structures Using C++ 2E 68

 Programming Exercises (cont’d.)

Data Structures Using C++ 2E 69