Chapter Three
Simple Sorting and
Searching Algorithms
1
Why do we study sorting and
searching algorithms?
These algorithms are the most common and
useful tasks operated by computer system.
Computers spend a lot of time searching and
sorting.
1. Simple Searching algorithms
Searching:- is a process of finding an element in a list
of items or determining that the item is not in
2
the list.
Tokeep things simple, we shall deal with a list of
numbers.
A searchmethod looks for a key, arrives by
parameter.
By convention, the method will return the index of
the element corresponding to the key or, if
unsuccessful, the value -1.
3
There are two simple searching algorithms:
a) Sequential Search, and
b) Binary Search
a). Sequential Searching (Linear)
The most natural way of searching an item.
Easy to understand and implement.
4
Algorithm:
In a linear search, we start with top
(beginning) of the list, and compare the
element at top with the key.
If we have a match, the search terminates and
the index number is returned.
If not, we go on the next element in the list.
If we reach the end of the list without finding a
match, we return -1.
5
Implementation:
Assume the size of the list is n.
int LinearSearch(int list[ ], int key)
{
index=-1;
for(int i=0; i<n; i++)
{
if(list[i]==key)
{
index=i;
break;
}
}
return index; 6
}
Complexity Analysis:
Big-Oh of sequential searching How
many comparisons are made in the
worst case ? n
O(n).
b). Binary Searching
Assume sorted data.
Use Divide and conquer strategy7
(approach).
Algorithm:
I. In a binary search, we look for the key in the middle
of the list. If we get a match, the search is over.
II. If the key is greater than the element in the middle
of the list, we make the top (upper) half the list to
search.
III. If the key is smaller, we make the bottom (lower)
half the list to search.
IV. Repeat the above steps (I,II and III) until one
element remains.
V. If this element matches return the index of the
element, else return -1 index. (-1 shows that the key
is not in the list). 8
Implementation: if(found==0)
int BinarySearch(int list[ ], int key) index=-1;
{ else
int found=0,index=0; index=middle;
int top=n-1,bottom=0,middle; return index;
}
do{
middle=(top + bottom)/2;
Complexity Analysis:
if(key==list[middle])
found=1;
Example: Find Big-Oh
else{
of Binary search
if(key<list[middle])
algorithm in the worst
top=middle-1; case analysis.
else
bottom=middle+1;
}
} 9
}while(found==0 && top>=bottom);
2. Sorting
Algorithms
10
2. Simple Sorting Algorithms
Sorting: is a process of reordering a list of items in
either increasing or decreasing order.
Ordering a list of items is fundamental problem of
computer science.
Sorting is the most important operation performed by
computers.
Sorting is the first step in more complex algorithms.
Two basic properties of sorting algorithms:
In-place:
It is possible to sort very large lists without the need to
allocate additional working storage. 11
Simple Sorting Algorithms
Stable:
If two elements that are equal remain in the same
relative position after sorting is completed.
Two classes of sorting algorithms:
O(n2):
Includes the bubble, insertion, and selection sorting
algorithms.
O(nlog n):
Includes the heap, merge, and quick sorting
algorithms. 12
Simple Sorting Algorithms
Simple sorting algorithms include:
i. Simple sorting
ii. Bubble Sorting
iii. Selection Sorting
iv. Insertion Sorting
13
I. Simple sorting
Algorithm:
In simple sort algorithm the first element is compared
with the second, third and all subsequent elements.
If any one of the other elements is less than the current
first element then the first element is swapped with
that element.
Eventually, after the last element of the list is
considered and swapped, then the first element has the
smallest element in the list.
The above steps are repeated with the second, third
14
and all subsequent elements.
Implementation:
Void SimpleSort(int list[])
{
for(int i=0; i<=n-2;i++)
for(int j=i+1; j<=n-1; j++)
if(list[i] > list[j])
{
int temp;
temp=list[i];
list[i]=list[j];
list[j]=temp;
}
15
}
Example:
`
0 1 2 n-1
--------
i
j
4 2 3 1
i=0 j=1 2 4 3 1
j=2 2 4 3 1
j=3 1 4 3 2
i=1 j=2 1 3 4 2
j=3 1 2 4 3
i=2 j=3 1 2 3 4
16
`
Analysis: O(?)
1st pass----- (n-1) comparisons
2nd pass---- (n-2) comparisons
|
|
|
(n-1)th pass--- 1 comparison
T(n)=1+2+3+4+-------+(n-2)+(n-1)
= (n*(n-1))/2
=n2/2-n/2
=O(n2)
17
Complexity Analysis:
Analysisinvolves number of
comparisons and swaps.
How many comparisons?
1+2+3+…+(n-1)= O(n2)
How many swaps?
1+2+3+…+(n-1)= O(n2)
18
Example: Suppose we have 32 unsorted data.
a). How many comparisons are made by sequential
search in the worst-case?
Number of comparisons =32.
b). How many comparisons are made by binary search
in the worst-case? (Assuming simple sorting).
Number of comparisons = Number of comparisons
for sorting + Number of comparisons for binary search
= (n*(n-1))/2 + logn
= 32/2(32-1)+ log 32
=16*31 + 5
c). How many comparisons are made by binary search
in the worst-case if data is found to be already sorted?
Number of comparisons = log2 32 = 5. 19
II. Bubble sort
Algorithm:
III. Compare each element (except the last one) with its
neighbor to the right.
If they are out of order, swap them
This puts the largest element at the very end
The last element is now in the correct and
final place
II. Compare each element (except the last two) with its
neighbor to the right.
If they are out of order, swap them
This puts the second largest element before
last
The last two elements are now in their
20
correct and final places
III. Compare each element (except the last three)
with its neighbor to the right.
IV. Continue as above until you have no unsorted
elements on the left.
Is the oldest, simplest, and slowest sort in use.
It works by comparing each item in a list with
an item next to it, and swap them if required.
This causes the larger values to “bubble” to the
end of the list while smaller values to “sink”
towards the beginning of the list.
In general case, bubble sort has O(n2) level of
complexity.
21
Advantage:
Simplicity and ease of implementation.
Disadvantage:
Horribly inefficient.
22
`
Example of Bubble sort
4 2 3 1
i=3 j=1 2 4 3 1
j=2 2 3 4 1
j=3 2 3 1 4
i=2 j=1 2 3 1 4
j=2 2 1 3 4
i=1 j=1 1 2 3 4
23
Example of bubble sort
72854 27548 25478 24578
27854 27548 25478 24578
27854 25748 24578 (done)
27584 25478
27548
More Example
Implementation:
Void BubbleSort(int list[ ])
{ int temp;
for (int i=n-2; i>=0; i--) {
for(int j=0;j<=i; j++)
if (list[j] > list[j+1]) {
temp=list[j];
list[j])=list[j+1];
list[j]=temp;
}
25
}
Complexity Analysis:
Analysis involves number of comparisons
and swaps.
How many comparisons?
1+2+3+…+(n-1)= O(n2)
How many swaps?
1+2+3+…+(n-1)= O(n2)
26
III. Selection Sort
Algorithm
The selection sort algorithm is in many ways similar
to simple sort algorithms.
The idea of algorithm is quite simple. Array is
imaginary divided into two parts - sorted one and
unsorted one.
At the beginning, sorted part is empty, while unsorted
one contains whole array.
At every step, algorithm finds minimal element in the
unsorted part and adds it to the end of the sorted one.
28
When unsorted part becomes empty, algorithm stops.
Works by selecting the smallest unsorted item remaining in the
list, and then swapping it with the item in the next position to
be filled.
Similar to the more efficient insertion sort.
It yields a 60% performance improvement over the bubble
sort.
Advantage:
Simple and easy to implement.
Disadvantage:
Inefficient for larger lists.
29
Example:
i=0 J=1 7 9 11 3
J=2 7 9 11 3
J=3 7 9 11 3
i=1 J=2 3 9 11 7
J=3 3 9 11 7
i=2 J=3 3 7 11 9
Finally 3 7 9 1130
31
Implementation:
void selectionSort(int list[ ] ) {
int minIndex, temp;
for (int i = 0; i <= n - 2; i++) {
minIndex = i;
for (j = i + 1; j <= n-1; j++)
if (list[j] < list[minIndex])
minIndex = j;
if (minIndex != i) {
temp = list[i];
list[i] = list[minIndex];
list[minIndex] = temp;
}
} 32
}
Complexity Analysis
Selection sort stops, when unsorted part becomes
empty.
As we know, on every step number of unsorted
elements decreased by one.
Therefore, selection sort makes n-1 steps (n is number
of elements in array) of outer loop, before stop.
Every step of outer loop requires finding minimum in
unsorted part. Summing up, (n - 1) + (n - 2) + ... + 1,
results in O(n2) number of comparisons. 33
Number of swaps may vary from zero
(in case of sorted array) to n-1 (in case
array was sorted in reversed order),
which results in O(n) number of swaps.
Overall algorithm complexity is O(n2).
Fact,
that selection sort requires n-1
number of
swaps at most, makes it very efficient
in situations, when write operation is
significantly more expensive, than read
operation. 34
IV. Insertion Sort
Algorithm:
Insertion sort algorithm somewhat resembles Selection
Sort and Bubble sort.
Array is imaginary divided into two parts - sorted one
and unsorted one.
At the beginning, sorted part contains first element of
the array and unsorted one contains the rest.
At every step, algorithm takes first element in the
unsorted part and inserts it to the right place of the
sorted one.
When unsorted part becomes empty, algorithm stops.
35
Using binary search
Itis reasonable to use binary search
algorithm to find a proper place for
insertion.
This variant of the insertion sort is
called binary insertion sort.
After position for insertion is found,
algorithm shifts the part of the array
and inserts the element.
36
Insertion sort works by inserting item into
its proper place in the list.
Insertion sort is simply like playing cards: To
sort the cards in your hand, you extract a
card, shift the remaining cards and then
insert the extracted card in the correct
place.
This process is repeated until all the cards
are in the correct sequence.
Is over twice as fast as the bubble sort and
is just as easy to implement as the selection
sort.
Advantage:
Relatively simple and easy to implement.
Disadvantage:
Inefficient for large lists. 37
4 4 3
3 4 4
2 2 2 2
1 1 1 1
3 3 2
3
4 4 3
4
2 4
1
1 1 1
38
2 2 1
3 3 2 2
4 4 3 3
1 4 4
39
40
C++ implementation
void InsertionSort(int list[])
{
for (int i = 1; i <= n-1; i++) {
for(int j = i;j>=1; j--) {
if(list[j-1] > list[j])
{
int temp = list[j];
list[j] = list[j-1];
list[j-1] = temp;
}
else
break;
}
}
41
}
Complexity Analysis
The complexity of insertion sorting is
O(n) at best case of an already sorted
array and O(n2) at worst case,
regardless of the method of insertion.
Number of comparisons may vary
depending on the insertion algorithm.
O(n2) for shifting or swapping
methods.
O(nlogn) for binary insertion sort.
42
