WOLKITE UNIVERSITY

College of Computing & Informatics

Department of Computer Science

Regular Program

Design and Analysis of Algorithms(CoSc3092)

Chapter One
Introduction and Elementary Data Structures
 Introduction to Algorithm analysis
• Asymptotic Notations
• Analysis of Algorithm
 Review of elementary Data Structures
• Heaps
• Hashing
• Set Representation
• UNION, FIND Operation

Design and Analysis of Algorithm Chapter-1 2

 Introduction
• To apply algorithm first of all there should be a problem that needs
solution
– Is there any problem for which we can’t design algorithm?

– Program = Algorithm + Data Structure

Design and Analysis of Algorithm Chapter-1 3
 What is an algorithm?
• An algorithm is a sequence of unambiguous instructions for
solving a problem, i.e., for obtaining a required output for any
legitimate input in a finite amount of time
• From the above definition, algorithm has the following five
properties: Sequence, Unambiguous, Input, Output, Finite

Design and Analysis of Algorithm Chapter-1 4

 Algorithms
 All algorithms must satisfy the following properties
1) Input. Zero more quantities are externally supplied.
2) Output. At least one quantity is produced.
3) Definiteness. Each instruction is clear and unambiguous.
4) Finiteness. the algorithm terminates after a finite number of
steps.
5) Effectiveness. It is not enough that each operation be
definite as in 3) it also must be feasible.

Design and Analysis of Algorithm Chapter-1 5

 Example: Searching

• Problem of searching an ordered list.

– Given a list L of n elements that are sorted into
a definite order (e.g., numeric, alphabetical),
– And given a particular element x,
– Determine whether x appears in the list, and if
so, return its index (position) in the list.

Design and Analysis of Algorithm Chapter-1 6

 Search alg. #1: Linear Search
//Searches for a given value in a given array by
the worst-case complexity is O(n)
sequential search
//Input: An array L[0 .. n -1] and a search key x
//Output: The index of the first element of L that
//matches x or -1 if there are no matching elements
i:=0
while i <n and L[i]  x do
i:=i+1
If i < n return i
else return -1
Design and Analysis of Algorithm Chapter-1 7
 Search alg. #2: Binary Search
• Basic idea: On each step, look at the middle element of the
remaining list to eliminate half of it, and quickly zero in on the
desired element.

<x <x <x >x

Design and Analysis of Algorithm Chapter-1 8

Search alg. #2: Binary Search

procedure binary search

(x:integer, a1, a2, …, an: distinct integers)
i := 1 {left endpoint of search interval}
j := n {right endpoint of search interval}
while i<j begin {while interval has >1 item}
m := (i+j)/2 {midpoint}
if x>am then i := m+1 else j := m
end
if x = ai then location := i else location := 0
return location

Design and Analysis of Algorithm Chapter-1 9

 Is Binary Search more efficient?
• Number of computations per iteration:
– Binary search does more computations than Linear Search per
iteration.
• Overall:
– If the number of components is small (say, less than 20), then Linear
Search is faster.
– If the number of components is large, then Binary Search is faster.

Design and Analysis of Algorithm Chapter-1 10

Algorithm Design & Analysis Process

Design and Analysis of Algorithm Chapter-1 11

 Analysis of Algorithm
• As we may have noted, there are often multiple algorithms one
can use to solve the same problem.
– In searching, one can use linear search, binary search, …..
–For sorting a sequence of list, we may consider bubble sort, selection sort,
quick sort, merge sort, heap sort,…
–We can come up with our own variants.
• Why all these algorithms? Do we need them?
• How do we choose which algorithm is the best?
–The fastest/most efficient algorithm.
–The one that uses the fewest resources.
–The clearest.
–The shortest, ...
Design and Analysis of Algorithm Chapter-1 12
 Analysis of Algorithm…
• The analysis of algorithms is the process of finding the computational
complexity of algorithms
—the amount of time, storage, or other resources needed to execute them.
–An algorithm that solves a problem but requires a year and GBs of main
memory is hardly of any use.
•The objective of algorithm analysis is
–To measure resources (e.g., time, space) requirements of an algorithm so as to
determine how quickly (with less memory) an algorithm executes in practice
• An algorithm should make efficient use of computer resources
–Most frequently, we look at efficiency, i.e.
• how long does the algorithm take to run
• What is the best way to represent the running time of an algorithm?

Design and Analysis of Algorithm Chapter-1 13

 Efficiency
• An algorithm must solve a problem with the least amount of
computational resources such as time and space.
–An algorithm should run as fast as possible using as little memory as
possible
•Two types of algorithmic efficiency evaluation
–Time efficiency-indicates how fast the algorithm runs
–Space efficiency-indicates how much memory the algorithm needs
• What to analyze?
–To keep things simple, we will concentrate on the running time of
algorithms and will not look at the space (the amount of memory) needed
or required.
–So, efficiency considerations of algorithm usually focus on the amount of
time elapsed (called running time of an algorithm) when processing data.

Design and Analysis of Algorithm Chapter-1 14

 Correct Algorithm
• A correct algorithm solves the given computational problem
–If the algorithm is not doing what it is supposed to do, it is worthless
• An algorithm is said to be correct if, for everyinput instance, it halts
with the correct output
• An incorrect algorithm
–might not halt at all on some input instances, or
–might halt with a wrong answer.
• In order to show that an algorithm is incorrect, we need just one
instance of its input for which the algorithm fails.

Design and Analysis of Algorithm Chapter-1 15

Why need algorithm analysis ?
• Suppose computers are infinitely fast and computer memory was free.
Would you have any reason to study algorithm?
–Yes, because we want to demonstrate that the algorithm is correct;
it terminates with the intended solution for all inputs given.
• However, the reality shows that :
– Computers may be fast, but they are not infinitely fast
– Memory may be cheap, but it is not free
– Computing time and resources are therefore a bounded resources

Design and Analysis of Algorithm Chapter-1 16

 Why need algorithm analysis ?...
• There are many ways to solve a given problem
– So writing a working program to solve a problem is not good enough
• The program may be inefficient and/or incorrect!
• If the program is run on a large data set, then the running time
becomes an issue.
• To discover the characteristics of algorithm in order to evaluate its
suitability for various applications or compare it with other
algorithms for the same application
• Always we have to undertake algorithm analysis before
implementation
• Example: Selection Problem
–Given a list of N numbers, determine the kth largest, where k ≤ N.

Design and Analysis of Algorithm Chapter-1 17

 Example: Selection Problem
 Algorithm 1:
(1) Read N numbers into an array
(2) Sort the array in decreasing order by some simple algorithm
(3) Return the element in kth position

Design and Analysis of Algorithm Chapter-1 18

 Example: Selection Problem…
 Algorithm 2:
(1) Read the first k elements into an array and sort them in decreasing
order
(2) Each remaining element is read one by one
–If smaller than the kth element, then it is ignored
–Otherwise, it is placed in its correct spot in the array, bumping one element out
of the array.
(3) The element in the kth position is returned as the answer.

Design and Analysis of Algorithm Chapter-1 19

 Areas of studies of Algorithms
1. How to devise algorithms-The act of creating an algorithm is an art
which may never be fully automated. A major goal of this course is
to study various design techniques which have proven to be useful
in that they have often yielded good algorithms. By mastering
these design strategies, it will become easier for you to devise new
and useful algorithms.
2. How to validate algorithms-Once an algorithm is devised it is
necessary to show that it computes the correct answer for all
possible legal inputs. We refer to this process as algorithm
validation.

Design and Analysis of Algorithm Chapter-1 20

 Areas of studies of Algorithms
3. How to analyze algorithms-This field of study is called analysis of
algorithms. Analysis of algorithms refers to the process of
determining how much computing time and storage an algorithm
will require. This area is a challenging one which sometimes
requires great mathematical skill. One important result of this study
is that it allows one to make quantitative judgments about the value
of one algorithm over another.
4. How to express algorithms-The clear and concise expression of
algorithms in a programming language.

Design and Analysis of Algorithm Chapter-1 21

 Areas of studies of Algorithms
5. How to test a program-Testing a program really consists of two
phases: debugging and profiling.
– Debugging is the process of executing programs on sample data
sets to determine if faulty results occur and, if so, to correct
them.
– Profiling is the process of executing a correct program on data
sets and measuring the time and space it takes to compute the
results.

Design and Analysis of Algorithm Chapter-1 22

 Types of Analysis
• The efficiencies of some algorithms may differ significantly for
inputs of the same size. In this course we will perform the
following types of analysis:
• Best Case running time
• Average Case running time
• Worst Case running time
• Which one is good? Discuss.
• Take as an Example the Linear Search

Design and Analysis of Algorithm Chapter-1 23

 best-case Running time…
• The best-case running time complexity of the algorithm is the
function defined by the minimum number of steps taken on any
instance of size N.
• The term best-case running time is used in computer science to
describe an algorithm's behavior under optimal conditions.
• Best-case running time: The shortest running time for any input
of size n. The algorithm will never be faster than this.
• For example,-the best case for a simple linear search on a list
occurs when the desired element is the first element of the list.
- The best case for a sorting algorithm would be data that's already sorted.
• Development and choice of algorithms is rarely based on best-case
performance
Design and Analysis of Algorithm Chapter-1 24
 Average-case Running time…
• the average-case running time complexity of the algorithm is the
function defined by an average number of steps taken on any
instance of size N.
• We take all possible inputs and calculate computing time for all of the inputs.
Sum all the calculated values and divide the sum by total number of inputs.
• We must know (or predict) distribution of cases.
• For the linear search problem, let us assume that all cases are uniformly
distributed (including the case of x not being present in array).
• So we sum all the cases and divide the sum by (n+1). Following is the value of
average case time complexity.

Design and Analysis of Algorithm Chapter-1 25

 worst-case Running time…
• The worst-case runtime complexity of the algorithm is the function defined by the
maximum number of steps taken on any instance of size N.
• We shall usually concentrate on finding only the worst-case running time, that is, the
longest running time for any input of size N.
• The worst-case running time of an algorithm gives us an upper bound on the running
time for any input.
• Knowing it provides a guarantee that the algorithm will never take any longer.
• When not otherwise specified, the function describing the performance of an
algorithm is usually an upper bound, determined from the worst case inputs to the
algorithm
• For example, the worst case for a sorting algorithm might be data that's sorted in
reverse order
– The worst case for a simple linear search on a list occurs when the desired element
is the last element of the list.Design and Analysis of Algorithm Chapter-1 26
 Performance Analysis
1. Space Complexity- S(P): The space complexity of a program is the
amount of memory it needs to run to completion.
• The space requirement S(P) of any program P is written as S(P) = c +Sp
where c is a constant(fixed part) and Sp(variable part/Instance
characteristics)
– A fixed part that is independent of the characteristics of the inputs and outputs.
typically includes the space for simple variables and space for constants, etc.
– A variable part that consists of the space needed by component variables whose
size is dependent on the particular problem instance being solved

Design and Analysis of Algorithm Chapter-1 27

 Performance Analysis
2. Time Complexity-T(P): The time, T(P), taken by a program P
is the sum of the compile time and the run (or execution) time.
• The compile time does not depend on the instance
characteristics. We focus on the run time of a program, denoted
by T(P) (instance characteristics).

Design and Analysis of Algorithm Chapter-1 28

 Computing running time
Algorithm 1 Algorithm 2
Cost Cost
arr[0] = 0; c1 for(i=0; i<N; i++) c2
arr[1] = 0; c1 arr[i] = 0; c1
arr[2] = 0; c1
...
arr[N-1] = 0; c1
----------- -------------
c1+c1+...+c1 = c1 x N (N+1) x c2 + N x c1 =
(c2 + c1) x N + c2

O(n)

Design and Analysis of Algorithm Chapter-1 29

Computing running time (cont.)
Cost
sum = 0; c1
for(i=0; i<N; i++) c2
for(j=0; j<N; j++) c2
sum += arr[i][j]; c3
------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N x N
O(n2)
Design and Analysis of Algorithm Chapter-1 30
 Examples (cont.’d)
if (i<j)
for ( i=0; i<N; i++ ) O(N)

X = X+i;
else O(1)
X=0;

Max ( O(N), O(1) ) = O (N)

Design and Analysis of Algorithm Chapter-1 31

 Example cont.’d
Algorithm swap(a, b)
{
temp= a;…………………1
a=b; ……………………..1
b= temp; …………………1
}
• Every simple statement is counted as 1 unit of time hence f(n)= 3, O(1)
• Space complexity is calculated for a, b and temp; hence S(n) =3, O(1)

Design and Analysis of Algorithm Chapter-1 32

 computational complexity of algorithm that sum up
the elements of an array
Algorithm sum(A, n)
{ s=0;………………… 1
for(i=0;i<n;i++)………… n+1
{ s=s+A[i]; } ……………… n
return s; …………………… 1
}}
• The time complexity is f(n)= 2n+3 O(n)
• The space complexity is calculated as 1unit for n,s,i and n for A, then
S(n)= n+3  O(n)

Design and Analysis of Algorithm Chapter-1 33

 computational complexity of algorithm that sum up the
elements of two a matrix

Algorithm(A, B, n)
{
for(i=0; i<n; i++){………………………………..n+1
for(j=0; j<n; j++)…………………………………..n*(n+1)
{
C[i, j]= A[i, j]+ B[i, j];………….. n*n
}}
• Time complexity is f(n)=2n2+2n+1O(n2)
• The space complexity is A=n2, B=n2, C=n2 , n=1, i=1, j=1
• hence S(n)=3n2 +3 O(n2)

Design and Analysis of Algorithm Chapter-1 34

 computational complexity of algorithm that multiply the elements of
the matrix
Algorithm(A, B, n)
{for(i=0; i<n; i++){………………………..n+1
for(j=0;j<n; j++)………………………… n*(n+1)
{ C[i,j]= 0;………….. n*n
for(k=0;k<n; k++)………………………………… n*n*(n+1)
{C[i,j]= C[i,j]+ A[i,k]* C[k,j]………….. n*n*n
} }
• Time complexity is f(n)=2n3 +3n2+2n+1O(n3)
• The space complexity for matrix is A=n2, B=n2, C=n2 , n=1, i=1, j=1, k=1
• Hence S(f)=3n2 +4 O(n2)

Design and Analysis of Algorithm Chapter-1 35

Time complexity of for loop

Design and Analysis of Algorithm Chapter-1 36

 Time complexity of while loop

i=0……………….. 1 a=1
While(i<n)…….n+1 While(a<b)
{ {
Statement;…………..n Statement;
i++;………………………………n a=a*2;
} }
T(n)=3n+2O(n) T(n)=log2bO(logn)

Design and Analysis of Algorithm Chapter-1 37

 Time complexity of if…else
Algorithm test(n)
{
if(n<7) {
statement;
}
else
{for(i=0;i,<n; i++){statement;}
}}
• If the condition is true f(n)=O(1) else f(n)= O(n)

Design and Analysis of Algorithm Chapter-1 38

 How do we analyze algorithms?
• We need to define a number of objective measures.
(1) Compare execution times?
Not good: times are specific to a particular computer !!
(2) Count the number of statements executed?
Not good: number of statements vary with the
programming language as well as the style of the
individual programmer.

Design and Analysis of Algorithm Chapter-1 39

 Example (number of statements)

Algorithm 1 Algorithm 2
arr[0] = 0; for(i=0; i<N; i++)
arr[1] = 0; arr[i] = 0;
arr[2] = 0;
arr[3] = 0;
...
arr[N-1] = 0;

Design and Analysis of Algorithm Chapter-1 40

 How do we analyze algorithms?
(3) Step count
Iterative function to sum a list of numbers
steps/execution
Statement s/e Frequency Total steps
float sum(float list[ ], int n) 0 0 0
{ 0 0 0
float tempsum = 0; 1 1 1
int i; 0 0 0
for(i=0; i <n; i++) 1 n+1 n+1
tempsum += list[i]; 1 n n
return tempsum; 1 1 1
} 0 0 0
Total 2n+3

Design and Analysis of Algorithm Chapter-1 41

 Step count
• Determining step counts help us to compare the time
complexities of two programs and to predict the growth in run
time as the instance characteristics change.
• But determining exact step counts could be very difficult. Since
the notion of a step count is itself inexact, it may be worth the
effort to compute the exact step counts.

Design and Analysis of Algorithm Chapter-1 42

 How do we analyze algorithms?...
(4) Express running time as a function of the input size n (i.e., f(n)). [Order
of growth] [Asymptotic Analysis]
– To compare two algorithms with running times f(n) and g(n), we need
a rough measure of how fast a function grows. [Order of growth]
– Such an analysis is independent of machine time, programming style,
etc.
• Running time analysis is determining how running time increases as the
size of the problem increases
• We usually consider one algorithm to be more efficient than another if its
worst case running time has a smaller order of growth.

Design and Analysis of Algorithm Chapter-1 43

 Asymptotic Analysis
Rate of Growth:
• The following notations are commonly used notations in
performance analysis and used to characterize the complexity
of an algorithm:
1. Big–OH (O)
2. Big–OMEGA (Ω)
3. Big–THETA (Θ)

Design and Analysis of Algorithm Chapter-1 44

 Big-O Notation (Upper Bound)
• f(n) = O(g(n)), (pronounced order of or big oh), says that the growth
rate of f(n) is less than or equal (<=) that of g(n).
• We can replace f(n) by g(n) and know we aren't “off too far.”
• We say f(n) is "in the order of g(n)" or, simply, f(n)  O(g(n)).
• Usually, g(n) is a simple function, like, n, n2, nlog(n), n3, 2n, etc., that's
easy to understand and use.
• Useful when f(n) is not known precisely, is complicated to compute,
and/or difficult to use.
• Basically, it tells you how fast a function grows or declines as the
input grows
Design and Analysis of Algorithm Chapter-1 45
Design and Analysis of Algorithm Chapter-1 46
 Big-O Notation…
• Let f , g be any function R  R. We say that
f (n) = O(g(n))
• if there are positive integers c and n0 such that for all n ≥ n0
f (n) ≤ cg(n)
• When f (n) = O(g(n)) we say that g is an asymptotic upper bound on f .
• In asymptotic analysis,
– The high order term will dominate low order terms when the input is
sufficiently large
– We also discard constant coefficients for convenience, since we are
estimating
• Example: The time complexity is given by the f(n) below:
– f(n) = 6n3 + 2n2 + 20n + 45O(n3)

Design and Analysis of Algorithm Chapter-1 47

Design and Analysis of Algorithm Chapter-1 48
 More Examples …
• We say that n4 + 100n2 + 10n + 50 is of the order of n4 or O(n4)
• We say that 10n3 + 2n2 is O(n3)
• We say that n3 - n2 is O(n3)
• We say that 10 is O(1),
• We say that 1273679 is O(1)
• In general, an O(n2) algorithm will be slower than O(n) algorithm.
• Warning: an O(n2) function will grow faster than an O(n) function

Design and Analysis of Algorithm Chapter-1 49

 Eight Growth Functions
• Eight functions O(n) that occur frequently in the analysis of
algorithms (in order of increasing rate of growth relative to n):
– Constant  1
– Logarithmic  log n
– Linear  n
– Log Linear  n log n
– Quadratic  n2
– Cubic  n3
– Exponential  2n
– Exhaustive Search  n!

Design and Analysis of Algorithm Chapter-1 50

 Growth Rates Compared
n=1 n=2 n=4 n=8 n=16 n=32
1 1 1 1 1 1 1
logn 0 1 2 3 4 5
n 1 2 4 8 16 32
nlogn 0 2 8 24 64 160
n2 1 4 16 64 256 1024
n3 1 8 64 512 4096 32768
2n 2 4 16 256 65536 4294967296
n! 1 2 24 40320 20.9T Don’t ask!
Design and Analysis of Algorithm Chapter-1 51
(g), at least order g [Lower Bound]
Let f,g be any function RR.
• We say that “f is at least order g”, written (g), if
c,k: f(x)  cg(x), x>k
– “Beyond some point k, function f is at least a constant c times g (i.e.,
proportional to g).”
• “f is at least order g”, or “f is (g)”, or “f= (g)” all just mean
that f (g).

52
Big-  Visualization

Design and Analysis of Algorithm Chapter-1 53

 Big- Θ
• Definition: f(n) = Θ(g(n)) iff there exist positive constants c1,
c2, and n0 such that c1g(n) ≤ f(n) ≤ c2g(n) for all n, n ≥ n0.
• If a program P has complexities Θ(n) and program Q has
complexities Θ(n2), then, in general, we can assume program P
is faster than Q for a sufficient large n.

Design and Analysis of Algorithm Chapter-1 54

Big- Θ Visualization

Design and Analysis of Algorithm Chapter-1 55

 Review of elementary Data Structures
• Heaps
• Hashing
• Set Representation
• UNION, FIND Operation

Design and Analysis of Algorithm Chapter-1 56

 The Heap Data Structure
• Heap is a data structure, which permits one to insert elements
into a set and also to find the largest element efficiently
• A heap is a complete binary tree

• The height of a complete binary tree is logn

57
 Heap Types
• Max-heaps (largest element at root), have the max-heap
property:
– for all nodes i,
A[PARENT(i)] ≥ A[i]
• Min-heaps (smallest element at root), have the min-heap
property:
– for all nodes i,
A[PARENT(i)] ≤ A[i]

58
 Array Representation of Heaps
• A heap can be stored as an array A.
– Root of tree is A[1]
– Left child of A[i] = A[2i]
– Right child of A[i] = A[2i + 1]
– Parent of A[i] = A[ i/2 ]
• Creating a heap from the given
array element is done in O(nlogn)
time

59
 Adding/Deleting Nodes
• New nodes are always inserted at the bottom level (left to
right)
• Nodes are removed from the root then replace the node with
the bottom element (right to left) then maintain the heap.
• Deleting or inserting an element from a heap is done in
O(logn) time

60
 Operations on Heaps
1. Maintain/Restore the max-heap property
- MAX-HEAPIFY
2. Create a max-heap from an unordered array
- BUILD-MAX-HEAP
3. Sort an array in place
- HEAPSORT

61
1. Maintaining the Heap Property (MAX-HEAPIFY)
• Suppose a node is smaller
than a child
– Left and Right sub trees of i
are max-heaps
• To eliminate the violation:
– Exchange with larger child
– Move down the tree
– Continue until node is not
smaller than children

62
 Example
MAX-HEAPIFY(A, 2, 10)

A[2]  A[4]

A[2] violates the heap property A[4] violates the heap property

A[4]  A[9]

Heap property restored

63
 Maintaining the Heap Property
• Assumptions: Alg: MAX-HEAPIFY(A, i, n)
– Left and Right 1. l ← LEFT(i)
subtrees of i are
2. r ← RIGHT(i)
max-heaps
3. if l ≤ n and A[l] > A[i]
– A[i] may be
smaller than its 4. then largest ←l
children 5. else largest ←i
6. if r ≤ n and A[r] > A[largest]
7. then largest ←r
8. if largest  i
9. then exchange A[i] ↔ A[largest]
10. MAX-HEAPIFY(A, largest, n)
64
 MAX-HEAPIFY Running Time
• Intuitively:
-
-
-
-

• Running time of MAX-HEAPIFY is O(logn)

• Can be written in terms of the height of the heap,

as being O(h)
– Since the height of the heap is logn

65
 2. Building a Heap- BUILD-MAX-HEAP
• Convert an array A[1 … n] into a max-heap (n = length[A])
• The elements in the subarray A[(n/2+1) .. n] are leaves
• Apply MAX-HEAPIFY on elements between 1 and n/2

Alg: BUILD-MAX-HEAP(A) 1

1. n = length[A] 4

for i ← n/2 downto 1

2 3
2. 1 3
4 5 6 7

3. do MAX-HEAPIFY(A, i, n) 8
2 9 10
16 9 10
14 8 7

A: 4 1 3 2 16 9 10 14 8 7

66
Example: A 4 1 3 2 16 9 10 14 8 7

i=5 i=4 i=3

1 1 1

4 4 4
2 3 2 3 2 3

1 3 1 3 1 3
4 5 6 7 4 5 6 7 4 5 6 7

8
2 9 10
16 9 10 8 2 9 10
16 9 10 8 14 9 10
16 9 10
14 8 7 14 8 7 2 8 7

i=2 i=1
1 1 1

4 4 16
2 3 2 3 2 3

1 10 16 10 14 10
4 5 6 7 4 5 6 7 4 5 6 7

8
14 9 10
16 9 3 8
14 9 10
7 9 3 8
8 9 10
7 9 3
2 8 7 2 8 1 2 4 1
67
 Running Time of BUILD MAX HEAP

Alg: BUILD-MAX-HEAP(A)
1. n = length[A]
2. for i ← n/2 downto 1
O(n)
3. do MAX-HEAPIFY(A, i, n) O(logn)

 Running time: O(nlogn)

68
 3. Heap Sort
• A conceptually simple sorting strategy which continually removes the
maximum value from the remaining unsorted elements.

• Goal: Sort an array using heap representations

• Idea:
– Build a max-heap from the array
– Swap the root (the maximum element) with the last element
in the array
– “Discard” this last node by decreasing the heap size
– Call MAX-HEAPIFY on the new root
– Repeat this process until only one node remains
69
Example: A=[7, 4, 3, 1, 2]

MAX-HEAPIFY(A, 1, 4) MAX-HEAPIFY(A, 1, 3) MAX-HEAPIFY(A, 1, 2)

MAX-HEAPIFY(A, 1, 1)

70
 Alg: HEAPSORT(A)

1. BUILD-MAX-HEAP(A) O(n)
2. for i ← length[A] downto 2
3. do exchange A[1] ↔ A[i] n-1 times

4. MAX-HEAPIFY(A, 1, i - 1) O(logn)

• Running time: O(nlogn)

71
 Set Representation
• Trees can be used to represent sets.
• A set is a collection of objects.

• Set A is a subset of set B if all elements of A are in B.

– Subsets are sets
• Union of two sets A and B is a set C which consists of all
elements in A and B
• Two sets are mutually disjoint if they do not have a common
element.

72
 Union and Find operations
• Disjoint set Union
• Need to form union of two different sets of a partition
– If Si and Sj are two disjoint sets, then their union
Si ∪Sj = {all elements x such that x is in Si or Sj}.

• Find (i)
– Find the set containing element i.
• Need to find out which set an element belongs to

73
Possible Tree Representation of Sets

Set 2 = {4 , 1 ,9}
S1 S2 S3

0 4 2

6 7 8 1 9 3 5

Set I = {0 , 6 ,7 ,8 } Set 3 = {2 , 3 , 5}

74
 Unions of Sets
• To obtain the union of two sets, just set the parent field of one of
the roots to the other root.
• To figure out which set an element is belonged to, just follow its
parent link to the root and then follow the pointer in the root to the
set name.
• It is useful to identify the existence of a cycle in the graph.

75
Possible Representations of S1 ∪S2

S1 S2

0 4

S2

6 7 8 4 1 9

1 9

76
Possible Representations of S1 ∪S2

S1 S2

0 4

S1
6 7 8 0 1 9

6 7 8

77
 Hashing
• A symbol table is a data structure which allows one to easily determine the
presence or absence of an arbitrary element.
• Hashing is the most practical technique for maintaining a symbol table.
• The implementation of hash tables is called hashing.
• Hashing is a technique used for performing insertions, deletions and finds in
constant average time (i.e. O(1))
• This data structure, however, is not efficient in operations that require any
ordering information among the elements, such as findMin, findMax and
printing the entire table in sorted order.

78
 Example
Hash
Table
0
1
Items
2
john 25000
3 john 25000
phil 31250 key Hash 4 phil 31250
Function
dave 27500 5
mary 28200 6 dave 27500
7 mary 28200
key 8
9

79
 Operations
• Initialization: all entries are set to NULL
• Find:
– locate the cell using hash function.
– sequential search on the linked list in that cell.
• Insertion:
– Locate the cell using hash function.
– (If the item does not exist) insert it as the first item in the list.
• Deletion:
– Locate the cell using hash function.
– Delete the item from the linked list.

80
 Do you see any problems
with this approach?
0

U
(universe of keys) h(k1)
h(k4)

K k1
h(k2) = h(k5)
(actual k4 k2
keys) Collisions!
k5 k3 h(k3)

m-1
•Two or more keys hash to the same slot!!
•Avoiding collisions completely is hard,
even with a good hash function
•If there are too many collisions, performance of the hash table will suffer
dramatically. 81
 Review literature on the following topics
1. Array vs. linked list data structure: Overview, pros & cons, types of arrays/linked list, algorithm for
basic operations, analyze their complexity
2. Stack vs. Queue data structures: Overview, pros & cons, types of stack/queue: algorithm for basic
operations, analyze their complexity
3. Tree data structures: Overview, unbalanced & balanced tree, tree traversal, algorithm for basic
operations, analyze their complexity
4. Sorting algorithms (include advanced algorithms in your discussion): Overview, discuss sorting
algorithm, show bubble & quick algorithm, analyze their complexity
5. Searching algorithms (include advanced algorithms in your discussion): Overview, discuss searching
algorithms, show linear & binary algorithms, analyze their complexity

Design and Analysis of Algorithm Chapter-1 82

THE END

Design and Analysis of Algorithm Chapter-1 83