Introduction to Algorithms

Md. Saidur Rahman

 Learning Outcomes/Objectives:

After undergoing this course, students should be able to:

• Understand the algorithmic concepts, analyze performance of

algorithms in terms of time and space, and prove correctness of
algorithms,
• formulate various algorithmic problems and design efficient
algorithms to solve those problems,
• solve real world problems using graph algorithms,
• utilize advanced data structures for efficient implementations of
algorithms,
• understand various complexity classes of algorithmic problems , and
• design backtracking, branch and bound and efficient approximation
algorithms to cope with hard combinatorial problems.
 Text and Reference Books:

• Algorithm Design, by Michael T. Goodrich and Roberto

Tamassia, John Wiley & Sons, Inc.
• Algorithms, by Sanjoy Dasgupta, Christos Papadimitriou
and Umesh Vazirani.
• Introduction to Algorithms, by Thomas H. Cormen, Charles
E. Leiserson, Ronald L. Rivest, and Clifford Stein, MIT Press.
• Algorithm Design , by Jon Kleinberg and Eva Tardos,
Pearsons Publishers.
• Introduction to the Design & Analysis of Algorithms, by
Anany Levitin.
 What is an Algorithm?

Algorithms are what we do in order not to have to do

something.

It is the work we do to avoid work.

Algorithms did not start with computers; they have

been with us from ancient times; nor they are limited
to computer science.
 What is an Algorithm?

Algorithms did not start with computers; they have

been with us from ancient times; nor they are limited
to computer science.
NYAB Talk 5
Graphs Algorithms

Leonhard Euler Muhammad ibn Musa al-Khwarizmi

(1707 – 1783) (c. 780 – 850)

NYAB Talk 6
 Algorithm
An algorithm is a sequence of instructions that one
must perform in order to solve a well-formulated
problem.

Problem ?

Input Output
?
Well-formulated Problem ? Unambiguous
precise
Why do we study algorithms?

Why do we need a good algorithm?

 Fibonacci Number: 1, 1, 2, 3, 5, 8, 13, ……

Fibonacci Problem: Calculate the nth Fibonacci number.

RecursiveFibonacci(n)

if n = 1 or n = 2
return 1
else
a  RecursiveFibonacci(n-1)
b  RecursiveFibonacci(n-2)
return a + b
Recursion Tree
n

n-1 n-2

n-2 n-3 n-3 n-4

n-3 n-4 n-4 n-5 n-4 n-5 n-5 n-6

As the tree grows larger, the number of duplicate vetices increases

exponentially 2i-2 at level i, while the number of regular vertices
increases linearly (2 per level).
Fibonacci(n)
F1  1
F2  1
for i  3 to n
Fi  Fi -1 + Fi-2

O(n) time
Algorithm Analysis

Correctness

Performance
Algorithm Correctness

A correct algorithm/program yields the

correct result for all legal input values.

?
Proving the correctness of an algorithm is not at all an easy
task. It consists of two steps

1) Proving that the algorithm will always terminate.

2) Proving that it will always produce correct result.
Linear Search Algorithm
Let X = [x1, x2, . . ., xn] be an unordered list of n
distinct items. We would like to search the list for a
specific item, called key. If key exists, in the list, the
algorithm should return the location of the key.
Algorithm linear_search(X,n, key, location)
0. Begin
1. x0  key
2. i n
3. while xi ≠ key do
4. ii–1
5. location  i
6. End
Theorem. Algorithm linear_search works correctly for
every n ≥ 0.
Proof. Let P(n): Algorithm linear_search returns the correct
location for every list of size n ≥ 0.

Basis: When n = 0, the while loop is skipped. The algorithm

returns the value 0 in location by line 5, which is correct.
Thus P(0) is correct.

Induction hypothesis: Assume that P(k) is true for an

arbitrary integer k ≥ 0; that is, the algorithm works when the
list contains k items.

Induction step: We will show that P(k + 1) is true; that is,

the algorithm works when the list contains k + 1 items.
Induction step: We will show that P(k + 1) is true; that is,
the algorithm works when the list contains k + 1 items.

We have two cases:

Case 1. If Xk + 1 = key in line 3, the while loop will not be

entered and the algorithm returns the correct value k + 1 for
location in line 5.

Case 2. If Xk + 1 ≠ key, then i = k at the end of first iteration.

This restricts us to a sublist with k elements. By induction
hypothesis, the algorithm works correctly for such a list.

In both cases, P(k +1) holds. Thus P(n) is true for n ≥ 0.

Algorithm Performance Growth of functions
The Big-Oh Notation Growth of functions

Let f, g: N  R. Then f(n) is of order at most g(n), if

a positive constant C and a positive integer n0 exist
such that |f(n)| ≤ C|g(n)| for every n ≥ n0.

In symbol we write f(n) = O(g(n).

The Big-Omega Notation Growth of functions

Let f, g: N  R. Suppose there is a positive

constant C and a positive integer n0 such that |
f(n)| ≥ C|g(n)| for every n ≥ n0. Then f(n) = Ω(g(n).
The Big-Theta Notation Growth of functions

Let f, g: N  R such that f(n) = O(g(n) and f(n) =

Ω(g(n). Then f(n) = Θ(g(n).
 A Survey of Common Running Times
• Computing the maximum O(n)
• Merging Two sorted Lists O(n)
• Sorting O(nlogn)
• Finding closest pair of points in 2D Brute-force O(n2)
• Finding two disjoint sets O(n3)
• Independent Set of Size k O(nk)
• Independent set of maximum size O(2n)
 Linear Time: O(n)
Linear time. Running time is proportional to
input size.
Computing the maximum. Compute maximum
of n numbers a1, …, an.

max  a1
for i = 2 to n {
if (ai > max)
max  ai
}
22
 Linear Time: O(n)
• Merge. Combine two sorted lists
• A = a1,a2,…,an with B = b1,b2,…,bn into sorted
whole.

i = 1, j = 1
while (both lists are nonempty) {
if (ai  bj) append ai to output list and increment i
else append bj to output list and increment j
}
append remainder of nonempty list to output list

23
 Quadratic Time: O(n ) 2

• Closest pair of points. Given a list of n points in the plane (x1,

y1), …, (xn, yn), find the pair that is closest.
• O(n2) solution. Try all pairs of points.

min  (x1 - x2)2 + (y1 - y2)2

for i = 1 to n {
for j = i+1 to n {
d  (xi - xj)2 + (yi - yj)2
if (d < min)
min  d
}
}
24
 Cubic Time: O(n ) 3
Set disjointness. Given n sets S1, …, Sn each of which is a subset of
1, 2, …, n, is there some pair of these which are disjoint?

• O(n3) solution. For each pairs of sets, determine if they are disjoint.

foreach set Si {
foreach other set Sj {
foreach element p of Si {
determine whether p also belongs to Sj
}
if (no element of Si belongs to Sj)
report that Si and Sj are disjoint
}
}
25
 Polynomial Time: O(n ) Time k
• Independent set of size k. Given a graph, are there k nodes such that no two
k is a constant
are joined by an edge?

• O(nk) solution. Enumerate all subsets of k nodes.

For each subset S of k nodes {
check whether S in an independent set
if (S is an independent set)
report S is an independent set
}
}

n  n (n 1) (n  2)  (n  k 1) nk

– Check whether S is an independent set=k  2).
O(k 
 
 k (k 1) (k  2)  (2) (1) k!
– Number of k element subsets =
– O(k2 nk / k!) = O(nk).
poly-time for k=17,
but not practical

26
 Exponential Time
• Independent set. Given a graph, what is
maximum size of an independent set?

• O(n2 2n) solution. Enumerate all subsets.

S*  
For each subset S of nodes {
check whether S in an independent set
if (S is largest independent set seen so far)
update S*  S
}
}
27
 Algorithm Design Techniques
• Exhaustive Search
• Branch and Bound
• Greedy Method
• Divide and conquer
• Dynamic Programming
• Machine Learning
• Randomized Algorithm
• Online algorithm