DAA Notes Module 1

Vivekananda
College
e of Engineering & Technolog
ogy
N
Nehru nagar post, Puttur, D.K. 5742013
Lecture Notes on
15CS43
Design and Analysis
lysis of
Algorithms
(CBCS Scheme)
Prepared by
Mr. Harivinod N
Dept. of Computer
C Science and Engineering,
VCET Puttur
Jan 2017
Module--1 : Introduction to Algorithms

ms
Contents
1. Introduction 4. Important Proroblem Types

1.1. What is an Algo
gorithm? 4.1. Sorting
1.2. Algorithm Speci
ecification 4.2. Searching
ing
1.3. Analysis Framewo
work 4.3. String pro
processing
2. Performance Analysiysis 4.4. Graph Problems
Pro
2.1. Space complexit
xity 4.5. Combinat
natorial Problems
2.2. Time complexityity 5. Fundamental al Data
D Structures
3. Asymptotic Notation ons 5.1. Linear
near Data Structures
3.1. Big-Oh notation
on 5.2. Graphs
3.2. Omega notationon 5.3. Trees
3.3. Theta notation 5.4. Sets andd Dictionaries.
D
3.4. Little-oh notatio
tion
3.5. Mathematical ana
nalysis
Cours
ourse website: www.techjourney.in
Lecture Notes | 15CS43
CS43 Design & Analysis of Algorithms | Module 1: Introdu
troduction
1. Introduction
1.1. What is an Algorithm??
An algorithm is a finite setet of instructions to solve a particular proble lem. In addition, all
algorithms must satisfy the following
fol criteria:
a. Input. Zero or morore quantities are externally supplied.
b. Output. At least one
on quantity is produced.
c. Definiteness. Eachch instruction is clear and unambiguous. It mus ust be perfectly clear
what should be donone.
d. Finiteness. If wee trace
t out the instruction of an algorithm, the hen for all cases, the
algorithm terminate
ates after a finite number of steps.
e. Effectiveness. Ever
very instruction must be very basic so that it ca can be carried out, in
principle, by a person
per using only pencil and paper. It is not ot enough that each
operation be defini
inite as in criterion c; it also must be feasible.
Algorithm design and analy lysis process - We now briefly discuss a seq
equence of steps one
typically goes through in desig
signing and analyzing an algorithm
Understanding the Prob oblem - From a practical perspective, the firstrst thing you need to
do before designing an algorithm
al is to understand completely the pr problem given. An
input to an algorithm spec
ecifies an instance of the problem the algorithm
thm solves. It is very
important to specify exactl
ctly the set of instances the algorithm needs to handle.
Ascertaining the Capab bilities of the Computational Device - Onc nce you completely
understand a problem, you
ou need to ascertain the capabilities of the com
computational device
the algorithm is intended
ded for. Select appropriate model from seq equential or parallel
programming model.
Prerpared by Harivinod N www.techjourney.in Page|1.2

troduction
Choosing between Exac act and Approximate Problem Solving - The T next principal
decision is to choose betw
tween solving the problem exactly and solving
ing it approximately.
Because, there are importa
rtant problems that simply cannot be solvedd exactly
e for most of
their instances and somee of
o the available algorithms for solving a prob
oblem exactly can be
unacceptably slow because
use of the problems intrinsic complexity.
Algorithm Design Tech chniques - An algorithm design techniquee (or strategy or
paradigm) is a generall approach
a to solving problems algorithmically
lly that is applicable
to a variety of problemss from
f different areas of computing. They provide
pro guidance for
designing algorithms foror new problems, i.e., problems for which tthere is no known
satisfactory algorithm.
Designing an Algorithm m and Data Structures - One should pay ay close attention to
choosing data structuress appropriate
a for the operations performed byy the algorithm. For
example, the sieve of Erat
ratosthenes would run longer if we used a linke
ked list instead of an
ion. Algorithms + Data Structures = Program
array in its implementation ams
Methods of Specifying an Algorithm- Once you have designed an algorithm;
al you need
to specify it in some fashion.
fas These are the two options that are re most widely used
nowadays for specifyingg algorithms.
a Using a natural language hass an a obvious appeal;
however, the inherent ambiguity
am of any natural language makes a concise and clear
description of algorithmss surprisingly difficult. Pseudocode is a mi mixture of a natural
language and programmin ing language like constructs. Pseudocode is usually
us more precise
than natural language, and
nd its usage often yields more succinct algorithm
thm descriptions.
Proving an Algorithms s Correctness - Once an algorithm has beenn sspecified, you have
to prove its correctness.. That
T is, you have to prove that the algorithm
hm yields a required
result for every legitimate
te input in a finite amount of time. For some algorithms, a proof
of correctness is quite easy
asy; for others, it can be quite complex. A com
ommon technique for
proving correctness is to use mathematical induction because an alg lgorithms iterations
provide a natural sequencece of steps needed for such proofs.
Analyzing an Algorithm m - After correctness, by far the most importartant is efficiency. In
o algorithm efficiency: time efficiency, indic
fact, there are two kindss of dicating how fast the
algorithm runs, and space
ce efficiency, indicating how much extra memo mory it uses. Another
desirable characteristic of an algorithm is simplicity. Unlike efficieniency, which can be
precisely defined and inve
vestigated with mathematical rigor, simplicity,
ty, like beauty, is to a
considerable degree in the
he eye of the beholder.
Coding an Algorithm - Most algorithms are destined to be ultimate ately implemented as
computer programs. Impleplementing an algorithm correctly is necessary ry but not sufficient:
you would not like to dim
iminish your algorithms power by an inefficie ient implementation.
Modern compilers do provrovide a certain safety net in this regard, especi
ecially when they are
used in their code optimiza
ization mode.

troduction
1.2. Algorithm Specification

tion
An algorithm can be specified
ed in
1) Simple English
2) Graphical representatio
tion like flow chart
3) Programming language
age like c++ / java
4) Combination of above
ve methods.
Using the combination of simple
si English and C++, the algorithm for selection sort is
specified as follows.
In C++ the same algorithm can be specified as follows
Here Type is a basic or user defined

de data type.
Recursive algorithms
An algorithm is said to be recursive
re if the same algorithm is invokedd iin the body (direct
recursive). Algorithm A is said
aid to be indirect recursive if it calls another
er algorithm which in
turn calls A.
Example 1: Factorial computa
tation n! = n * (n-1)!
Example 2: Binomial coefficie
cient computation
Example 3: Tower of Hanoii problem

pr
Example 4: Permutation Gene
nerator

troduction
1.3. Analysis Framework

General framework for analy lyzing the efficiency of algorithms is discuss
ssed here. There are
two kinds of efficiency: timee efficiency and space efficiency. Time efficie
iciency indicates how
fast an algorithm in questionn runs; space efficiency deals with the extra space
s the algorithm
requires.
In the early days of electronic
ic computing, both resources time and space were w at a premium.
Now the amount of extra spac ace required by an algorithm is typically nott oof as much concern,
In addition, the research exper
perience has shown that for most problems, we can achieve much
more spectacular progress in speed than in space. Therefore, followingg a well-established
tradition of algorithm textbook
oks, we primarily concentrate on time efficienc
ency.
Measuring an Inputs Size

It is observed that almost all
ll algorithms
a run longer on larger inputs. For
Fo example, it takes
longer to sort larger arrays,, multiply
m larger matrices, and so on. Therefo
efore, it is logical to
investigate an algorithm's efficiency
ef as a function of some parameterter n indicating the
algorithm's input size.
There are situations, where the choice of a parameter indicating an input ut size does matter.
The choice of an appropriatete size metric can be influenced by operationss of the algorithm in
question. For example, how ow should we measure an input's size for a spell-checking
algorithm? If the algorithm examines individual characters of its inpu put, then we should
measure the size by the numb mber of characters; if it works by processing
ng words, we should
count their number in the inpu
put.
We should make a special note
no about measuring the size of inputs for algorithms
alg involving
properties of numbers (e.g e.g., checking whether a given integer n is prime). For such
ists prefer measuring size by the number b off bits
algorithms, computer scientist b in the n's binary
representation: log n 1. This metric usually gives a better ideaa aabout the efficiency
of algorithms in question.
Units for Measuring Runnin

ing lime
To measure an algorithm's efficiency,
eff we would like to have a metric thahat does not depend
on these extraneous factors.. One possible approach is to count the numbber of times each of
the algorithm's operations iss executed.
e This approach is both excessivelyy difficult
d and, as we
shall see, usually unnecessaryry. The thing to do is to identify the most impportant operation of
the algorithm, called the basasic operation, the operation contributing the most to the total
running time, and compute the number of times the basic operation is execu ecuted.
For example, most sorting algorithms
al work by comparing elements (keys)(ke of a list being
sorted with each other; for suc
uch algorithms, the basic operation is a key com
omparison.
As another example, algorith
rithms for matrix multiplication and polyn
lynomial evaluation
require two arithmetic operatio
tions: multiplication and addition.

troduction
Let cop be the execution timee of an algorithm's basic operation on a partic

ticular computer, and
let C(n) be the number of tim
imes this operation needs to be executed for this
th algorithm. Then
we can estimate the runningg time T(n) of a program implementing this is algorithm on that
computer by the formula:
unless n is extremely large or very small, the formula can give a reasona
nable estimate of the
algorithm's running time.
It is for these reasons that the
he efficiency analysis framework ignores multi
ltiplicative constants
and concentrates on the count nt's order of growth to within a constant mul
ultiple for large-size
inputs.
Orders of Growth
Why this emphasis on the count's
co order of growth for large input sizes?
s? Because for large
values of n, it is the function's
n's order of growth that counts: just look at ta
table which contains
values of a few functions parti
rticularly important for analysis of algorithms.
Table: Values
of several
functions
important for
analysis of
algorithms
Algorithms that require an exponential

ex number of operations are practica
tical for solving only
problems of very small sizes.
Worst-Case, Best-Case, and

d Average-Case Efficiencies
Definition: The worst-casee efficiency
e of an algorithm is its efficiencyy for the worst-case
input of size n, for which thee algorithm
a runs the longest among all possible
le inputs of that size.
Consider the algorithm for seq
equential search.

troduction
The running time of above algorithm

al can be quite different for the same
me list size n. In the
worst case, when there are no matching elements or the first matching element
el happens to
be the last one on the list,, the algorithm makes the largest number of key comparisons
si n: Cworst(n) = n.
among all possible inputs off size
In general, we analyze the algorithm
alg to see what kind of inputs yield thee llargest value of the
basic operation's count C(n) among
a all possible inputs of size n and then compute
c this worst-
case value Cworst (n). The wors orst-case analysis provides algorithm's efficien
ency by bounding its
running time from above. Thu hus it guarantees that for any instance of size n,n the running time
will not exceed Cworst (n), itss running
ru time on the worst-case inputs.
Definition: The best-case efficiency
eff of an algorithm is its efficiency for
or the best-case input
of size n, for which the algorit
rithm runs the fastest among all possible inputs
uts of that size.
We determine the kind of inputs
inp for which the count C(n) will be thee ssmallest among all
possible inputs of size n. Forr example, for sequential search, best-case inp
nputs are lists of size
n with their first elements equ
qual to a search key; Cbest(n) = 1.
The analysis of the best-case
se efficiency is not nearly as important as tha
hat of the worst-case
efficiency. Also, neither the
he worst-case analysis nor its best-case coun unterpart yields the
necessary information about an
a algorithm's behavior on a "typical" or "random"
"ra input. This
information is provided by ave
verage-case efficiency.
Definition: the average-casese complexity of an algorithm is the amountt oof time used by the
algorithm, averaged over alll possible
p inputs.
Let us consider again sequenti
ntial search. The standard assumptions are that at (a) the probability
of a successful search is equ
qual top (0 p 1) and (b) the probability ty of the first match
th
occurring in the i position of the list is the same for every i. We can findd tthe average number
of key comparisons Cavg (n) as follows.
In the case of a successfull search,
s the probability of the first match ooccurring in the ith
position of the list is p/n for every
e i, and the number of comparisons mad ade by the algorithm
in such a situation is obviouiously i. In the case of an unsuccessful searc arch, the number of
robability of such a search being (1- p). Therefo
comparisons is n with the prob efore,
Investigation of the average-ccase efficiency is considerably more difficult

ult than investigation
of the worst-case and best-cacase efficiencies. But there are many importortant algorithms for
which the average case efficficiency is much better than the overly pess essimistic worst-case
efficiency would lead us to believe.
be Note that average-case efficiency can
annot be obtained by
taking the average of the wors
rst-case and the best-case efficiencies.

troduction
Summary of analysis framew

ework
Both time and space efficie
iciencies are measured as functions of the algororithm's input size.
Time efficiency is measuasured by counting the number of times the algorithm's basic
operation is executed. Space
Sp efficiency is measured by counting the th number of extra
memory units consumed by b the algorithm.
The efficiencies of somee algorithms
a may differ significantly for input
uts of the same size.
For such algorithms, wee need
n to distinguish between the worst-case,se, average-case, and
best-case efficiencies.
The framework's primary ry interest lies in the order of growth of thee algorithm's
a running
time (or extra memory uninits consumed) as its input size goes to infinity
ity.
2. Performance Analysis
lysis
2.1 Space complexity

Total amount of computer memory
m required by an algorithm to comple
plete its execution is
called as space complexity of that algorithm. The Space required by an algorithm
al is the sum
of following components
A fixed part that is independent
ind of the input and output. This inclu
cludes memory space
for codes, variables, constants
co and so on.
A variable part that depends
de on the input, output and recursion sta
stack. ( We call these
parameters as instance
ce characteristics)
Space requirement S(P) of an algorithm P, S(P) = c + Sp where c is a constant depends
on the fixed part, Sp is the inst
nstance characteristics
Example-1: Consider followin
wing algorithm abc()
Here fixed component depend aracteristics Sp=0

nds on the size of a, b and c. Also instance char
Example-2: Let us considerr the
th algorithm to find sum of array.
re the problem instances are characterized by n, the number of
For the algorithm given here
elements to be summed. Thee space
s needed by a[ ] depends on n. So the spa space complexity can
be written as; Ssum(n) (n+3)
(n+ n for a[ ], One each for n, i and s.

troduction
2.2 Time complexity

Usually, the execution time oro run-time of the program is refereed as its
it time complexity
denoted by tp (instance chara
aracteristics). This is the sum of the time ta
taken to execute all
instructions in the program.
Exact estimation runtime iss a complex task, as the number of instru truction executed is
dependent on the input data. Also
A different instructions will take differentt ttime to execute. So
for the estimation of the timee complexity
c we count only the number of program
pr steps.
A program step is loosely defined
de as syntactically or semantically meani
aning segment of the
program that has and execut
cution time that is independent of instance characteristics.
c For
example comment has zero steps;
ste assignment statement has one step and so on.
We can determine the stepss needed
n by a program to solve a particularr pproblem instance in
two ways.
oduce a new variable count to the program whhich is initialized to
In the first method we introd
atements to increment count by an appropriat
zero. We also introduce state iate amount into the
program. So when each timee original program executes, the count alsoo incremented
i by the
step count.
lgorithm sum( ). After the introduction of the
Example-1: Consider the algo he count the program
will be as follows.
timate that invocation of sum( ) executes tota

From the above we can estim otal number of 2n+3
steps.
The second method to determ rmine the step count of an algorithm is to bui
uild a table in which
we list the total number of step
teps contributed by each statement. An example
ple is shown below.

troduction
Example-2: matrix addition
The above thod is both exces essively difficult and, usually unnecessary. Th
The thing to do is to
identify the most important operation of the algorithm, called the bas asic operation, the
operation contributing the moost to the total running time, and compute ththe number of times
the basic operation is executed
ted.
Trade-off
There is often a time-space-tr
tradeoff involved in a problem, that is, it can
annot be solved with
few computing time and loww memory consumption. One has to make a compromise
c and to
exchange computing time forfo memory consumption or vice versa, dependingdep on which
algorithm one chooses and how one parameterizes it.
3. Asymptotic Notations
ions
The efficiency analysis frame
mework concentrates on the order of growth th of an algorithms
basic operation count as the principal
pr indicator of the algorithms efficienc
ncy. To compare and
rank such orders of growth, th, computer scientists use three notations: O(bigO oh), (big
omega), (big theta) and o(lit
(little oh)
3.1. Big-Oh notation

Definition: A function t(n) is said to be in O(g(n)), denoted t(n)O(g(n)),)), if t (n) is bounded
above by some constant mul ultiple of g(n) for all large n, i.e., if there eexist some positive
constant c and some nonnegati
ative integer n0 such that
t(n) c g(n) for all n n0.

troduction
Informally, O(g(n)) is the sett of

o all functions with a lower or same order of growth as g(n)
Examples:
ormally prove 100n + 5 O(n2)

As another example, let us for
100n + 5 100nn + n (for all n 5) = 101n 101n2. (c=101,, n0=5)
Note that the definition gives alues for constants c
es us a lot of freedom in choosing specific val
and n0.
Example: To prove n2 + n = O(n3)
Strategies for Big-O Someti

etimes the easiest way to prove that f(n) = O(g
(g(n)) is to take c to
be the sum of the positive
ve coefficients of f(n). We can usually ig ignore the negative
coefficients.

troduction
3.2. Omega notation

Definition: A function t(n) is said to be in (g(n)), denoted t(n) (g(n)) )), if t(n) is bounded
below by some positive cons nstant multiple of g(n) for all large n, i.e.,, iif there exist some
positive constant c and somee nonnegative
n integer n0 such that
t(n) c g(n) for all n n0.
Here is an example of the formrmal proof that n3 (n2):

n3 n2 for all n 0,
i.e., we can select c = 1 and n0 = 0.
Example:
Example: To prove n3 + 4n2 = (n2)
3.3. Theta notation

A function t(n) is said to bee in (g(n)), denoted t(n) (g(n)), if t (n (n) is bounded both
above and below by some pos ositive constant multiples of g(n) for all largee n,
n i.e., if there exist
some positive constants c1 and
nd c2 and some nonnegative integer n0 such thattha
c2 g(n)
g t(n) c1g(n) for all n n0 .

troduction
Example: n2 + 5n + 7 = (n2)
Strategies for and

Proving that a f(n) = (g(n
g(n)) often requires more thought.
Quite often, we have
hav to pick c < 1.
A good strategy is to pick a value of c which you think will w work, and determine
which value of n0 is
i needed.
Being able to do a little algebra helps.
We can sometim imes simplify by ignoring terms of f(n)) with the positive
coefficients.
The following theorem shows
sho us that proving f(n) = (g(n)) is nothingg new:
Theorem: f(n) = (g(n))
if and only if f(n) = O(g(n)) and f(n)) = (g(n)).
Thus, we just apply the previous
pre two strategies.

troduction

troduction
3.4. Little Oh The function f(n)

f = o(g(n)) [ i.e f of n is a little oh of g off n ] if and only if
lim 0

Example:
For comparing the order of growth

gro limit is used
If the case-1 holds good in the

he above limit, we represent it by little-oh.

troduction
3.5. Basic asymptotic Efficien

ficiency Classes
Class Name Comments

troduction
3.6. Mathematical Analysis

sis of Non-recursive & Recursive Algorithms
ithms
Analysis of Non-recursive Algorithms
Al
General Plan for Analyzing the
th Time Efficiency of Nonrecursive Algorithm
hms
1. Decide on a parameterter (or parameters) indicating an inputs size.
2. Identify the algorithms basic operation. (As a rule, it is located inn iinnermost loop.)
3. Check whether the number
num of times the basic operation is execute ted depends only on
the size of an input.. If
I it also depends on some additional proper erty, the worst-case,
average-case, and, if necessary, best-case efficiencies have to t be investigated
separately.
4. Set up a sum express essing the number of times the algorithms s basic operation is
executed.
5. Using standard formuulas and rules of sum manipulation, either er find a closedform
formula for the countt or,
o at the very least, establish its order of growowth.
Example-1: To find maximu
um element in the given array
Algorithm
Here comparison is the basic

ic operation.
Note that number of comparisrisions will be same for all arrays of size n. Therefore,
Th no need to
distinguish worst, best and ave
verage cases.
Total number of basic operatio
tions (comparison) are,
Example-2: To check whethe

ther all the elements in the given array are distinct
d
Algorithm

troduction
Here basic operation is compa parison. The maximum no. of comparisons hhappen in the worst
th array are distinct and algorithms return tru
case. (i.e. all the elements in the rue).
Total number of basic operatio
tions (comparison) in the worst case are,
Other than the worst case, the

he total comparisons are less than . ( Forr example if the first
two elements of the array are
re equal, only one comparison is computed).
). So
S in general C(n)
2
=O(n )
Example-3: To perform mat
atrix multiplication
Algorithm
Number of basic opera

erations
(multiplications) is
Total running time:

Suppose if we take into accoun
ount of addition; Algoritham also have same nu
number of additions
3
A(n) = n
Total running time:

troduction
Example-4: To count the bit

its in the binary representation
Algorithm
The basic operation is count=c

t=count + 1 repeats no. of times
es
Analysis of Recursive Algori
rithms
General plan for analyzing the time efficiency of recursive algorithms
1. Decide on a parameterter (or parameters) indicating an inputs size.
2. Identify the algorithm
ms basic operation.
3. Check whether the number
nu of times the basic operation is exec xecuted can vary on
different inputs of thee same size; if it can, the worst-case, average--case, and best-case
efficiencies must bee investigated
i separately. Set up a recurrence ce relation, with an
appropriate initial cond
ndition, for the number of times the basic operaeration is executed.
4. Solve the recurrence or,
or at least, ascertain the order of growth of its solution.
Example-1
Algorithm
Since the function F(n) is com

mputed according to the formula
ns M(n) needed to compute it must satisfy thee eequality

The number of multiplications

troduction
Such equations are called recu

currence Relations
Condition that makes the alg lgorithm stop if n = 0 return 1. Thus recur urrence relation and
initial condition for the algorit
rithms number of multiplications M(n) can be stated as
We can use backward substitu

itutions method to solve this
Example-2: Tower of Hanoi oi puzzle. In this puzzle, There are n disks of different sizes that
can slide onto any of three peg
egs. Initially, all the disks are on the first pegg iin order of size, the
largest on the bottom and thehe smallest on top. The goal is to move all the disks to the third
peg, using the second one ass ana auxiliary, if necessary. We can move only ly one disk at a time,
and it is forbidden to place a larger
la disk on top of a smaller one.
The problem has an elegant recursive
re solution, which is illustrated in Figur
ure.
To move n>1 disks from pegp 1 to peg 3 (with peg 2 as auxiliary),
o we first move recur
cursively n-1 disks from peg 1 to peg 2 (with pe
peg 3 as auxiliary),
o then move the large
rgest disk directly from peg 1 to peg 3, and,
o finally, move recurursively n-1 disks from peg 2 to peg 3 (using peg
p 1 as auxiliary).
If n = 1, we move the singl
ngle disk directly from the source peg to the des
destination peg.
Figure: Rec
ecursive solution to the Tower of Hanoi puzzle
zle
The number of moves M(n) depends
de only on n. The recurrence equationn is
i
We have the following recurre

rrence relation for the number of moves M(n):

troduction
We solve this recurrence by the

th same method of backward substitutions:
The pattern of the first three sums

su on the left suggests that the next one will
ill be
24 M(n 4) + 23 + 22 + 2 + 1, and generally, after i substitutions, we get
Since the initial condition iss specified

s for n = 1, which is achieved for i = n - 1, we get the
following formula for the solulution to recurrence,
Alternatively, by counting the

th number of nodes in the tree obtained byy recursive calls, we
can get the total number of calls
cal made by the Tower of Hanoi algorithm:
Figure: Tree of recursivee calls

c made by the recursive algorithm for thee T
Tower of Hanoi
puzzle.
Example-3
The recurrence relation can be written as

.
Also note that A(1) = 0.

troduction
The standard approach to solv fo n = 2k and then

olving such a recurrence is to solve it only for
take advantage of the theore rem called the smoothness rule which claim ims that under very
k
broad assumptions the orderr ofo growth observed for n = 2 gives a correcrect answer about the
order of growth for all valuess of
o n.

troduction
4. Important Problem
m Types
Ty
In this section, we are goin
oing to introduce the most important proble blem types: Sorting,
Searching, String processing,, Graph problems, Combinatorial problems.
4.1. Sorting
The sorting problem is to rear
earrange the items of a given list in non-decre
creasing order. As a
practical matter, we usuallyy need
n to sort lists of numbers, characters from
fro an alphabet or
character strings.
Although some algorithms are indeed better than others, there is no algori orithm that would be
the best solution in all situatio
tions. Some of the algorithms are simple but relrelatively slow, while
others are faster but more complex;
co some work better on randomly ord rdered inputs, while
others do better on almost-so sorted lists; some are suitable only for listss rresiding in the fast
memory, while others can bee adapted
a for sorting large files stored on a disk
isk; and so on.
Two properties of sorting alglgorithms deserve special mention. A sortingg algorithm is called
stable if it preserves the relat
lative order of any two equal elements in its input. The second
notable feature of a sorting algorithm
alg is the amount of extra memory thee algorithm requires.
An algorithm is said to be in-pplace if it does not require extra memory, exc
xcept, possibly, for a
few memory units.
4.2. Searching
The searching problem deals ls with finding a given value, called a searchh key,
k in a given set.
(or a multiset, which permits
its several elements to have the same value). There
T are plenty of
searching algorithms to choosose from. They range from the straightforward rd sequential search
to a spectacularly efficient but
bu limited binary search and algorithms bassed on representing
the underlying set in a differe
rent form more conducive to searching. The latter
la algorithms are
of particular importance for real-world
re applications because they are indisp
ispensable for storing
and retrieving information from
rom large databases.
4.3. String Processing
In recent decades, the rapid proliferation
pr of applications dealing with non--numerical data has
intensified the interest off researchers and computing practitionerss in string-handling
algorithms. A string is a sequence
s of characters from an alphabet. et. String-processing
algorithms have been impo portant for computer science in conjunctio tion with computer
languages and compiling issueues.
4.4. Graph Problems
One of the oldest and most interesting
int areas in algorithmics is graph algorit
rithms. Informally, a
graph can be thought of as a collection
co of points called vertices, some of which
w are connected
by line segments called edges.
ed Graphs can be used for modeling a wide variety of
applications, including transpo
sportation, communication, social and economi mic networks, project
scheduling, and games. Stud udying different technical and social aspects ts of the Internet in

troduction
particular is one of the acti

ctive areas of current research involving co
computer scientists,
economists, and social scientis
tists.
4.5. Combinatorial Problems
lems
Generally speaking, combinat
natorial problems are the most difficult proble
blems in computing,
from both a theoretical and practical
p standpoint. Their difficulty stemss ffrom the following
facts. First, the number off combinatorial
c objects typically grows extr tremely fast with a
problems size, reaching unimaginable
un magnitudes even for modera rate-sized instances.
Second, there are no known wn algorithms for solving most such proble blems exactly in an
acceptable amount of time.
5. Fundamental Data
a Structures
Stru
Since the vast majority off algorithms of interest operate on data, particular
p ways of
organizing data play a critical
al role in the design and analysis of algorithmss. A data structure
can be defined as a particularr scheme
s of organizing related data items.
5.1. Linear Data Structures
res
The two most important eleme
mentary data structures are the array and the linked
lin list.
A (one-dimensional) array is a sequence of n items of the same data ty
type that are stored
contiguously in computer mem
emory and made accessible by specifying a value
v of the arrays
index.
A linked list is a sequence of zero or more elements called nodes, each containing
co two kinds
of information: some data and nd one or more links called pointers to otherr nnodes of the linked
list. In a singly linked list, each
eac node except the last one contains a single le pointer to the next
element. Another extension is i the structure called the doubly linked list,li in which every
node, except the first and thee last,
l contains pointers to both its successor and
an its predecessor.
A list is a finite sequence off data

d items, i.e., a collection of data items ar
arranged in a certain
linear order. The basic ope perations performed on this data structuree are searching for,

troduction
inserting, and deleting an element.

e Two special types of lists, stacks
cks and queues, are
particularly important.
A stack is a list in which insertions
ins and deletions can be done only at the
th end. This end is
called the top because a stack
ck is usually visualized not horizontally but ve
verticallyakin to a
stack of plates whose operatio
ations it mimics very closely.
A queue, on the other hand, d, is a list from which elements are deleted from
fr one end of the
structure, called the front (thi
this operation is called dequeue), and new elem
lements are added to
the other end, called the rear
rea (this operation is called enqueue). Cons nsequently, a queue
operates in a first-infirst-ou
out (FIFO) fashionakin to a queue of cust stomers served by a
single teller in a bank. Queu eues also have many important applications,ns, including several
algorithms for graph problemsms.
Many important applications ns require selection of an item of the highest est priority among a
dynamically changing set off candidates. A data structure that seeks to sa satisfy the needs of
such applications is called a priority queue. A priority queue is a collec lection of data items
from a totally ordered univ niverse (most often, integer or real numbe bers). The principal
operations on a priority queue
ue are finding its largest element, deleting itss la
largest element, and
adding a new element.
5.2. Graphs
A graph is informally thoughght of as a collection of points in the plane ccalled vertices or
nodes, some of them connecnected by line segments called edges or arcs.
ar A graph G is
called undirected if every edge
ed in it is undirected. A graph whose every ry edge is directed is
called directed. Directed grap
aphs are also called digraphs.
The graph depicted in Figuree (a)
( has six vertices and seven undirected edge
ges:
V = {a, b, c, d, e, f },
} E = {(a, c), (a, d), (b, c), (b, f ), (c, e), (d, e)
e), (e, f )}.
The digraph depicted in Figure
ure 1.6b has six vertices and eight directed edge
ges:
V = {a, b, c, d, e, f },, E = {(a, c), (b, c), (b, f ), (c, e), (d, a), (d, e), (e
(e, c), (e, f )}.
Graph Representations - Graphs

Gr for computer algorithms are usually represented
rep in one of
two ways: the adjacency matr
trix and adjacency lists.
The adjacency matrix of a graph
g with n vertices is an n x n boolean matrix
m with one row
and one column for each of the the i row and the jth
th graphs vertices, in which the element in th th

troduction
i an edge from the ith vertex to the jth vertex

column is equal to 1 if theree is tex, and equal to 0 if
there is no such edge.
The adjacency lists of a graph
aph or a digraph is a collection of linked lists,, oone for each vertex,
that contain all the vertices adjacent
ad to the lists vertex (i.e., all the vertices
ces connected to it by
an edge).
Weighted Graphs: A weigh ghted graph (or weighted digraph) is a graphph (or digraph) with
numbers assigned to its edges.
es. These numbers are called weights or costs.
Among the many propertiess of o graphs, two are important for a great numb
mber of applications:
connectivity and acyclicity. Both
B are based on the notion of a path. A path
pat from vertex u to
vertex v of a graph G can be b defined as a sequence of adjacent (conn nnected by an edge)
vertices that starts with u andd ends
e with v.
A graph is said to be connecte

cted if for every pair of its vertices u and v there
the is a path from u
to v. Graphs with several cononnected components do happen in real-world rld applications. It is
important to know for many ny applications whether or not a graph under er consideration has
cycles. A cycle is a path of a positive length that starts and ends at the sam
same vertex and does
not traverse the same edge moore than once.
5.3. Trees
A tree (more accurately, a free
fre tree) is a connected acyclic graph. A graphph that has no cycles
but is not necessarily connecte
cted is called a forest: each of its connected co
components is a tree.
Trees have several importantt properties
p other graphs do not have. In particu
ticular, the number of
edges in a tree is always onee less
le than the number of its vertices: |E| = |V| - 1

troduction
Rooted Trees: Another veryy important

i property of trees is the fact that for
or every two vertices
in a tree, there always existss exactly
e one simple path from one of these vertices
ve to the other.
This property makes it possib
sible to select an arbitrary vertex in a free treeee and consider it as
the root of the so-called roote
oted tree. A rooted tree is usually depicted byy placing its root on
the top (level 0 of the tree),, the
t vertices adjacent to the root below it (le (level 1), the vertices
two edges apart from the root still below (level 2), and so on.
The depth of a vertex v is the length of the simple path from the root to v. The height of a
tree is the length of the longes
est simple path from the root to a leaf.
Ordered Trees- An orderedd treet is a rooted tree in which all the childrenen of each vertex are
ordered. It is convenient to assume
as that in a trees diagram, all the children
ren are ordered left to
right. A binary tree can be defined
de as an ordered tree in which every vertertex has no more than
two children and each child is designated as either a left child or a rightt child
ch of its parent; a
binary tree may also be emptyty.
If a number assigned to eachh parental vertex is larger than all the number
ers in its left subtree
and smaller than all the nummbers in its right subtree. Such trees are cal
alled binary search
trees. Binary trees and binary
ry search
trees have a wide varie riety of
applications in computer scien
ience.

troduction
5.4. Sets and Dictionaries

A set can be described as an unordered collection (possibly empty) of distinct di items called
elements of the set. A specificific set is defined either by an explicit listing of its elements (e.g.,
S = {2, 3, 5, 7}) or by specif cifying a property that all the sets elementss anda only they must
satisfy (e.g., S = {n: n is a prim
rime number smaller than 10}).
The most important set opera rations are: checking membership of a givenn iitem in a given set;
finding the union of two sets,
s, which comprises all the elements in either or both of them; and
finding the intersection of twoo sets, which comprises all the common eleme
ments in the sets.
Sets can be implemented inn computer
c applications in two ways. The first considers only sets
that are subsets of some large
ge set U, called the universal set. If set U has n elements, then any
subset S of U can be represen ctor, in which the ith
ented by a bit string of size n, called a bit vect
element is 1 if and only if thee ith element of U is included in set S.
The second and more common on way to represent a set for computing purpos oses is to use the list
structure to indicate the sets
s elements. This is feasible only for finite sets
ets. The requirement
for uniqueness is sometimes es circumvented by the introduction of a m multiset, or bag, an
unordered collection of itemss that are not necessarily distinct. Note that if a set is represented
by a list, depending on the application
ap at hand, it might be worth maint intaining the list in a
sorted order.
Dictionary: In computing, theth operations we need to perform for a set et or a multiset most
often are searching for a given
giv item, adding a new item, and deleting ng an item from the
collection. A data structure that
th implements these three operations is call alled the dictionary.
An efficient implementation on of a dictionary has to strike a compro romise between the
efficiency of searching and the
th efficiencies of the other two operations.. They
T range from an
unsophisticated use of arrays
ys (sorted or not) to much more sophisticatedd techniques such as
hashing and balanced search trees.
tr
A number of applications in computing require a dynamic partition off ssome n-element set
into a collection of disjoint subsets.
su After being initialized as a collection
ion of n one-element
subsets, the collection is subje
bjected to a sequence of intermixed union and nd search operations.
This problem is called the set
et union problem.
*****

DAA Notes Module 1

Uploaded by

DAA Notes Module 1

Uploaded by

Vivekananda

Module--1 : Introduction to Algorithms

1. Introduction 4. Important Proroblem Types

Prerpared by Harivinod N www.techjourney.in Page|1.2

Prerpared by Harivinod N www.techjourney.in Page|1.3

1.2. Algorithm Specification

In C++ the same algorithm can be specified as follows

Here Type is a basic or user defined

Example 3: Tower of Hanoii problem

Prerpared by Harivinod N www.techjourney.in Page|1.4

1.3. Analysis Framework

Measuring an Inputs Size

Units for Measuring Runnin

Prerpared by Harivinod N www.techjourney.in Page|1.5

Let cop be the execution timee of an algorithm's basic operation on a partic

Algorithms that require an exponential

Worst-Case, Best-Case, and

Prerpared by Harivinod N www.techjourney.in Page|1.6

The running time of above algorithm

Investigation of the average-ccase efficiency is considerably more difficult

Prerpared by Harivinod N www.techjourney.in Page|1.7

Summary of analysis framew

2.1 Space complexity

Here fixed component depend aracteristics Sp=0

Prerpared by Harivinod N www.techjourney.in Page|1.8

2.2 Time complexity

timate that invocation of sum( ) executes tota

Prerpared by Harivinod N www.techjourney.in Page|1.9

Example-2: matrix addition

3.1. Big-Oh notation

Prerpared by Harivinod N www.techjourney.in Page|1.10

Informally, O(g(n)) is the sett of

ormally prove 100n + 5 O(n2)

Example: To prove n2 + n = O(n3)

Strategies for Big-O Someti

Prerpared by Harivinod N www.techjourney.in Page|1.11

3.2. Omega notation

Here is an example of the formrmal proof that n3 (n2):

Example: To prove n3 + 4n2 = (n2)

3.3. Theta notation

Prerpared by Harivinod N www.techjourney.in Page|1.12

Strategies for and

Prerpared by Harivinod N www.techjourney.in Page|1.13

Prerpared by Harivinod N www.techjourney.in Page|1.14

3.4. Little Oh The function f(n)

For comparing the order of growth

If the case-1 holds good in the

Prerpared by Harivinod N www.techjourney.in Page|1.15

3.5. Basic asymptotic Efficien

Prerpared by Harivinod N www.techjourney.in Page|1.16

3.6. Mathematical Analysis

Here comparison is the basic

Example-2: To check whethe

Prerpared by Harivinod N www.techjourney.in Page|1.17

Other than the worst case, the

Number of basic opera

Total running time:

Prerpared by Harivinod N www.techjourney.in Page|1.18

Example-4: To count the bit

The basic operation is count=c

Since the function F(n) is com

ns M(n) needed to compute it must satisfy thee eequality

Prerpared by Harivinod N www.techjourney.in Page|1.19

Such equations are called recu

We can use backward substitu

We have the following recurre

Prerpared by Harivinod N www.techjourney.in Page|1.20

We solve this recurrence by the

The pattern of the first three sums

Since the initial condition iss specified

Alternatively, by counting the

Figure: Tree of recursivee calls

The recurrence relation can be written as

Prerpared by Harivinod N www.techjourney.in Page|1.21

The standard approach to solv fo n = 2k and then

Prerpared by Harivinod N www.techjourney.in Page|1.22