Unit 5 Greedy Algorithm

Greedy algorithm
• In an algorithmic strategy like greedy, the decision of solution is taken based
on the information available. The greedy method is a straightforward
method. This method is popular for obtaining the optimized solutions.
• In greedy techniques, the solution is constructed through a sequence of
steps, each expanding a partially constructed solution obtained so far, until a
complete solution to the problem is reached. At each step the choice made
should be,
• Feasible- it should satisfy the problem’s constraints.
• Locally optimal- among all feasible solutions the best choice is to be
made.
• Irrevocable- once the particular choice is made then it should not get
changed on subsequent steps.
Comparison between Greedy and Dynamic
Problem
Problem solving using greedy algorithm
• Making change problem
• Knapsack problem
• Prim’s algorithm
• Krushkal’s algorithm
• Activity selection problem
Problem Definition
🞂
Make Change – Greedy Solution
🞂

Total required coins =

Selected coins Amount
5
are Selected coins = {10,
coin quanti 20
s ty 5, 2, 1}
0
5
7
8
10 2
5 1
2 1
1 1
Analysis ???????????
Minimum spanning tree
• Let G = <N,A> be a connected, undirected graph.
• N is the set of nodes
• A is the set of edges.
• Each edge has a given positive length or weight.
• Spanning tree: A spanning tree of a graph G is a subgraph which is
basically a tree and it contains all the vertices of G containing no circuit.
• Minimum spanning tree: A minimum spanning tree of weighted connected
graph G is a spanning tree with minimum or smallest weight.
• Weight of the tree: A weight of the tree is defined as the sum of weights of
all its edges.
Properties of spanning tree
[Link] may be several minimum spanning trees of the same weight having
the minimum number of edges.
[Link] all the edge weights of a given graph are the same, then every spanning
tree of that graph is minimum.
[Link] each edge has a distinct weight, then there will be only one, unique
minimum spanning tree.
4.A connected graph G can have more than one spanning trees.
5.A disconnected graph can't have to span the tree, or it can't span all the
vertices.
[Link] Tree doesn't contain cycles.
[Link] Tree has (n-1) edges where n is the number of vertices.
Minimum spanning tree
• For example:
Consider a graph G as given below. This graph is called weighted
connected graph because some weights are given along every edge and the
graph is a connected graph.
• Graph and two spanning tree out of which (b) is a minimum spanning tree
Applications of Spanning tree
1. Spanning trees are very important in designing efficient routing
algorithms.
2. Spanning trees have wide applications in many areas such as network
design.
• For example,
Algorithm to find minimum spanning tree
• There two ways to find minimum spanning tree.
• Prim’s algorithm
• Krushkal’s algorithm
Prim’s algorithm
• In prim’s algorithm, the minimum spanning tree grows in natural way.
• Now we will consider all the vertices first.
• Then we will select an edge with minimum weight.
• The algorithm proceeds by selecting adjacent edges with minimum weight.
• Care should be taken for not forming circuit.
• Step-1: select pair with minimum weight
• Step-2: select the adjacent vertex and select the minimum weighted edge
using this adjacent vertex. The selected adjacent vertex should not form the
circuit.
• Step-3: repeat step 1 and step 2 until all the vertices are getting covered.
 Cost =
17

Step Edge Set B Edges Considered

Selected
Prim’s Algorithm

{u, v}
Init. - {1} --
1 {1, 2} {1,2} {1,2} {1,4}
2 {2, 3} {1,2,3} {1,4} {2,3} {2,4} {2,5}
3 {1, 4} {1,2,3,4} {1,4} {2,4} {2,5} {3,5} {3,6}
4 {4, 5} {1,2,3,4,5} {2,4} {2,5} {3,5} {3,6} {4,5} {4,7}
5 {4, 7} {1,2,3,4,5,7} {2,4} {2,5} {3,5} {3,6} {4,7} {5,6}
{5,7}
6 {6,7} {1,2,3,4,5,6,7} {2,4} {2,5} {3,5} {3,6} {5,6} {5,7}
{6,7}
Example Prim’s algorithm
Analysis Prim’s algorithm
• The algorithm spends most of the time in selecting the edge with minimum
length. Hence basic operation of this algorithm is to find the edge with
minimum path length.
Krushkal’s algorithm
• Krushkal’s algorithm is another algorithm of obtaining minimum spanning
tree. This algorithm is discovered by a second year graduate student Joseph
Krushkal.
• In this algorithm always the minimum cost edge has to be selected. But it is
not necessary that selected optimum edge is adjacent.
• Algorithm:
• Step-1: sort all the edges in non-decreasing order of their weight.
• Step-2: select the pair with minimum weight each time and make the union
of the selected edges. The selected vertex should not form the circuit.
• Step-3: repeat step 1 until all the vertices are getting covered.
Kruskal’s Algorithm
A Step 2: Taking Step 4: Taking
4 5
6 next next
6
min edge (B,C) min edge (A,B)
B 3
E B A
4
5 7 2
2
C D B E
C D 1 3
1
2
Step 3: Taking

Step 1: Taking
next C D
min edge (B,E)
min B E 1
edge (C,D) 3
So, we obtained a
C D 2
1 C D minimum
1 spanning tree of
cost:
4+2+1+3=
 Step:
Kruskal’s Algorithm 1Sort the edges in increasing order
of their weight.
Edges Weight

1 2 {1, 1
1 2 3 2}
{2, 2
4 6 3}
6 4 5
{4, 3
5}
4 3 5 8 6 {6, 3
7}
7 {1, 4
4 3 4}
{2, 4
5}
7
{4, 4
7}
{3, 5
 Step:
Kruskal’s Algorithm 2
Select the minimum weight edge
but no cycle.
Edges Weight

1 2 {1, 1 √
1 2 3 2}
√
{2, 2
3} √
4 6 4 5 6
{4, 3
√
5} √
4 3 5 8 6 {6, 3
7}
√
7 {1, 4
4 3 4}
{2, 4
5}
7
{4, 4
7}
{3, 5
 Step:
Kruskal’s Algorithm 3
The minimum spanning tree for the
given graph.
Edges Weight
1 2 √
1 2 3 {1,
2}
1
√
{2, 2 √
4 3} √
{4, 3
5} √
4 3 5 6 {6, 3 √
7}
Total
{1, Cost
4 =
4 3 17
4}
{4, 4
7 7}
Example krushkal’s algorithm
Activity Selection Problem
• Activity selection problem is defined as follows:
• Given a resource such as CPU or a lecture hall and n set of activities, such
as task or lectures that want to utilize the resource.
• The activity i have the starting and finishing time which can be denoted by
Si and Fi.
• Then activity selection problem is of find max_size subset A of mutually
compatible activities.

• Mutually compatible activities are 1,4,8,11

 Algorithm
• Step-1: Sort Fi i.e. finish time into non-deceasing order.
• Step-2: Select the next activity i to the solution set of i is compatible with
each activity in the solution set.
• Step-3: Repeat step-2, until all the activities get examined.
Example: i 1 2 3 4 5 6 7 8 9 10 11
Si 1 3 0 5 3 5 6 8 8 2 14
Fi 4 5 6 7 8 9 10 11 12 15 16

Output:
i 1 4 8 11 Selected activities are:
Si 1 5 8 14 a1,a4,a8,a11
Fi 4 7 11 16
Activity Selection algorithm
Graphs: Shortest Path
• Many times, Graph is used to represent the distances between the two cities
Everybody is often interested in moving from one city to other as quickly as
possible. The single source shortest path is based on this interest.
• In single source shortest path problem the shortest distance from a
single vertex called source is obtained. This shortest path algorithm is
called Dijkstra's shortest path algorithm.
• Let G(V, E) be a graph, then in single source shortest path the shortest paths
from vertex vo to all remaining vertex is determined. The vertex v o is then
called as source and the last vertex is called destination.
• It is assumed that all the distances are positive. This algorithm does not
work for negative distances.
Graphs: Shortest Path
• Consider graph G is given below.
The single source shortest
path from each vertex is
summarized below:

1,2 10
1,2,3 30
1,2,3,4 45
1,2,3,5 35
1,2,3,4,7 65
1,2,3,5,7 42
1,6 30
1,6,7 65
Analysis
• Time complexity of the algorithm:
• Single source shortest path takes O() time.
Job scheduling problem

• At any time t=1,2,3… only exactly one job is to be executed.

• Consider that there are n jobs that are to be executed.

• The profit pi are given.

• Example, n=4
n pi di
1 70 2
2 12 1
3 18 2
4 35 1
Job scheduling problem
• Each job takes one unit of time.
• If job starts before or at its deadline, profit is obtained,
otherwise no profit.
• Goal is to schedule jobs to maximize the total profit.
• Consider all possible schedules and compute the minimum
total time in the system.

0 j4 1 j1 2

Job 4 and 1 is scheduled and maximum profit is 105

Job scheduling problem
• Example, n=7
n pi di
1 3 1
2 5 3
3 18 3
4 20 4
5 6 1
6 1 2
7 38 1
 Huffman coding
• The Huffman's algorithm was developed by David. Huffman when he was a
PhD student at MIT.
• This algorithm is basically a coding technique for encoding data. Such an
encoded data is used in data compression techniques.
• In Huffman's coding method, the data is inputted as a sequence of
character Then a table of frequency of occurrence of each character in the
data is built
• From the table of Frequencies Huffman's tree is constructed.
• The Huffman's tree is further used for encoding each character, so that
binary encoding is obtained for given data.
• In Huffman's tree is further is a specific method of representing each
symbol. This method produces a code in such a manner that no code word
is Prefix of some other code word. Such codes are called Prefix codes or
Prefix Free codes. Thus this method is useful for obtaining optimal data
compression. Let us understand the technique of Huffman's coding with
the help of some example
 Huffman coding
• Example,

a b c d e f
Frequency 45 13 12 16 9 5
Fixed 7 bits ASCII 1000001 1000010 100001 1000100 100101 100110
1
7*45 7*13 7*12 7*16 7*9 7*5
Total size of 315 91 84 112 63 35
character
 Huffman coding
• For compression of file we can have two types of code.
• Fixed length code
• Variable length code
• Every character can have either a fixed length code or variable length code.

a b c d e f
Frequency 45 13 12 16 9 5
Fixed 3 bits 000 001 010 011 100 101
3*45 3*13 3*12 3*16 3*9 3*5
Total size of character 135 39 36 48 27 15
Total Size of file 300
 Huffman coding
• Variable length code

a b c d e f
Frequency 45 13 12 16 9 5
Fixed 3 bits 0 101 100 111 1101 1100
1*45 3*13 3*12 3*16 4*9 4*5
Total size of character 45 39 36 48 36 20
Total Size of file 224
 Prefix code problem
• Variable length code can reduce more size.
• But in variable length there is maximum chance of prefix code problem.
• Prefix code problem: for example, there are four characters a,b,c, and d,
and their corresponding variable length codes be 00,01,0 and 1.

• This coding leads to ambiguity because code assigned to c is prefix of code

assigned of a and b.
• If the compressed bit stream is: 0001
• The de-compressed output may be: ‘cccd’ or ‘ccb’ or ‘acd’ or ‘ab’.
 Steps to build Huffman coding
Input is array of unique characters along with their frequency of occurrences
and output is Huffman tree.
1. Write all the characters in ascending order based on their frequencies.
2. Extract two nodes with the minimum frequency.
3. Create a new internal node with frequency equal to the sum of the two
nodes frequencies. Make the first extracted node as its left child and other
extracted node as its right child. Add this node to the table as per
ascending order. The encoding should start from top to down. If we follow
the left branch then we should encode it as “0” and if we follow the right
branch then we should encode it as “1”.
4. Repeat step 2 and 3 until the table contains only one node. The remaining
node is the root node and the tree is complete.
 Huffman coding
• Example: obtain Huffman’s encoding for following data
a:39 b:10 c:9 d:25 e:7 f:3
Step:1 arrange data in ascending order based on their frequency.
Character Frequency
f 3
e 7
c 9
b 10
d 25
a 39
 Huffman coding
Step:2
10 19 64

f:3 e:7 c:9 b:10 d:25 a:39

Step-3: 29
0
1

10 19 64
0 0 1
1

f:3 e:7 c:9 b:10 d:25 a:39

 Huffman coding
Step:4 93
0 1

29 64
0
1 1

10 19 64
0 0 1 0 1
1

f:3 e:7 c:9 b:10 d:25 a:39

 Huffman coding
Total length required= Frequency x Number of bits used for representation
= (39*3) + (10*3) + (9*3) + (25*3) + (7*3) + (3*3)
= 117+30+27+75+21+9
= 279- bits
If we want to encode “aeed” then we get 111001001110
symbol Code word
a 111
b 011
c 010
d 110
e 001
f 000
Huffman tree with variable length
93
0 1

a:39 54
0 1

d:25 29
0 1

b:10 19
1
0

c:9 10
1
0

f:3 e:7
Huffman coding
Total length required= Frequency x Number of bits used for representation
= (39*1) + (25*2) + (10*3) + (9*4) + (3*5) + (7*5)
= 39+50+30+36+15+35
= 205- bits

symbol Code word

a 0
b 110
c 1110
D 10
e 11111
f 11110
 Huffman coding
• Example,
• Construct the Huffman tree for the following data and obtain its Huffman
code.
Character A B C D E -
probability 0.5 0.35 0.5 0.1 0.4 0.2

Encode text DAD-BE using above code.

Decode the text 1100110110 using above information.
Thank You