Huffman Coding

Introduction to Huffman Coding

• Huffman coding is a famous example of Greedy algorithm.

• Huffman coding is developed by David A. Huffman in 1951.
• Huffman coding is a lossless data compression algorithm and it uses
variable length encoding.
• In variable length encoding, lengths of the assigned codes are based on the
frequencies of corresponding characters.
• The most frequent character gets the smallest code and the least frequent
character gets the largest code.
2
Huffman Coding

• In Huffman coding, the variable-length codes assigned to input characters

are Prefix Codes, means the codes(bit sequences) are assigned in such a
way that the code assigned to one character is not the prefix of code
assigned to any other character.
• This is how Huffman coding makes sure that there is no ambiguity when
decoding the generated bitstream.

3
Huffman Coding

• Let us understand prefix codes with a counter example. Let there be four
characters a, b, c and d, and their corresponding variable length codes be
00, 01, 0 and 1. This coding leads to ambiguity because code assigned to c
is the prefix of codes assigned to a and b. If the compressed bit stream is
0001, the de-compressed output may be “cccd” or “ccb” or “acd” or “ab”.
• On the other hand 00, 01, 10, 11 are the prefix code that can be assigned to
a, b, c and d respectively.

4
Major Steps in Huffman Coding

There are two major steps in Huffman Coding-

1. Building a Huffman Tree using only the unique characters in the data
stream provided.
2. Assigning code to the characters by traversing the Huffman Tree.

Sample Footer Text 5

Numerical problem of Huffman coding
• Let’s us understand with an example:
• The following table shows each character with their frequencies:
Character Frequency
a 3
b 7
c 6
d 5
e 9

• Step 1. Build a min heap that contains 5 nodes where each node represents
root of a tree with single node.
6
Huffman Coding Example

• Extract two minimum frequency nodes from min heap. Add a new internal
node with frequency 3 + 5 = 8.

7
Huffman Coding Example

• Now min heap contains 4 nodes where 3 nodes are roots of trees with
single element each, and one heap node is root of tree with 3 elements
Character Frequency

c 6

b 7

Internal Node 8

e 9

8
Huffman Coding Example

• Extract two minimum frequency nodes from min heap. Add a new internal
node with frequency 6 + 7 = 13.

9
Huffman Coding Example

• Now min heap contains 3 nodes where 1 nodes are roots of trees with
single elements, and two heap node is root of tree with 3 elements
Character Frequency

Internal Node 8

e 9

Internal Node 13

10
Huffman Coding Example

• Extract two minimum frequency nodes from min heap. Add a new internal
node with frequency 8 + 9 = 17.

11
Huffman Coding Example

• Now min heap contains 2 nodes only.

Character Frequency

Internal Node 13

Internal Node 17

12
Huffman Coding Example

• Extract two minimum frequency nodes from min heap. Add a new internal
node with frequency 13 + 17 = 30.

13
Huffman Coding Example

• Now only one node is remaining.

Character Frequency

Internal Node 30

• Since only one node is remaining in min heap, so algorithm will stop here.

14
Steps to print codes from Huffman Tree

• Traverse the tree formed starting from the root.

• Maintain an auxiliary array.
• While moving to the left child, write 0 to the array.
• While moving to the right child, write 1 to the array.
• Print the array when a leaf node is encountered.

Sample Footer Text 15

Steps to print codes from Huffman Tree

16
Steps to print codes from Huffman Tree
Character Frequency Code
e 9 11
b 7 01
c 6 00
d 5 101
a 3 100

• As you can see that the most frequent character gets the least number of
bits.

17
Algorithm of Huffman Coding

• The method which is used to construct optimal prefix code is called

Huffman coding.
• This algorithm builds a tree in bottom up manner. We can denote this tree
by T
• Let, |c| be number of leaves
• |c| -1 are number of operations required to merge the nodes. Q be the
priority queue which can be used while constructing binary heap.

Sample Footer Text 18

Algorithm of Huffman Coding

Algo:
• n = |c|
• Q=c
• for i = 1 to n-1
• Allocate a new node z
• z.left = x = Extract-min(Q)
• z.right = y Extract-min(Q)
• z.freq = x.freq + y.freq
• Insert(Q, z)
• return Extract-min(Q) //return the root node of the tree
19
Time Complexity
• Building Huffman Tree: The time complexity of building the Huffman tree depends on
method used to construct it. Using a priority queue to merge nodes has a time
complexity of the O(N log N) where N is the number of the unique characters in the
input.
• Encoding: Once the Huffman tree is constructed encoding a message involves traversing
the tree to find the code for each character. This traversal has a time complexity of O(N)
where n is the length of input message.
• Decoding: Decoding a Huffman-encoded message also requires traversing the Huffman
tree. This traversal has a time complexity of O(N) where N is the number of the bits in
the encoded message.
Sample Footer Text 20
Application of Huffman Coding
• Image compression: By compressing image data, Huffman coding can reduce
the amount of storage space required for digital images.
• Audio compression: Similar to image compression, Huffman coding can also be
used to compress audio data. This can save storage space and bandwidth when
streaming or downloading audio files.
• Text compression: Text files can also be compressed using Huffman coding.
This can be useful for reducing the size of text documents or email attachments.
• Data transmission: When data is transmitted over a network, it is often
compressed using Huffman coding to reduce the amount of bandwidth required.
This can help to speed up data transfer and reduce costs associated with
transmitting data. 21
Thank
you!
Sample Footer Text 22