EC 370: Digital Image Processing

Module 6 (Part 1)
Image Compression
Ms. Sherry Ann Sacharias
Assistant Professor
Dept. of ECE
RSET
Reference Textbooks

• Digital Image Processing by Rafael C. Gonzalez and Richard E. Woods

(Chapter 8)

• Digital Image Processing by S. Jayaraman, S. Esakkirajan and T.

Veerakumar (Chapter 9)

Rajagiri School of Engineering & Technology 2

 Image Compression
• Image Compression is the method of reducing the amount of data needed to
represent digital images.
• The objective is to represent an image with minimum number of bits of an
acceptable image quality.

Need for Image Compression:

1. High quality image data requires large amounts of storage space
2. Reduction of transmission bandwidth
3. Increase in processing speed

Rajagiri School of Engineering & Technology 3

Redundancy in Images
• Images, represented as 2D array of intensity values are highly
coherent in nature.
• Representations that contain irrelevant information or repeated
information leads to data redundancy
• Image compression is achieved by removing this redundancy

Rajagiri School of Engineering & Technology 4

Classification of Redundancy

Rajagiri School of Engineering & Technology 5

Coding Redundancy
• Each pixel of a digital image is represented in the form of a code. The
number of bits used to represent an intensity value may be more than
that actually required.
• Example:
256 x 256 x 8 bit image with
coding redundancy

Rajagiri School of Engineering & Technology 6

• Compression ratio is defined as

• Data redundancy is related to compression ratio as

• In any coding scheme, the average no. of bits used to represent each
pixel is given by
Lavg = σ𝐿−1
𝑘=0 𝑙 𝑟𝑘 𝑝(𝑟𝑘 )

• For an image of size M x N, the number of bits used to represent the

image is 𝑀𝑁 × 𝐿𝑎𝑣𝑔

Rajagiri School of Engineering & Technology 7

Example

• Using code 1, 𝐿𝑎𝑣𝑔 = 8 • Using code 2,

• The number of bits used to
represent the image is given by
• The number of bits used to represent
256 × 256 × 8 = 5 , 24, 288 the image is given by

256 × 256 × 1.81 = 118, 621

Rajagiri School of Engineering & Technology 8
Thus the compression ratio and relative data redundancy achieved by
code 2 is given by

b 256 × 256 × 8
𝐶= = = 4.42
b′ 256 × 256 × 1.81

1 1
𝑅 =1− =1− = 0.774
𝐶 4.42

Rajagiri School of Engineering & Technology 9

Spatial & Temporal Redundancy
• As the pixels of most 2D intensity arrays are correlated spatially,
information is duplicated
• Example:
• 256 x 256 x 8 bit image with
spatial redundancy

In a video sequence, temporally correlated

pixels also duplicate information

Rajagiri School of Engineering & Technology 10

Psychovisual Redundancy
• Most 2D intensity arrays contain information that is ignored by the
human visual system
• Example:
• 256 x 256 x 8 bit image with
irrelevant information

Rajagiri School of Engineering & Technology 11

Classification
Image compression schemes can be classified into two types:
1. Lossless Compression:
The reconstructed image is identical to original image.
It has a modest compression rate
E.g.: Medical image compression
2. Lossy Compression:
The reconstructed image has degradations with respect to original image.
It has a higher compression rate
E.g.: Multimedia applications

Rajagiri School of Engineering & Technology 12

Fundamentals of Information Theory
• Given a source of statistically independent random events from a
discrete set of possible events 𝑎1 , 𝑎2 , … 𝑎𝑘 with probabilities
𝑃(𝑎1 ), 𝑃(𝑎2 ), … 𝑃(𝑎𝑘 ),
• The entropy or average information per source output is defined as
𝑘
1
𝐻 = ෍ 𝑃 𝑎𝑖 log
(𝑃 𝑎𝑖 )
𝑖=1
𝐻= − σ𝑘𝑖=1 𝑃 𝑎𝑖 log( 𝑃 𝑎𝑖 )

Rajagiri School of Engineering & Technology 13

Image Compression Scheme

Rajagiri School of Engineering & Technology 14

Loss Less Compression Techniques

• Huffman Coding: It assigns a binary code for each intensity level and
uses shorter codes for intensities with higher probability.

• Arithmetic Coding: It assigns a single arithmetic code to a sequence

of source symbols (message).

Rajagiri School of Engineering & Technology 15

Huffman Coding
Procedure:
1. The source symbols are arranged in the order of decreasing
probabilities.
2. The two symbols of lowest probability are assigned 0 and 1.
3. These two symbols are combined into a new symbol with probability
equal to the sum of the two original probabilities. The probability of the
new symbol is placed in the list in the order of decreasing probabilities.
4. The procedure is repeated until we are left with a final list of symbols of
only two for which a 0 and 1 are assigned.
5. The code for each source symbol is found by working backward and
tracing the sequence of 0s and 1s assigned to that symbol
Rajagiri School of Engineering & Technology 16
Qn. Given a source with symbols 𝒔𝟏 , 𝒔𝟐 , 𝒔𝟑 , 𝒔𝟒 , 𝒔𝟓 with probabilities 0.4, 0.2, 0.2, 0.1 and
0.1. Construct a binary code by applying Huffman encoding procedure. Find H(S),average
code length, code efficiency and variance of the code

S Code Length
𝒔𝟏 1 1
𝒔𝟐 01 2
𝒔𝟑 000 3
𝒔𝟒 0010 4
𝒔𝟓 0011 4

Rajagiri School of Engineering & Technology 17

• Average code length is given by Lavg = σ𝑞𝑖=1 𝑙 𝑠𝑖 𝑝(𝑠𝑖 )

• 𝐿𝑎𝑣𝑔 = (0.4 x 1) + (0.2 x 2) + (0.2 x 3) + (0.1 x 4) + (0.1 x 4) = 2.2 bits/symbol

S Code Length
𝒔𝟏 1 1
• The entropy of source is given by 𝒔𝟐 01 2
𝒒
𝟏 𝒔𝟑 000 3
H(S) = ෍ 𝒑(𝒔𝒊 ) 𝒍𝒐𝒈𝟐 𝒔𝟒 0010 4
𝒑(𝒔𝒊 )
𝒊=𝟏
𝒔𝟓 0011 4

• H s = −{ 0.4 log 2 0.4 + 0.2 log 2 0.2 + 0.2 log 2 0.2 + 0.1 log 2 0.1 +
0.1 log 2 0.1 }
= 2.12 bits/ symbol

Rajagiri School of Engineering & Technology 18

• The efficiency of encoding scheme is given by

𝜂 = 𝐻(𝑧)/𝐿𝑎𝑣𝑔
S Code Length
2.12
= = 0.963 𝒔𝟏 1 1
2.2
𝒔𝟐 01 2
𝒔𝟑 000 3
• Variance of the code is given by, 𝒔𝟒 0010 4
𝒔𝟓 0011 4
𝒒
𝜎 2 = σ𝒊=𝟏 𝒑𝒊 (𝒍𝒊 − 𝑳𝒂𝒗𝒈 )𝟐
2 2 2 2
= 0.4 1 − 2.2 + 0.2 2 − 2.2 + 0.2 3 − 2.2 + 2 × 0.1 4 − 2.2
= 1.36

Rajagiri School of Engineering & Technology 19

Arithmetic Coding
• A sequence of source symbols (message) is assigned a single
arithmetic code
• The code word defines an interval of real numbers between 0 and 1
• As the number of symbols in the message increases, the interval
used to represent it becomes smaller.
• Arithmetic coding performs well for sequences with low entropy
compared to Huffman Coding

Rajagiri School of Engineering & Technology 20

Procedure:

1. In arithmetic coding, the interval from 0 to 1 is divided into the

number of symbols according to the probabilities of occurrences of
the symbol.
2. Expand the first symbol to be encoded along with its range. Further
subdivide this range into the number of symbols.
3. Repeat this procedure until the termination character is encoded.

Rajagiri School of Engineering & Technology 21

Example:
Qn. A source emits four symbols {a,b,c,d} with probabilities 0.4, 0.2, 0.1 and 0.3
respectively. Construct arithmetic coding to encode and decode the word ‘dad’

Step 1: An interval of (0, 1) is divided into 4 based on the symbol probabilities as

follows:

The individual symbol range in the interval 0 to 1 is shown below:

Rajagiri School of Engineering & Technology 22

• Step 2: The first symbol to be transmitted is d. Hence the new range is
from 0.7 to 1.0 as shown below

• Find the sub range for each symbol in between the interval 0.7 and 1.0
• The new range for each symbol is

𝑙𝑜𝑤𝑒𝑟 𝑙𝑖𝑚𝑖𝑡 ∶ 𝑙𝑜𝑤𝑒𝑟 𝑙𝑖𝑚𝑖𝑡 + 𝑑 × 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑠𝑦𝑚𝑏𝑜𝑙

𝑤ℎ𝑒𝑟𝑒 𝑟𝑎𝑛𝑔𝑒, 𝑑 = 𝑢𝑝𝑝𝑒𝑟 𝑏𝑜𝑢𝑛𝑑 − 𝑙𝑜𝑤𝑒𝑟 𝑏𝑜𝑢𝑛𝑑

Rajagiri School of Engineering & Technology 23

• Range d in interval 0.7 to 1.0 = 0.3
• The new range for symbol a in interval 0.7 to 1.0 is computed as
0.7 ∶ 0.7 + 0.3 × 0.4
0.7 ∶ 0.82

• Similarly the new range for symbol b is computed as

0.82 ∶ 0.82 + 0.3 × 0.2
0.82 ∶ 0.88
Rajagiri School of Engineering & Technology 24
• The new range for symbol c is computed as
0.88 ∶ 0.88 + 0.3 × 0.1
0.88 ∶ 0.91

Rajagiri School of Engineering & Technology 25

• Step 3: The next symbol to be transmitted is a. Hence the new
interval corresponds to symbol a i.e. 0.7 to 0.82

• Now find the sub range for each symbol in the range 0.7 to 0.82

Rajagiri School of Engineering & Technology 26

• Range d in interval 0.7 to 0.82 = 0.12
• The new range for symbol a in interval 0.7 to 0.82 is computed as
0.7 ∶ 0.7 + 0.12 × 0.4
0.7 ∶ 0.748

Rajagiri School of Engineering & Technology 27

• Similarly the new range for symbol b is computed as
0.748 ∶ 0.748 + 0.12 × 0.2
0.748 ∶ 0.772

Rajagiri School of Engineering & Technology 28

• The new range for symbol c is computed as
0.772 ∶ 0.772 + 0.12 × 0.1
0.772 ∶ 0.784

Rajagiri School of Engineering & Technology 29

• Step 4: The next symbol to be transmitted is d. Hence the new
interval corresponds to symbol d i.e. 0.784 to 0.82

Rajagiri School of Engineering & Technology 30

• Step 5: After considering all the symbols to be encoded i.e. ‘d’, ‘a’ and
‘d’ , the tag value is computed in the final interval. Tag is the average
of the lower and upper limits of final interval.

• Here the tag value corresponding to the final interval 0.784 to 0.82 is
calculated as:

0.784 + 0.82
𝑡𝑎𝑔 = = 0.802
2

Hence the tag value 0.802 is the arithmetic code corresponding to the
message ‘dad’
Rajagiri School of Engineering & Technology 31
Decoding Procedure
The tag value and symbol probabilities will be sent to the receiver. The
data is decoded from the tag value
Example: For the above problem, decode the transmitted data given
the tag value 0.802 and symbol probabilities as follows:

Rajagiri School of Engineering & Technology 32

• Step 1: The initial interval is assumed to be between 0 and 1.

• The tag value is compared with the symbol sub range. The tag value 0.802
lies in the sub interval 0.7 and 1.0 which shows that d is the first decoded
symbol.
• Step 2: Since the first decoded symbol is d, the new interval is fixed as 0.7
to 1.0

Rajagiri School of Engineering & Technology 33

• Step 3: The new tag value is computed as follows
𝑡𝑎𝑔−𝑙𝑜𝑤𝑒𝑟 𝑙𝑖𝑚𝑖𝑡
𝑡∗ =
𝑟𝑎𝑛𝑔𝑒
0.802−0.7
Here 𝑡∗ = = 0.34
0.3

This tag value lies in the range 0 to 0.4, hence the next decoded symbol is a
• Step 4: Since the decoded symbol is a, the new interval is 0 to 0.4

Rajagiri School of Engineering & Technology 34

• Step 5: The new tag value for the interval 0 to 0.4 is calculated as

0.34 − 0
𝑡∗ = = 0.85
0.4
This tag value lies between 0.7 and 1.0 and hence decoded symbol is d

From the above steps, it is clear that that the decoded word is dad
Drawback: An end of message indicator needed to separate one message
from another
Rajagiri School of Engineering & Technology 35
Summary
• Image compression involves methods to reduce amount of data
required to represent data
• The need for compression and types of redundancies found in images
was discussed
• Error free compression techniques such as Huffman coding and
Arithmetic coding

Rajagiri School of Engineering & Technology 36