CODING

Shannon model for digital

communications
Analog
TRANSMITTER (TX) EMW

Digital Source (bits) Channel (bits)

Modulator
(bits) Encoder Claude Elwood
Encoder Shannon: An
American
mathematician,
electrical engineer,
Analog and cryptographer
Digital Source Channel
(bits) (bits)
Demodulator EMW known as a "father
Decoder of information
Decoder theory".
Published in 1948 in
Scope of this lecture Bell Labs's research
RECEIVER (RX)
journal in 1948.
2
“Coding”
“Coding” means different thing to different people:
◆ Channel Coding / Error control (adding redundancy) ◆ Source Coding (removing redundancy)
- Error detection coding – EDC - Encoding original format of a signal
- Forward Error Correction – FEC. - Using information theory to represent source
efficiently (e.g. speech, video) – Data compression
- Examples: MPEG, JPEG.

◆ Encryption and ‘Coded’ ◆ Line Coding

data
- Encryption used in most datalinks - Mapping of binary data to channel symbols
- Cryptography, Security - Includes Gray coding and symbol mapping in
- Commercial standards, proprietary or military modulation constellations.
- May yield Error Extension: one data bit error at decoder
input  many data bits in error at output
◆ Software ‘Coding’
- Common term for software writing.
6
WHAT IS SOURCE CODING?

• Source coding is the process by which data generated by a source is represented

efficiently
• The redundant data bits are reduced by applying the concepts of information
theory in source encoder.
• The device that performs source encoding is called a source encoder.
• Source coding removes the redundant bits and increases data rate.
• The source encoder compresses the data into minimum number of bits.
• This process helps in effective utilization of the bandwidth.
SOURCE CODING TECHNIQUE
• The codewords generated by the source encoder
For example, in telegraphy, we
should be in binary form.
use Morse code, in which the
• The concept of variable-length code for each source alphabets are denoted
symbol should be applied. by Marks and Spaces. If the
letter E is considered, which is
• If some source symbols are likely to occur more
mostly used, it is denoted
often than others, then short codewords can be by “.” Whereas the
assigned to them and if rarely then longer letter Q which is rarely used, is
codewords can be assigned to them. denoted by “--.-”
• The source code should be uniquely decodable so
that the original source sequence can be
reconstructed.
Encoding messages
 Encode a message composed of a string of characters
 Codes used by computer systems
 ASCII
• uses 8 bits per character
• can encode 256 characters
Each character will require similar 8 bit
 Unicode whether a character is used multiple
• 16 bits per character times or single time

• can encode 65536 characters

• includes all characters encoded by ASCII
 ASCII and Unicode are fixed-length codes
 all characters represented by same number of bits
Problems
 Suppose that we want to encode a message
constructed from the symbols A, B, C, D, and E
using a fixed-length code
 How many bits are required to encode each
symbol?
 at least 3 bits are required
 2 bits are not enough (can only encode four

symbols)
 How many bits are required to encode the

message DEAACAAAAABA?
 there are twelve symbols, each requires 3 bits
 12*3 = 36 bits are required
Drawbacks of fixed-length codes
 Wasted space
 Unicode uses twice as much space as ASCII
• inefficient for plain-text messages containing only ASCII characters
 Same number of bits used to represent all characters
 ‘a’ and ‘e’ occur more frequently than ‘q’ and ‘z’

 Potential solution: use variable-length codes

 variable number of bits to represent characters when frequency of
occurrence is known
 short codes for characters that occur frequently
Advantages of variable-length codes
 The advantage of variable-length codes over fixed-length is short codes
can be given to characters that occur frequently
 on average, the length of the encoded message is less than fixed-
length encoding
 Potential problem: how do we know where one character ends and
another begins? The problem occurs during decoding A=0
• not a problem if number of bits is fixed! B=1
C = 10
D = 11
A = 00 0010110111001111111111
B = 01
C = 10 ACDBADDDDD
D = 11
Prefix property
 A code has the prefix property if no character code is the prefix (start of
the code) for another character
 Example:

Symbol Code
P 000
01001101100010
Q 11
R 01 RSTQPT
S 001
T 10
 000 is not a prefix of 11, 01, 001, or 10
 11 is not a prefix of 000, 01, 001, or 10 …
Code without prefix property
 The following code does not have prefix property

Symbol Code
P 0
Q 1
R 01
S 10
T 11

 The pattern 1110 can be decoded as QQQP, QTP, QQS, or TS

Problem
 Design a variable-length prefix-free code such that the message
DEAACAAAAABA can be encoded using 22 bits
 Possible solution:
 A occurs eight times while B, C, D, and E each occur once
 represent A with a one bit code, say 0
• remaining codes cannot start with 0
 represent B with the two bit code 10
• remaining codes cannot start with 0 or 10
 represent C with 110
 represent D with 1110
 represent E with 11110
Encoded message

DEAACAAAAABA

Symbol Code
A 0
B 10
C 110
D 1110
E 11110

1110111100011000000100 22 bits
Another possible code

DEAACAAAAABA

Symbol Code
A 0
B 100
C 101
D 1101
E 1111

1101111100101000001000 22 bits
Better code

DEAACAAAAABA

Symbol Code
A 0
B 100
C 101
D 110
E 111

11011100101000001000 20 bits
What code to use?
 Question: Is there a variable-length code that makes the most efficient
use of space?

Answer: Yes!
Huffman coding tree
 Binary tree
 each leaf contains symbol (character)
 label edge from node to left child with 0
 label edge from node to right child with 1
 Code for any symbol obtained by following path from David Albert
root to the leaf containing symbol Huffman (August
9, 1925 –
 Code has prefix property October 7, 1999)
 leaf node cannot appear on path to another leaf was an American
pioneer in
 note: fixed-length codes are represented by a computer
complete Huffman tree and clearly have the prefix science, known
for his Huffman
property coding. He was
also one of the
pioneers in the
field of
mathematical
Building a Huffman tree
 Find frequencies of each symbol occurring in message
 Begin with a forest of single node trees
 each contain symbol and its frequency
 Do recursively
 select two trees with smallest frequency at the root
 produce a new binary tree with the selected trees as children and
store the sum of their frequencies in the root
 Recursion ends when there is one tree
 this is the Huffman coding tree
https://www.youtube.com/watch?v=LfWa6_xMy6Y&list=PLOuGMjEXHeeAv5Ejeo4xz3BHM2-0cN8nN&index=6

Step-1 Find the frequency of each symbol

Step 2: Arrange the sequence in Ascending order
Step 3: Form a binary tree starting from leaf (lowest first) to root
First pick the two lowest and merge. Here we will pick B and D and add them.
Then pick B&D and A and add them.
Finally it will be equal to the number of symbols.
Left handed will be given 0 and Right handed 1
Step 4: Start from the root and with it down against
each symbol
Step 5: with the symbol, then codeword, then frequency
Now multiply the code length with frequency and add them all together
Decoding
First it will check from the root
Then from root to leaf
Here. It will start with 1 and decode as C
Then it will again start from root and will get a 0 and then it will come to right side and will get 1 and it will be decoded as A
Then it again start from root and descend upto 3 zero and will decode as B
CHANNEL CODING
CHANNEL ENCODING
• Channel coding is widely used in digital communication systems to protect the digital
information from noise and interference and reduce the number of bit errors.
• During the transmission of the signal, due to the noise in the channel, the signal may get altered
and hence to avoid this, the channel encoder adds some redundant bits to the transmitted data.
• Channel coding enhances the capability of transmitted digital data to detect and correct errors
at the receiver.
• Channel coding is mostly accomplished by selectively introducing redundant bits into the
transmitted information stream.
• These additional bits will allow detection and correction of bit errors in the received data stream
and provide more reliable information transmission.
• The cost of using channel coding to protect the information is a reduction in data rate or an
expansion in bandwidth.
IMPORTANT PARAMETERS OF
CODES
Main parameters to be considered in a (satcom) system:

 Code length (related to both coding gain and latency)

 Code rate (related to bandwidth efficiency and coding gain)

 Error-correcting power / Hamming minimum distance (related to coding gain)

 Coding gain (related to code rate, code length, code type)

 Latency / delay (related to code length and interleaver and

complexity)
32
TYPES OF CHANNEL CODES

• There are two main types of channel codes

• Block codes: It works on the fixed size of code length.
• It accepts a block of k information bits and produce a block of n coded bits by
predetermined rules and referred as (n,k) block codes.
• Example: Hamming codes, Golay codes, BCH codes, and Reed Solomon codes
• Convolutional codes: Convolutional codes convert the entire data stream into
one single codeword.
• The encoded bits depend not only on the current k input bits but also on past
input bits.
TYPES OF ERRORS
LINE CODING
 LINE CODING
• Line coding (also called digital baseband modulation or digital baseband
transmission) is a process carried out by a transmitter that converts
data, in the form of binary digits, into a baseband digital signal that will
represent the data on a transmission line. The receiver is responsible for
converting the incoming line-coded signal back into binary data.
• Line coding is the process of converting digital data to digital signals. By
this technique we convert a sequence of bits to a digital signal. At the
sender side digital data are encoded into a digital signal and at the
receiver side the digital data are recreated by decoding the digital
signal.
• Line code manipulates voltage or current levels in order to generate
electrical pulses that represent data values. In optical fibre channels, it
represents the data values by modifying the intensity of pulses of light.
TYPES OF LINE CODING

Many different line coding techniques, ranging in complexity from very basic
unipolar schemes in which the presence or absence of a voltage is used to
represent a binary one or a binary zero, to highly sophisticated multilevel schemes
in which different signal amplitudes are used, each representing a unique grouping
of binary digits.
There are 3 types of Line Coding

• Unipolar Non-return to zero (NRZ)

• Polar Return-to-zero (RZ)
• Bi-polar
• Differential (Multi Transition)
• Multi-level
 NRZ
• Each data bit is represented by a level. A high level may represent a
logic 1, where as a low level may represent a logic 0.
• 1s are by usually represented a positive voltage, while 0s are
represented by some other significant condition, usually a negative
voltage.
• There are just two levels and no pauses between bauds.
• A problem arises when using NRZ to encode data for a synchronous
link that may have long runs of consecutive bits with the same value.
If the encoded data contains long 'runs' of logic 1's or 0's, this does
not result in any bit transitions.
 RZ

• Uses a positive or negative voltage to represent a binary digit and

the voltage must return to zero in the middle of the bit time.
• This has the advantage of providing the receiver with an
opportunity to re-synchronise itself with the transmitter by using
the additional transition as a clock signal.
• The down side is that it increases the complexity of both the line
coding scheme and the circuitry needed to generate and decode
it. It also effectively doubles the bandwidth required to transmit
the signal.
UNIPOLAR, POLAR, AND BIPOLAR

• Unipolar, Polar: all of the signal levels are present either

below or above the axis.
• Polar Coding: voltages are present on both sides of an axis.
• Bipolar Coding:: there are three voltage levels positive,
negative and zero.
FEATURES OF LINE CODING

• Minimal complexity - complex line coding schemes require sophisticated electronic

circuitry in the transmitter to encode the digital data, and in the receiver to decode. The
complexity should therefore be kept to a minimum in order to reduce the cost of hardware.
• Embedded clock signal - in most applications, it is impractical to provide a separate timing
circuit between a transmitter and a receiver to maintain synchronisation. Since timing is
absolutely critical for the reliable transmission of large amounts of data at high signalling
rates, the timing information must be embedded in the signal itself so that the receiver can
extract the clock signal from the incoming bit stream.
• Bandwidth limited - Limit the bandwidth of a baseband signal, to maximise the data
signalling rate that can be used on a channel with limited bandwidth. Higher frequencies
tend to generate more electromagnetic radiation, and are thus more likely to create
crosstalk and interference.
FEATURES OF LINE CODING

• Power efficiency - the transmitted signal power should be as low as possible for the required data
rate and probability of error in order to make the most efficient use of power and minimise the
amount of electromagnetic noise on the transmission line.
• Target spectral density - the power spectral density (PSD) is the distribution of power over the
frequencies that make up the signal. The PSD of the transmitted signal should be compatible with
the communication channel's frequency response. If the signal bandwidth (i.e. the frequency
interval that contains most of the signal's power) is greater than that of the channel, the higher
frequencies will be cut off, causing the signal to spread out in the time domain.
• Avoidance of baseline wandering - the average signal-power at the receiver is used as a baseline
reference against which the value of incoming data elements is determined. The baseline may
drift up or down over time if long strings of ones or zeros are transmitted, making it difficult for
the receiver to decode the incoming signal correctly. Most line coding schemes are designed to
prevent this from happening.
 UNIPOLAR: NON-RETURN TO ZERO NRZ

• All the signal levels are either above

or below the axis.
• High in data is represented by a
positive pulse called as Mark,
which has a duration T0 equal to the
symbol bit duration.
• Positive voltage defines bit 1 and the
zero voltage defines bit 0. Signal
does not return to zero at the middle
of the bit thus it is called NRZ.
• A Low in data input has no pulse.
 ADVANTAGES AND DISADVANTAGES OF NRZ

• Advantages
• It is simple.
• A lesser bandwidth is required.
• Disadvantages
• No error correction done.
• No clock is present.
• Loss of synchronization is likely to occur (especially for
long strings of 1s and 0s).
 UNIPOLAR: RETURN TO ZERO (RZ)

• In this type of unipolar signaling, a High in data, though

represented by a Mark pulse, its duration T0 is less than the
symbol bit duration. Half of the bit duration remains high but it
immediately returns to zero and shows the absence of pulse
during the remaining half of the bit duration.
 ADVANTAGES AND DISADVANTAGES OF NRZ

• Advantages
• It is simple.
• The spectral line present at the symbol rate can be used as
a clock.
• Disadvantages
• No error correction done.
• Occupies twice the bandwidth as unipolar NRZ.
• Larger bandwidth
 POLAR SIGNALING
• In polar schemes, the voltages are on the
both sides of the axis.
• There are two methods of Polar Signaling.
They are
• Polar NRZ
• Polar RZ
 POLAR: NRZ
• High in data is represented by a positive pulse,
while a Low in data is represented by a negative
pulse.
• NRZ-L and NRZ-I – These are somewhat similar to
unipolar NRZ scheme but here we use two levels
of amplitude (voltages).
• NRZ-L(NRZ-Level): the level of the voltage
determines the value of the bit, typically binary 1
maps to logic-level high, and binary 0 maps to
logic-level low.
• NRZ-I(NRZ-Invert): The idea is that whenever
we encounter logical 1 then the signal will be
inverted, but when it encounters logical 0 then it
remains on the same side. This is the NRZ-Invert..
 ADVANTAGES AND DISADVANTAGES OF POLAR
NRZ
• Advantages
• It is simple.
• Disadvantages
• No error correction.
• No clock present.
• The Baseline wandering.
 POLAR: RZ
• A High in data, though represented by
a Mark pulse, its duration T0 is less than the
symbol bit duration. Half of the bit duration
remains high but it immediately returns to
zero and shows the absence of pulse during
the remaining half of the bit duration.
• However, for a Low input, a negative pulse
represents the data, and the zero level
remains same for the other half of the bit
duration.
• Uses three values: positive, negative and
zero. In this scheme signal goes to 0 in the
middle of each bit.
 ADVANTAGES AND DISADVANTAGES OF POLAR
RZ
• Advantages
• Return to zero proved out to be a nice alternative or say a solution to
NRZ drawbacks.
• Disadvantages
• No error correction.
• No clock present.
• Complex encoding as it uses three levels of voltages
• Occupies twice the bandwidth of Polar NRZ
• Signal drop is caused at the places where the signal is non-zero at 0 Hz
 BIPOLAR CODING
• Bipolar consists of three voltage levels
namely +, - and 0. Such a signal is called
as duo-binary signal.
• An example of this type is Alternate Mark
Inversion (AMI). For a 1, the voltage level
gets a transition from + to – or from – to
+, having alternate 1s to be of equal
polarity. A 0 will have a zero voltage level.
• Two types.
• Bipolar NRZ
• Bipolar RZ
 BIPOLAR CODING
• Two types.
• Bipolar NRZ
• Bipolar RZ

• The pulse duration and symbol

bit duration are equal in NRZ
type, while the pulse duration is
half of the symbol bit duration in
RZ type.
 ADVANTAGES AND DISADVANTAGES OF
BIPOLAR SIGNALING
• Advantages
• It is simple.
• No low-frequency components are present..
• Disadvantages
• No error correction.
• No clock present.
• Occupies twice the bandwidth of Polar NRZ
• Signal droop is caused at the places where the signal is non-zero at 0 Hz
• The bipolar encoding proved to be an alternative for NRZ encoding because bipolar
has the same signal rate as NRZ, and also it was not having any low frequency or DC
component in it, as one bit is represented by zero volts and the other is represented
by alternating voltages.
DISADVANTAGES OF RZ CODING

• Main disadvantage of RZ encoding is that it requires greater

bandwidth.
• Another problem is the complexity as it uses three levels of
voltage.
• As a result of all these deficiencies, this scheme is not used
today.
• Instead, it has been replaced by the better-performing
Manchester and differential Manchester schemes.
CHANNEL CODING
CHANNEL ENCODING
• Channel coding is widely used in digital communication systems to protect the digital
information from noise and interference and reduce the number of bit errors.
• During the transmission of the signal, due to the noise in the channel, the signal may get altered
and hence to avoid this, the channel encoder adds some redundant bits to the transmitted data.
• Channel coding enhances the capability of transmitted digital data to detect and correct errors
at the receiver.
• Channel coding is mostly accomplished by selectively introducing redundant bits into the
transmitted information stream.
• These additional bits will allow detection and correction of bit errors in the received data stream
and provide more reliable information transmission.
• The cost of using channel coding to protect the information is a reduction in data rate or an
expansion in bandwidth.
Important Parameters of Codes
Main parameters to be considered in a (satcom) system:

 Code length (related to both coding gain and latency)

 Code rate (related to bandwidth efficiency and coding gain)

 Error-correcting power / Hamming minimum distance (related to coding gain)

 Coding gain (related to code rate, code length, code type)

 Latency / delay (related to code length and interleaver and

complexity)

 Decoder complexity (related to latency) 59

WHY CANNEL CODING

• In digital communications, the information is represented as a sequence of binary

bits. The binary bits are then mapped (modulated) to analog signal waveforms and
transmitted over a communication channel. The communication channel
introduces noise and interference to corrupt the transmitted signal. At the
receiver, the channel corrupted transmitted signal is mapped back to binary bits.
• The received binary information is an estimate of the transmitted binary
information. Bit errors may result due to the transmission and the number of bit
errors depends on the amount of noise and interference in the communication
channel.
TYPES OF ERRORS
ERROR DETECTION AND CORRECTION
CODES
• Error Detection Codes: The codes which are used for error detecting are
called as Error Detection Codes
Backward error
• Parity Checking
correction: Once the
• Cyclic Redundancy Check (CRC) error is discovered, the
• Longitudinal Redundancy Check (LRC) receiver requests the
sender to retransmit the
• Check Sum
entire data unit.
• Error Correction Codes: The codes which are used for both error Forward error correction:
detecting and error correction are called as Error Correction Codes In this case, the receiver
uses the error-correcting
• Backward Error Correction (also known as Automatic Repeat reQuest, ARQ) code which
• Forward Error Correction automatically corrects
the errors.
• Block codes
• Convolutional Codes
 PARITY CHECK
• Parity bit is an additional bit added to the data.
• Before adding the parity bit, number of 1 or 0 is calculated in the
data. Based on this calculation of data an extra bit is added to the
actual data.
• If we have a 7 bit data, then after adding a parity bit, it will be 8 bit.
• There are two types of parity bits in error detection, they are
• Even parity
• If the data has even number of 1’s, the parity bit is 0. Ex: data is
1000001 -> parity bit 0
• Odd number of 1’s, the parity bit is 1. Ex: data is 1001001 -> parity bit 1
• Odd parity
• If the data has odd number of 1’s, the parity bit is 0. Ex: data is
10011101 -> parity bit 0
• Even number of 1’s, the parity bit is 1. Ex: data is 10010101 -> parity bit
1
CYCLIC REDUNDANCY CHECK (CRC)
• This technique involves binary division of the
data bits being sent.
• A division operation is performed on the bits
being sent and calculates the remainder.
Before sending the actual bits, the sender
adds the remainder at the end of the actual
bits. Actual data bits plus the remainder is
called a codeword. The sender transmits
data bits as codewords.
• At the other end, the receiver performs division
operation on codewords using the same CRC divisor. If
the remainder contains all zeros the data bits are
accepted, otherwise it is considered as there some data
corruption occurred in transit.
LONGITUDINAL REDUNDANCY CHECK

• In longitudinal redundancy method, a block of

bits are arranged in a table format (in rows
and columns) and the parity bit for each
column is calculated separately. The set of
these parity bits are sent along with our
original data bits.
• Longitudinal redundancy check is a bit by bit
parity computation and the parity of each
column is calculated individually.
• This method can easily detect burst errors and
single bit errors. However, it fails to detect the
2 bit errors occurred in same vertical slice.
CHECKSUMS

• A checksum is created based on the data values in the data blocks using some
algorithm and appended to the data. When the receiver gets this data, a new
checksum is calculated and compared with the existing checksum. A non-match
indicates an error.
• For error detection by checksums, data is divided into fixed sized frames or segments.
• Sender’s End − The sender adds the segments using 1’s complement arithmetic to get
the sum. It then complements the sum to get the checksum and sends it along with
the data frames.
• Receiver’s End − The receiver adds the incoming segments along with the checksum
using 1’s complement arithmetic to get the sum and then complements it.
• If the result is zero, the received frames are accepted; otherwise they are discarded.
CHECKSUM EXAMPLE
• Suppose that the sender wants to send 4 frames each of 8
bits, where the frames are 11001100, 10101010, 11110000
and 11000011.
• The sender adds the bits using 1s complement arithmetic.
While adding two numbers using 1s complement arithmetic,
if there is a carry over, it is added to the sum.
• After adding all the 4 frames, the sender complements the
sum to get the checksum, 11010011, and sends it along with
the data frames.
• The receiver performs 1s complement arithmetic sum of all
the frames including the checksum. The result is
complemented and found to be 0. Hence, the receiver
assumes that no error has occurred.
CONSTRAINTS IN ERROR DETECTION

• Error detection needs retransmissions until correct data is

received.
• Delays will be variable, depending on how many
retransmissions are needed.
• As such, it is difficult to use with real-time applications.