MIT-WPU

Communication Systems (ECE2002B)

Module III: Noise and Random Processes:

 Sources of noise, types of noise, noise figure, noise factor, noise

resistance, noise temperature, bandwidth, SNR, Friis’s formula, noise
calculations.
 Introduction to random process, Stationary processes, Mean,
Correlation & Covariance functions, Ergodic processes, Transmission
of a random process through a LTI filter, Power spectral density,
Gaussian process. [8 hrs]
Random Processes
Importance of Random Processes
 Random variables and processes talk about quantities and signals
which are unknown in advance
 The data sent through a communication system is modeled as
random variable
 The noise, interference, and fading introduced by the channel can all
be modeled as random processes
 Even the measure of performance (Probability of Bit Error) is
expressed in terms of a probability
 Physical Phenomenon can be represented by a mathematical model
 Model can be : Deterministic –no uncertainty , Random/ stochastic –
uncertainty
5/26/2023 Random Processes 4
Deterministic and random processes
• both continuous functions of time (usually)

• Deterministic processes :
physical process is represented by explicit mathematical
relation (No Uncertainty about its time behavior)

• Random(Stochastic) processes :
result of a large number of separate causes. Described in
probabilistic terms and by properties which are averages

5/26/2023 Random Processes 5

 Example of Random process
 RADIO COMMUNICATION SYSTEM
o Information bearing signal component
o A Voice signal, consist of randomly spaced bursts of energy of random
duration
o A random Interference Component
o Spurious Electromagnetic waves produced by other communication
system operating in the vicinity of the radio receiver
o Channel Noise
o Random motion of electrons in conductors and devices at the front end of
the receiver.
✓ Received Signal is random in Nature

5/26/2023 Random Processes 6

Random Processes: significance
• In practical problems, we deal with time varying
waveforms whose value at a time is random in nature.
Randomness or unpredictability is a fundamental property of
information.
For example,
• The speech waveform recorded by a microphone,
• The signal received by communication receiver
• The temperature of a certain city at noon or
• The daily record of stock-market data represents
random variables that change with time.
• How do we characterize such data? Such data are
characterized as random or stochastic processes.

5/26/2023 Random Processes 7

Properties of Random Process
1. They are function of time

2. They are random in sense that before conducting experiment, it is not

possible to define exactly waveforms that will be observed in future.

5/26/2023 Random Processes 8

 Random Experiment
 Sample Space S
 Collection of all possible separately identifiable outcomes
 Coin toss: head, tail , 2 outcomes = sample space
 Sample Element or Point
 Each outcome is an sample element or sample point.
 Random Variables
 Outcome of a Random Experiment is mapped into a number.

5/26/2023 Random Processes 9

 Difference Between Random Processes and
Random variable
• A random variable maps each sample point in the
sample space to a point in the real line(Number (need not be
function of time)).

• A random process maps each sample point to a

waveform.
•Thus a random process is a function of the sample
point ‘s’ and index variable ’t’ and may be written as

for fixed t=t0 X(t0, s) is a random variable.

5/26/2023 Random Processes 10
Random Variable (RV)
 RV maps to a point on a real number line

Sample space
S1 S6

S2 S4

S5
S3

Real Number line

5/26/2023 MIT WPU TY EC

 Sample Function of random Process

5/26/2023 Random Processes 12

 Ensemble and sample function

• The collection of all possible waveform is

known as Ensemble. (sample space in random
variable)
• Each waveform in the collection is a sample
function (sample point)
• Amplitudes of all the sample functions at t=t0
is ensemble statistics.
• Ensemble of random variables , is called as Random Process.

5/26/2023 Random Processes 13

Comparison between RV and RP
Random variable Random process
Set of numbers Set of rv, or set of waveforms or
functions
Sample space Ensemble
Need not be a function of time It is a function of time
Discrete Stationary
Continuous Ergodic

Average for the sample space Ensemble average and the time
average can be calculated

5/26/2023 Random Processes 14

 Classification of random processes

• Stationary and Non stationary

• Wide-Sense or Weakly Stationary

• Ergodic

5/26/2023 Random Processes 15

 Terminology Describing Random Processes
 A stationary random process has statistical properties which do not
change at all time
 A wide sense stationary (WSS) process has a mean and
autocorrelation function which do not change with time
 A random process is Ergodic if the time average always converges to
the statistical average
 Unless specified, we will assume that all random processes are WSS
and Ergodic

 Statistical properties : mean , autocorrelation, variance, co-variance

5/26/2023 MIT WPU TY EC

Random Process
 Strictly Stationary  Non Stationary
Process Process
 Statistical  Statistical
Characterization of Characterization of
Process is time Process is function of
invariant time.
 Generated from a stable  Generated from
physical phenomenon. unstable physical
 Ex. noise
phenomenon.
 Ex. Temp. of a city

5/26/2023 Random Processes 17

5/26/2023 Random Processes 18
  White Noise is  Cymbal clashing is
Stationary Process non-stationary

5/26/2023 Random Processes 19

 Mathematical Explanation
 Random process X(t), that is initiated at
t=-∞
• Random Variables X(t1),X(t2),…..X(tk) observed at times t1,t2,….,tk
• Joint distribution function of this set of Random Variables is
• Shift all observation by a fixed amount
• New set of random variables

• Joint distribution function for this set

5/26/2023 Random Processes 20

 Strictly Stationary random process :
• Following condition should satisfied

For all time shift all k, all possible choices

of observation times t1,t2,….,tk.
strictly
stationary

is invariant with respect to the location of the

origin t=0.

5/26/2023 Random Processes 21

1. First order distribution function of a stationary random process is
independent of time
for k=1

2. Second order distribution function depends only on time difference

between the observation times and not on the particular times at
which the random process is observed.
For k=2and

5/26/2023 Random Processes 22

 Example
Consider below fig, depicting three spatial windows located at times t ,t ,t . We wish to
1 2 3
evaluate the probability of obtaining a sample function x(t) of a random process X(t) that
passes through this set of windows, that is, the probability of the joint event.

5/26/2023 Random Processes 23

 A process X(t) is
strictly stationary if
the probability of
the set of sample
functions of this
process passing
through the
windows of fig.(a)
is equal to
the probability of
the set of sample
functions of this
process passing
through the
windows of fig.(b)

5/26/2023 Random Processes 24

 Mean

 Expectation of random variable obtained by observing the process at

some time t.

First order
probability
density
function of the
process
5/26/2023 Random Processes 25
 Contd…
 For a strictly stationary random process, is independent of
time.
So,

Use this equation to find that whether process is strict stationary or

not in Example.

5/26/2023 Random Processes 26

 Autocorrelation
 Expectation of the product of two random
variables, X(t1) and X(t2),
obtained by observing the process X(t) at times t1
and t2 respectively.

Second order
Probability
Density function

5/26/2023 Random Processes 27

 Auto covariance
 Simplifying above eq. we get

 Can find covariance from mean and

autocorrelation.

5/26/2023 Random Processes 28

 Wide sense or Weakly or second order
stationary :
• Mean value is constant
• Autocorrelation function is independent
of the shifts
Rx (t1 , t2 ) = Rx (t2 − t1 ) = Rx ( )

• All stationary processes are wide-sense

stationary but converse is not true.

5/26/2023 Random Processes 29

 Stationarity

5/26/2023 Random Processes 30

5/26/2023 Random Processes 31
Trigonometric Equations
sin(x+y)=sin(x)cos(y)+cos(x)sin(y)

sin(x–y)=sin(x)cos(y)–cos(x)sin(y)

cosx⋅cosy=(1/2)*cos(x+y)+cos(x−y)

sin(x + 2π) = sin(x) sin(x − 2π) = sin(x)

𝒙+𝒚 𝒙−𝒚
𝒔𝒊𝒏𝒙 − 𝒔𝒊𝒏𝒚 = 𝟐(cos 𝒔𝒊𝒏( ))
𝟐 𝟐

cos(2x)=2cos2(x)−1

Sin A cos B=1/2(sin (A+B) + sin (A-B))

5/26/2023 Random Processes 32

 Ergodic random process :
• Ensemble averages are equal to time averages of
any sample function.
T 2 T 2

E  X (T ) = x(t ) = lim  x(t )dt = lim   X dt =  X

1 1
T → T T → T
−T 2 −T 2

• stationary process in which averages from a single

record are the same as those obtained from
averaging over the ensemble
• Most stationary random processes can be treated
as Ergodic
5/26/2023 Random Processes 33
 Ergodic

5/26/2023 Random Processes 34

 Mean, Correlation and covariance function
• Mean value :

x(t)

x

time, t T


 X (t ) = EX (t ) =  xf X (t ) ( x)dx
−

5/26/2023 Random Processes 35

Mean, Correlation and covariance function
• Mean value of a stationary random
process is a constant.  (t ) = 
X X

• Autocorrelation function of a random

process X(t)is given as

RX (t1 , t2 ) = EX (t1 ) X (t2 )

5/26/2023 Random Processes 36

5/26/2023 Random Processes 37
5/26/2023 Random Processes 38
 • Autocorrelation :
x(t)

time, t T

• The autocorrelation, or autocovariance,

describes the general dependency of x(t)
with its value at a short time later, x(t+)

5/26/2023 Random Processes 39

 Autocorrelation properties

Symmetry

2. Power of W.S.S.
process
3. Maximum value

RX (t1 , t2 ) = EX (t1 ) X (t2 )

Tau = t1-t2 =0 , t1 =t2

5/26/2023 Random Processes 40
 Autocorrelation function of slowly and
rapidly fluctuating random processes

Correlation and decorrelation time

concept
5/26/2023 Random Processes 41
Mean, Correlation and covariance function
• Autocovariance function of a stationary
random process X(t)is given as

C X (t1 , t 2 ) = E ( X (t1 ) −  X )( X (t 2 ) −  X )
= RX (t 2 − t1 ) −  2
X

5/26/2023 Random Processes 42

5/26/2023 Random Processes 43
 ( ) ( )
_________________  
Rxy ( t1 , t2 ) = x t y t =   x1 y2 f xy ( x1 , y2 ) dx1dy2
1 2 − −

5/26/2023 Random Processes 44

5/26/2023 Random Processes 45
 If τ=0 then cross-correlation results in zero i.e. the random
variables are orthogonal

5/26/2023 Random Processes 46

5/26/2023 Random Processes 47
 Spectral density

5/26/2023 Random Processes 48

 Transmission of a random process through a
Linear filter
Y (t ) = E Y (t )
X(t)
w.s.s Y(t)
 
I Impulse 
= E   h( ) X (t −  )d 
Response h(t)

 − 


random process −

The mean of the output = h( ) E[ X (t −  )]d

Y(t) is given as 
= X 
−
h ( ) d  =  X H ( 0 )

5/26/2023 Random Processes 49

 Points to remember

 Mean of random process Y(t) produced at the output of a LTI system in

response to input random process X(t) equals to the mean of X(t)
multiplied by the dc response of the system.
 Autocorrelation function of the random process Y(t) is a constant.

5/26/2023 Random Processes 50

 Filtering of random signals

5/26/2023 Random Processes 51

5/26/2023 Random Processes 52
 Gaussian Random Processes
 Gaussian random processes have some special
properties

- If a Gaussian random process is wide-sense

stationary,
then it is also stationary
- If the input to a linear system is a Gaussian random
process, then the output is also a Gaussian random
process
- if the random variable X(t1), X(t2),…..X(tn)
obtained by sampling a Gaussian process X(t) at times
t1, t2, ……tn are uncorrelated then these random
variables are statistically independent.

5/26/2023 MIT WPU TY EC

 Gaussian process
 A stochastic process is said to be a normal or Gaussian process if its
joint probability distribution is normal.

 A Gaussian process is strictly and weakly stationary because the

normal distribution is uniquely characterized by its first two
moments.

1  ( y − Y )2  1  y2 
fY ( y ) = exp− = exp− 
2  Y  2 Y 
2
2  2

5/26/2023 Random Processes 54

PDF

5/26/2023 Random Processes 55

 • Central limit theorem
• Mathematical Justification for using a Gaussian process as a model
– The sum of a large number of independent and identically
distributed random variables getting closer to Gaussian
distribution
• Thermal noise can be closely modeled by Gaussian process

• The theorem states that the probability distribution of N

random variables approaches a normalized Gaussian
distribution in the limit as N approaches infinity.

5/26/2023 Random Processes 56

 Properties of a Gaussian Process
Property-I
If a Gaussian process X(t) is applied to a stable linear filter , then the
random process Y(t) developed at the output of the filter is also Gaussian.

5/26/2023 Random Processes 57

58