Addis Ababa Science & Technology University

Department of Electrical & Electronics Engineering

Probability and Random Process (EEEg-2114)

Chapter 4: Random Processes

 Random Processes
Outline
 Introduction
 Definition of a Random Process
 Characterization of Random Processes
 Mean, Correlation, and Covariance Functions
 Classification of Random Processes
 Power Spectral Densities of Random Processes
 Response of Linear Systems to Random Inputs

Chapter 4 Random Process 2

 Introduction

 The theory of random processes was first developed in

connection with the study of fluctuations and noise in physical
systems.

 A random process is the mathematical model of an empirical

process whose development is governed by probability laws.

 Random processes provides useful models for the studies of

such diverse fields as statistical physics, communication and
control, time series analysis, population growth, and
management sciences.
Chapter 4 Random Process 3
 Definition of a Random Process

 A random process is a family of random variables {X(t), tϵT}

defined on a given probability space, indexed by the parameter
t, where t varies over an index set T.

 In a random process {X(t), tϵT}, the index set T is called the

parameter set of the random process.

 The values assumed by X(t) are called states, and the set of all
possible values forms the state space E of the random process.

 If the index set T of a random process is discrete, then the

process is called a discrete-time random process.
Chapter 4 Random Process 4
 Definition of a Random Process Cont’d…..

 A discrete-time random process is also called a random

sequence and is denoted by {Xn , n = 1, 2, 3, . . .).

 If T is continuous, then we have a continuous-time random

process.

 In fact, a random process {X(t), tϵT} is a function of two

arguments {X(t, ω), tϵT, ωϵΩ}.

 For a fixed time t=tk, X(tk, ω) = Xk(ω) is a random variable

denoted by X(tk), as ω varies over the sample space Ω.

Chapter 4 Random Process 5

 Definition of a Random Process Cont’d…..

 On the other hand, for a fixed sample point ωi ϵΩ, X(t, ωi) =
Xi(t) is a single function of time t, called a sample function or a
realization of the process.

 The totality of all sample functions is called an ensemble.

 Of course if both ω and t are fixed, X(tk , ωi) is simply a real

number.

 In the following discussion, we use the notation X(t) to

represent X(t, ω).

Chapter 4 Random Process 6

 Characterization of Random Processes

 If X(t) is a random process, then for fixed t=t1, X1=X(t1)

represents a random variable.
 Its distribution function is given by:
FX ( x1 , t1 )  P{X (t )  x1}
 Notice that FX(x, t) depends on t, since for a different t, we
obtain a different random variable.
 The first-order probability density function of the process X(t)
is defined as:
dF ( x1 , t1 )
f ( x1 , t1 ) 
X
X

dx1

Chapter 4 Random Process 7

 Characterization of Random Processes Cont’d…..

 For t = t1 and t = t2, X(t) represents two different random

variables X1 = X(t1) and X2 = X(t2) respectively.
 Their joint distribution is given by:
FX ( x1 , x2 , t1 , t2 )  P{ X (t1 )  x1 , X (t2 )  x2 }
 The second-order probability density function of the random
process X(t) is:
 2 FX ( x1 , x2 , t1 , t2 )
f X ( x1 , x2 , t1 , t2 ) 
x1 x2

 Similarly f X ( x1 , x2 , xn , t1 , t2 , tn ) represents the nth order

density function of the process X(t).

Chapter 4 Random Process 8

 Mean, Correlation, and Covariance Functions

 As in the case of random variables, random processes are often

described by using statistical averages.

 The mean of X(t) is defined by:

 X (t )  EX (t )
where X(t) is treated as a random variable for a fixed value of t.

 In general, μX(t) is a function of time, and it is often called the

ensemble average of X(t).

Chapter 4 Random Process 9

 Mean, Correlation, and Covariance Functions …...

 A measure of dependence among the random variables of X(t)

is provided by its autocorrelation function, defined by:

RXX (t1 , t 2 )  EX (t1 ) X ( 2 )

 Note that:

R XX (t1 , t 2 )  R XX (t 2 , t1 ) and R XX (t , t )  E X 2 (t ) 
 The autocovariance function of X(t) is defined by:

C XX (t1 , t 2 )  CovX (t1 ) , X (t 2 )  EX (t1 )   X (t1 )X (t 2 )   X (t 2 )

 R XX (t1 , t 2 )   X (t1 )  X (t 2 )

Chapter 4 Random Process 10

 Mean, Correlation, and Covariance Functions …...

 It is clear that if the mean of X(t) is zero, then:

C XX (t1 , t 2 )  RXX (t1 , t 2 )

 Note that the variance of X(t) is given by:


 X 2 (t )  VarX (t )  E X (t )   X (t )2 
 If X(t) is a complex random process, then its autocorrelation
function RXX(t1, t2) and autocovariance function CXX(t1, t2) are
defined, respectively, by:
 
R XX (t1 , t 2 )  E X (t1 ) X * (t 2 ) and


C XX (t1 , t 2 )  E X (t1 )   X (t1 )X (t 2 )   X (t 2 )
*

Chapter 4 Random Process 11
 Classification of Random Processes

i. Stationary Processes
 A random process {X(t), tϵT} is said to be stationary or strict-
sense stationary (SSS) if, for all n and for every set of time
instants (ti ϵT, i = 1,2, . . . , n),
FX ( x1 ,........, xn , t1 , ....., t n )  FX ( x1 , ........, x n , t1   , ......, t n   )

 Hence, the distribution of a stationary process will be

unaffected by a shift in the time origin, and X(t) and X(t+τ)
will have the same distributions for any τ.
 Nonstationary processes are characterized by distributions
depending on the points t1, t2, . . . , tn.

Chapter 4 Random Process 12

 Classification of Random Processes Cont’d……

ii. Wide-Sense Stationary Processes

 A random process X(t) is wide-sense stationary (WSS) if:

1. EX (t )   X (constant)

2. R XX (t1 , t 2 )  EX (t1 ) X (t 2 )  R XX  t 2  t1 

 Note that a strict-sense stationary process is also a WSS

process, but, in general, the converse is not true.

Chapter 4 Random Process 13

 Power Spectral Densities of Random Processes

 The autocorrelation function of a continuous-time random

process X(t) is defined as:

RXX ( )  EX (t ) X (t   )

 Properties of RXX(τ):

1. R XX ( )  R XX ( )

2. R XX ( )  R XX (0)


3. R XX (0)  E X 2 (t )  0 
Chapter 4 Random Process 14
 Power Spectral Densities of Random Processes……

 In case of a discrete-time random process X(n), the

autocorrelation function of X(n) is defined by:

RXX (k )  EX (n) X (n  k )

 Properties of RX(k):

1. R XX (k )  R XX (k )

2. R XX (k )  R XX (0)


3. R XX (0)  E X 2 (n)  0 
Chapter 4 Random Process 15
 Power Spectral Densities of Random Processes……

 Two processes X(t) and Y(t) are called (mutually) orthogonal if:

RXY ( )  0 , for all 

 Similarly, the cross-correlation function of two discrete-time

jointly WSS random processes X(n) and Y(n) is defined by:

RXY (k )  EX (n)Y (n  k )

 The various properties of RXY(k) similar to those of RXY(τ) can

be obtained by replacing τ by k in the above equations.

Chapter 4 Random Process 16

 Power Spectral Densities of Random Processes……

 The power spectral density (or power spectrum) SXX(ω) of a

continuous-time random process X(t) is defined as the Fourier
transform of RXX(τ), i.e. ,

S XX     R XX ( )e  j  d


 Thus, taking the inverse Fourier transform of SX(ω), we obtain:


R XX   
1
2 
S XX ( )e j  d

 The above equations are known as the Wiener-Khinchin

relations.
Chapter 4 Random Process 17
 Power Spectral Densities of Random Processes……

 Properties of SXX(ω):

1. S XX ( ) is real and S XX ( )  0

2. S XX ( )  S XX ( )

 
3. E X (t )  R XX (0) 
2 1
2 



S XX ( )d

 Similarly, the power spectral density SXX(Ω) of a discrete-time

random process X(n) is defined as the Fourier transform of
RXX(k): 
S XX    XX
R ( k )e  jk

k  
Chapter 4 Random Process 18
 Power Spectral Densities of Random Processes……

 Thus, taking the inverse Fourier transform of SXX(Ω), we obtain:

1 
R XX (k ) 
2 

S XX ()e jk d

 Properties of SXX(Ω):
1. S XX (  2 )  S XX ()

2. S XX () is real and S XX ()  0

3. S XX ()  S XX ()

 
3. E X (n)  R XX (0) 
2 1
2

S

XX ()d

Chapter 4 Random Process 19

 Power Spectral Densities of Random Processes……

 The cross power spectral density (or cross power spectrum)

SXY(ω) of two continuous-time random processes X(t) and Y(t)
is defined as the Fourier transform of RXY(τ):

S XY     R XY ( )e  j  d


 Thus, taking the inverse Fourier transform of SXY(ω), we get:


R XY   
1
2 
S XY ( )e j  d

Chapter 4 Random Process 20

 Power Spectral Densities of Random Processes……

 Properties of SXY(ω):
 Unlike SXX(ω), which is a real-valued function of ω, SXY(ω), in
general, is a complex-valued function.

1. S XY ( )  S YX ( )

2. S XY ( )  S XY ( )
*

 Similarly, the cross power spectral density SXY(Ω) of two

discrete-time random processes X(n) and Y(n) is defined:

S XY ()   XY
R (
k  
k )e  jk

Chapter 4 Random Process 21

 Power Spectral Densities of Random Processes……

 Taking the inverse Fourier transform of SXY(Ω), we get:

1 
R XY (k ) 
2 

S XY ()e jk d

 Properties of SXY(ω):
 Unlike SXX(Ω), which is a real-valued function of Ω, SXY(Ω), in
general, is a complex-valued function.

1. S XY (  2 )  S XY ()

2. S XY ()  S YX ()

3. S XY ()  S XY ()
*

Chapter 4 Random Process 22

Example on Random Processes
Example:
Consider a random process X(t) defined by
X (t )  A cos( 0 t   )

where  0 and A are constants and  is a uniform

random variable over the interval (0, 2 )

a. Find the mean  X (t ).

b. Find the autocorrelation function R XX (t1 , t 2 ).

c. Find the autocovariance function C XX (t1 , t 2 ).

d . Determine whether X (t ) is WSS random process or not.

e. Find the power spectral density of X (t ).
Chapter 4 Random Process 23
Example on Random Processes Cont’d……
Solution:
a.  X (t )  E X (t )  E A cos( 0 t   )  AE cos( 0 t   )

But, cos( 0 t   )  cos( 0 t)cos - sin( 0 t)sin

  X (t )  E X (t )  AE cos( 0 t)cos - sin( 0 t)sin 

 A cos( 0 t) E cos   A sin( 0 t) E sin  

2
E cos  
1
2 
0
cosd  0

2
Similarly , E sin   
1
2 0
sin d  0

  X (t )  E X (t )  0

Chapter 4 Random Process 24

Example on Random Processes Cont’d……
Solution:
b. R XX (t1 , t 2 )  E X (t1 ) X (t 2 )

 E A cos( 0 t1   ) A cos( 0 t 2   )

 A 2 E cos( 0 t1   ) cos( 0 t 2   )

A2
 E cos 0 (t 2  t1 )  cos( 0 (t1  t 2 )  2 )
2
But , E cos 0 (t 2  t1 )  cos 0 (t 2  t1 ) and

E cos( 0 (t1  t 2 )  2 )  0

A2
 R XX (t1 , t 2 )  cos 0 (t 2  t1 )
2
Chapter 4 Random Process 25
Example on Random Processes Cont’d……
Solution:
c. C XX (t1 , t 2 )  R XX (t1 , t 2 )   X (t1 )  X (t 2 )

A2
 cos 0 (t 2  t1 )  0
2
A2
 C XX (t1 , t 2 )  cos 0 (t 2  t1 )
2
d . Since the mean is constant and the autocorrelation function
depends on time differenceonly, X (t ) is a WSS random process.

Chapter 4 Random Process 26

Example on Random Processes Cont’d……
Solution:
e. Since X (t ) is a WSS random process,the autocorrelation
function can be simply written as :

A2
R XX ( )  cos( 0 )
2
The power spectral density of X (t ) is given by :

S XX ( )   R XX ( )e  j  d


But from Fourier transform pair table, we have :

FT cos( 0 t )   (   0 )   (   0 )

A 2 A 2
 S XX ( )   (   0 )   (   0 )
2 2
Chapter 4 Random Process 27
 Response of Linear Systems to Random Inputs

 If a WSS random process X(t) with autocorrelation function

RXX(τ) is applied to a linear system with impulse response h(t),
then the cross correlation function RXY(τ) and the output
autocorrelation function RYY(τ) are given as follows.

X(t) h(t) Y(t)

R XY ( )  R XX ( ) * h * ( )

And ,
RYY ( )  R XY ( ) * h( )

 R XX ( ) * h * ( ) * h( )
Chapter 4 Random Process 28
 Response of Linear Systems to Random Inputs…..

 Using properties of Fourier transform, we get:

f (t ) 
FT
F ( ) and g (t ) 
FT
G( )

 f (t ) * g (t ) 
FT
F ( )G( )
 Then using the above property, the cross and output power
spectral densities can be evaluated as:
 
S XY ( )  FT R XX ( ) * h * ( )  S XX ( ) H * ( )

And ,
S YY ( )  FT RYY ( )  FT R XY ( ) * h( )

 S XY ( ) H ( )  S XX ( ) H ( )
2

Chapter 4 Random Process 29

Exercise on Random Processes
1. Consider a random process X(t) defined by

X (t )  A cos(0t   )
where 0 and  are constants and A is a uniform
random variable over the interval (0, 2)
a. Find the mean  X (t ).
b. Find the autocorrelation function RXX (t1 , t 2 ).
c. Find the autocovariance function C XX (t1 , t 2 ).
d . Determine whether X (t ) is WSS random process or not.
e. Find the power spectral density of X (t ).

Chapter 4 Random Process 30

Exercise on Random Processes
2. Consider a random process X(t) defined by:

X (t )  A sin( 0 t   )
where A and  are independent random variables which are
  
uniformly distributed over the intervals [0, 1] and  , 
 2 2
respectively and  0 is a constant.
a. Find the mean  X (t ).
b. Find the autocorrelation function R XX (t1 , t 2 ).
c. Find the autocovariance function C XX (t1 , t 2 ).
d . Determine whether X (t ) is WSS random process or not.
e. Find the power spectral density of X (t ).

Chapter 4 Random Process 31

Exercise on Random Processes Cont’d……

3. Two random processes X(t) and Y(t) are given by:

X (t )  A cos(0t   ) and Y (t )  A sin(0t   )

where A and 0 are constants and  is a uniform random
variable over the interval (0, 2 ).
a. Find the cross correlation function of X (t ) and Y (t ).
b. Verify that RXY (- )  RXY ( )

Chapter 4 Random Process 32