ECE 603

Probability and Random

Processes
Lessons 7-9
Chapter 4
Continuous and Mixed Random Variables

© 2020 UMass Amherst Global. All rights reserved.

Objectives
• Explore tools used and methods used to work with continuous random variables.
• Examine mixed random variables that are mixtures of discrete and continuous
random variables.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

2
Rationale
• Consider that a continuous random variable X has a range in the form of an
interval or a union of non-overlapping interview on the real line. This leads to
the need for new tools to help you focus on continuous random variables.
• The theory of continuous random variables is analogous to the theory of
discrete random variables. So, you may take any formula about discrete
random variables and replace sums with integrals to come up with the
corresponding formula for continuous random variables.
• Chapter 4 focuses on these relationships.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

3
Prior Learning
• Basic Concepts
• Counting Methods
• Random Variables
• Access to the online textbook: https://www.probabilitycourse.com/

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

4
Continuous Random Variables
So far, we have discussed discrete random variables:
For a discrete random variable , , is a countable set, e.g.,

the CDF , looks like a series of steps with jumps at

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

5
Discrete Random Variables: PMF & CDF
The jump at is given by the PMF

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

6
Cumulative Distributive Function (CDF)
Recall the general properties of a CDF:
1)

2)

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

7
Continuous Random Variables
If is not countable then is not a discrete random variable.
Example.
Remember: etc.
Example.
• : Lifetime of a light bulb,
• : Voltage across a resistor,

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

8
Continuous Random Variables
Now suppose we have a continuous function having those properties – for
example:
• A function like this is also a valid CDF
. – that is, it can also represent
. for some random variable .

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

9
Continuous Random Variables
•Definition.
  A random variable having a CDF that is a continuous
function for all in is said to be a continuous random variable.

Example. Let be an interval in the real line (where and are real
numbers with ). Let be a number chosen at random from that
interval.
“Chosen at random” means: If , then

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

10
Continuous Random Variables
Let’s find the CDF
a)

b)

c)

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

11
Continuous Random Variables
In this case, is called a random variable.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

12
Continuous Random Variables
Example. Consider a line of length 1, i.e., . Place a dot at random on the
line.
Location of the dot,

a) What is

b) What is

c) For , what is

d) Obtain the CDF of .

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

13
Continuous Random Variables

e) What is ? why?

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

14
Continuous Random Variables
• For a discrete random variable: the rate of increase in the CDF is
characterized by the PMF - that is, by the locations and sizes of the
jumps in the CDF.

• What characterizes the rate of increase for a continuous function?

The derivative of the function

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

15
Continuous Random Variables
• If the CDF is a continuous function, then is said to be a continuous
random variable.
• PMF is Not well-defined for continuous random variables. Instead, we define
probability density function (pdf).

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

16
Probability Density Function (PDF)
Definition. pdf:

Remember that,

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

17
Probability Density Function (PDF)

Definition. Consider a continuous random variable with an absolutely

continuous CDF . Then we have

is called the probability density function (PDF) of .

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

18
Probability Density Function (PDF)
Theorem.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

19
Probability Density Function (PDF)
Example. Say that .

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

20
Probability Density Function (PDF)
Back to our dot and line example:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

21
Probability Density Function (PDF)
Comparison with discrete random variable:

discrete continuous

If we integrate over the entire real line, we must get 1, i.e.,

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

22
Probability Density Function (PDF)
In math: A density function for a quantity is a non-negative function that is
integrated to give that quantity.
Let’s look at the properties of a PDF:
1) Since is monotone non-decreasing, its derivative must satisfy

2)

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

23
Probability Density Function (PDF)
3) In general:

4) We can extend the property 3 to:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

24
Probability Density Function (PDF)
• So, for a discrete random variable having range to find
. , we sum the PMF over the points

• For a continuous random variable, to find , we integrate the PDF

over the set .

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

25
Probability Density Function (PDF)
Example. Let be a continuous random variable with the following PDF

a) Find .
b) Find .
c) Find .

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

26
Probability Density Function (PDF)

Important note: is not equal to . In fact, for a continuous

random variable we have for every point . Also, it is
possible in general to have for some values of .

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

27
Expected Value and Variance

So, we get:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

28
Expected Value and Variance
Law of the unconscious statistician (LOTUS) for continuous random variables:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

29
Expected Value and Variance
Remember that the variance of any random variable is defined as

So for a continuous random variable, we can write

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

30
Expected Value and Variance
Example. In a medical ultrasound system, the energy in a pulse scattered by
diffuse reflectors is a random variable having a pdf of the form:

a) Find the value of

b) Find

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

31
Expected Value and Variance
c) Find

d) Find

e) Let , find

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

32
Functions of Continuous Random Variables

Suppose that is a continuous random variable having CDF and

PDF . g .: Let
R® R be some function, and let

Since is a random variable, so is . We already know that we can find

by

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

33
Functions of Continuous Random Variables
Question: How do we find the CDF and PDF for ?

Usually: It is easier to first find the CDF for , and then take the derivative to
find the PDF.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

34
Functions of Continuous Random Variables
Example. Suppose that and .
Find the CDF and PDF of .
Note that:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

35
Functions of Continuous Random Variables
Example. Suppose that and .
Find the CDF and PDF of .
Note that:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

36
Functions of Continuous Random Variables
There’s another approach called the Method of Transformations that
sometimes gives a quicker way of finding directly from .

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

37
Functions of Continuous Random Variables
If is a continuous random variable and is a function of , then
itself is a random variable.
Steps:

1) Find

2) Find CDF of

3) Find

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

38
Functions of Continuous Random Variables
Example. Now suppose that and
Find CDF and PDF of

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

39
Summary of Continuous Random Variable

• PDF:

• Expected Value:

• LOTUS:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

40
Summary of Continuous Random Variable
Definition. Consider a continuous random variable with an absolutely
continuous CDF . The function defined by

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

41
Summary of Continuous Random Variable
Consider a continuous random variable with PDF . We have

1)

2)

3)

4) More generally, for a set ,

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

42
Summary of Continuous Random Variable
Discrete RVs Continuous RVs
PMF PDF

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

43
Summary of Continuous Random Variable
Functions of Continuous RVs:

Steps:

1)

2)

Read method of Transformations (4.1.3).

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

44
Special Distributions
• Uniform Distribution:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

45
Special Distributions
Example. Let

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

46
Special Distributions
• Exponential Distribution:

Where denote the unit step function.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

47
Special Distributions
We find the CDF using the equation

So

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

48
Special Distributions
Example. CDF of

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

49
Special Distributions
Expected value

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

50
Special Distributions

Thus, we obtain

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

51
Special Distributions
If , then

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

52
Special Distributions
If is exponential with parameter , then is a memoryless random
variable, that is

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

53
Special Distributions
Proof:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

54
Special Distributions
Exponential random variables are often used to model times between arrivals
at service centers (these are called interarrival times) – the memoryless
property says that no matter how long you have waited for an arrival ( ),
the probability that you will have to wait for more than more time instants is
always (that is, it doesn’t depend on ).

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

55
Special Distributions
• Normal (Gaussian) Distribution:
The normal distribution is by far the most important probability distribution.
One of the main reasons for that is the Central Limit Theorem (CLT).

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

56
Special Distributions
As we’ll see later: The Central Limit Theorem says that random variables that
are formed as sums of many independent random variables have distributions
that are approximately Gaussian (e.g., the random voltage caused by electronic
noise that results from the summed contributions of a very large number of
randomly-located charge carriers).

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

57
Special Distributions
• Standard normal random variable:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

58
Special Distributions
Expected value of the standard normal:

Since is and odd function.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

59
Special Distributions
Variance of the standard normal:

Integration by parts.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

60
Special Distributions
CDF of the standard normal: The CDF of the standard normal distribution is
denoted by the function.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

61
Special Distributions
Here are some properties of the function:

1)

2)

3)

4)

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

62
Special Distributions
The standard normal CDF can’t be evaluated in closed form (except at a
few specific values of ), but numerical approximations are widely available
(e.g., the normcdf command in MATLAB).

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

63
Special Distributions
• General Normal random variables:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

64
Special Distributions
• General Normal random variables:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

65
Special Distributions
CDF and PDF of Normal random variables ( ):

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

66
Special Distributions
If ,

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

67
Special Distributions
Example. PDF (top) and CDF (bottom) of random variable (left) and
. random variable (right):

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

68
Special Distributions
Example. Say that , find

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

69
Special Distributions
Example. The IQ of randomly selected individuals is often assumed to follow a
Normal distribution with mean and standard deviation . Find
the probability that a randomly selected individual has an IQ:
a) Above 140
b) Between 120 and 130
c) Find the value of such that 99% of the population has IQ at least .

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

70
Special Distributions
Note: In MATLAB, the command calculates

• So, for example: If and we want to calculate

we can use the command:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

71
Special Distributions
Example. Suppose that . Then:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

72
Special Distributions

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

73
Special Distributions
• So, the probability that a Gaussian random variable takes a value within 1
standard deviation from its mean is 0.6827.

• The probability that a Gaussian random variable takes a value within 3

standard deviations from its mean is 0.9973.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

74
Special Distributions
Theorem. If , and , where , then
, where

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

75
Special Distributions
Example. Suppose that , and let .
Find

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

76
Gamma Distribution
Gamma function:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

77
Gamma Distribution
Gamma Distribution:

If

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

78
Gamma Distribution
If are independent, then

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

79
Mixed Random Variables
We will:
1) Define PDF (generalized PDF) for discrete random variables by using the
delta function.
2) Introduce mixed random variables.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

80
Mixed Random Variables

CDF for discrete RVs

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

81
Mixed Random Variables

Idea: Define the “derivative” of and that we can extend the

PDF to the discrete RVs.

delta function

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

82
Mixed Random Variables
Delta function:
Where is small.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

83
Mixed Random Variables

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

84
Mixed Random Variables
Lemma. Let be a continuous function. We have

Proof:

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

85
Mixed Random Variables
Example. Let

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

86
Mixed Random Variables
Definition: Properties of the delta function
We define the delta function  as an object with the following properties:

1)

2) , where is the unit step function.

3) for any

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

87
Mixed Random Variables
4) For any and any function that is continuous over ,
we have

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

88
Mixed Random Variables
Discrete Random Variable

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

89
Mixed Random Variables
generalized PDF

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

90
Mixed Random Variables
For a discrete random variable with range and PMF
, we define the (generalized) probability density function (PDF) as

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

91
Orchestrated Conversation:
Mixed Random Variables

Is it true ?

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

92
Mixed Random Variables

Example.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

93
Mixed Random Variables
Random variables:
 Discrete
 Continuous
 Mixed random variable

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

94
Mixed Random Variables
The (generalized) PDF of a mixed random variable can be written in the form

Continuous
Discrete

where , and does not contain any delta

functions. Furthermore, we have

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

95
Mixed Random Variables
Example. Let

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

96
Mixed Random Variables
a) Find the CDF of .
b) Find the generalized PDF of .
c) Find and .

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

97
Mixed Random Variables
So mixed random variable:

Discrete part Continuous part

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

98
Orchestrated Conversation:
Mixed Random Variables

Example. Let
a) Find and .

b) Find .

c) Find .

d) Find and .

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

99
Post-work for Lesson
• Complete homework assignment for Lessons 7-9: HW#4 and HW#5

Go to the online classroom for details.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

100
To Prepare for the Next Lesson
• Read Chapter 5 in your online textbook:
https://www.probabilitycourse.com/chapter5/5_1_0_joint_distributions.php
• Complete the Pre-work for Lessons 10-13.

Visit the online classroom for details.

Chapter 4 © 2020 UMass Amherst Global. All rights reserved.

101