30

4. Week 4

Theorem 4.1. Let X be an RV defined on a probability space (Ω, F, P) with law PX = P ◦ X −1

and DF FX . Then

(a) FX is non-decreasing, i.e. FX (x) ≤ FX (y), ∀x < y.

(b) FX is right continuous, i.e. limh↓0 FX (x + h) = FX (x), ∀x ∈ R.
(c) FX (−∞) := limx→−∞ FX (x) = 0 and FX (∞) := limx→∞ FX (x) = 1.

Proof. For all x < y, observe that (−∞, x] ⊊ (−∞, y]. Since PX is a probability measure, we have
PX ((−∞, x]) ≤ PX ((−∞, y]). The statement (a) follows.
By definition, FX takes values in [0, 1] and hence it is bounded. Since FX is non-decreasing, the
limit FX (x+) = limh↓0 FX (x + h) exists for all x ∈ R. Using the non-decreasing property, we use
the following fact from real analysis that FX (x+) = limn→∞ FX (x + n1 ). By Proposition 3.17, we
have
1 1
   
FX (x+) = n→∞
lim FX x + lim PX ( −∞, x + ) = PX ((−∞, x]) = FX (x).
= n→∞
n n
This proves statement (b). Here, we use the fact that (−∞, x + n1 ] ↓ (−∞, x].
Similar to the proof of statement (b), we have

FX (−∞) = lim FX (−n) = lim PX ((−∞, −n]) = PX (∅) = 0,

n→∞ n→∞

and
FX (∞) = lim FX (n) = lim PX ((−∞, n]) = PX (R) = 1.
n→∞ n→∞

Here, we use that facts that (−∞, −n] ↓ ∅ and (−∞, n] ↑ R. This proves statement (c). □

The next theorem is stated without proof. The arguments required to prove this statement is
beyond the scope of this course.

Theorem 4.2. Let F : R → R be a non-decreasing and right continuous function such that
limx→−∞ F (x) = 0 and limx→∞ F (x) = 1. Then there exists an RV X defined on some probability
space (Ω, F, P) such that F = FX , i.e. F (x) = FX (x), ∀x.
 31

Remark 4.3. Given any function F : R → R, as soon as we check the relevant conditions, we can
claim that it is the DF of some RV by Theorem 4.2.

Example 4.4. Consider the function F : R → R defined by





 0, if x < 0,


F (x) = x, if 0 ≤ x < 1,




1, if x ≥ 1.



The function is a constant on (−∞, 0) and on [1, ∞). Moreover, it is non-decreasing in the interval
[0, 1). Further for x < 0, y ∈ (0, 1), z > 1, we have

F (x) = F (0) < F (y) < F (1) = F (z).

Hence, F in non-decreasing over R. Again, by definition F is continuous on the intervals (−∞, 0),
(0, 1) and (1, ∞). We check for right continuity at the points 0 and 1. We have

lim F (0 + h) = lim h = 0 = F (0), lim F (1 + h) = lim 1 = 1 = F (1).

h↓0 h↓0 h↓0 h↓0

Hence, F is right continuous on R. Finally, limx→−∞ F (x) = limx→−∞ 0 = 0 and limx→∞ F (x) =
limx→∞ 1 = 1. Hence, F is the DF of some RV. Later on, we shall identify the corresponding RV.

Proposition 4.5 (Further properties of a DF). Let X be an RV defined on a probability space

(Ω, F, P) with law PX and DF FX .
(a) For all x ∈ R, the limit FX (x−) = limh↓0 FX (x − h) exists and equals PX ((−∞, x)) =
P(X < x).
Proof. Since FX in non-decreasing and bounded, as argued in Theorem 4.1, the limit
FX (x−) = limh↓0 FX (x − h) exists and moreover, by Proposition 3.17 we have
1 1
   
FX (x−) = n→∞
lim FX x− lim PX ( −∞, x − ) = PX ((−∞, x)) = P(X < x).
= n→∞
n n
Here, we use the fact that (−∞, x − n1 ] ↑ (−∞, x). □
(b) For all x ∈ R, P(X ≥ x) = 1 − FX (x−).
32

Proof. We have, P(X ≥ x) = PX ([x, ∞)) = PX ((−∞, x)c ) = 1 − PX ((−∞, x)) = 1 −

FX (x−). □
(c) For any x ∈ R, FX (x−) ≤ FX (x+).
Proof. By the non-decreasing property of FX , for all x ∈ R and positive integers n, we have,
FX (x − n1 ) ≤ FX (x + n1 ). Letting n go to infinity in this inequality, we get the result. □
(d) FX is continuous at x if and only if FX (x) = FX (x−).
Proof. A real valued function is continuous at a point x if and only if the function is both
right continuous and left continuous at the point x. Now, by construction, FX is right
continuous on R. Hence, FX is continuous at x if and only if FX is left continuous at x.
The last statement is exactly the statement to be proved. □
(e) Only possible discontinuities of FX are jump discontinuities.
Proof. As discussed in Theorem 4.1 and in part (a), for any x ∈ R, both the limits FX (x+)
and FX (x−) exist and FX (x+) = FX (x). Since FX (x−) ≤ FX (x+), the only possible
discontinuity appears if and only if FX (x−) < FX (x+). These discontinuities are jump
discontinuities. This completes the proof. □
(f) For all x ∈ R, we have FX (x+) − FX (x−) = P(X = x).
Proof. By the finite additivity of PX , we have FX (x+) − FX (x−) = P(X ≤ x) − P(X <
x) = PX ((−∞, x]) − PX ((−∞, x)) = PX ({x}) = P(X = x). □
(g) If FX has a jump at x, then the jump is given by FX (x+) − FX (x−) = P(X = x).
Proof. If FX has a jump at x, then the jump is given by FX (x+) − FX (x−). The result
follows from statement (f). □
(h) FX is continuous at x if and only if P(X = x) = 0.
Proof. Recall that FX (x+) = FX (x). Then by statement (d) and (f), we have FX is
continuous at x if and only if FX (x+) = FX (x−) and hence, if and only if P(X = x) = 0. □
(i) Consider the set D := {x ∈ R : FX is discontinuous at x} = {x ∈ R : FX (x−) <
FX (x+)} = {x ∈ R : P(X = x) > 0}. Then D is either finite or countably infinite.
(Note that if FX is continuous on R, then D = ∅.)
 33

Proof. Left as an exercise in practice problem set 4. □

(j) For all x < y, we have

P(x < X ≤ y) = FX (y) − FX (x),

P(x < X < y) = FX (y−) − FX (x),

P(x ≤ X < y) = FX (y−) − FX (x−),

P(x ≤ X ≤ y) = FX (y) − FX (x−).

Proof. We prove the first two equalities. Proof of the last two equalities are similar.
By the finite additivity of PX , we have FX (y) − FX (x) = PX ((−∞, y]) − PX ((−∞, x]) =
PX ((x, y]) = P(x < X ≤ y).
Again, FX (y−) − FX (x) = PX ((−∞, y)) − PX ((−∞, x]) = PX ((x, y)) = P(x < X < y).
This completes the proof. □

Example 4.6. Consider the function F : R → R defined by


0, if x < 0,






1 + x, if 0 ≤ x ≤ 1,


4 2
F (x) =
1
+ x4 , if 1 < x < 2,





 2


1, if x ≥ 2.



Assume that F is the DF of some RV X (left as an exercise in practice problem set 4). Since F is
continuous on the intervals (−∞, 0), (0, 1), (1, 2) and (2, ∞), discontinuities may arise only at the
points 0, 1, 2.
We have F (0−) = limh↓0 F (0 − h) = 0 and F (0) = 14 . Therefore F is discontinuous at 0 with jump
F (0) − F (0−) = 14 .
We have F (1−) = limh↓0 F (1 − h) = limh↓0 [ 14 + 1−h
2
] = 3
4
and F (1) = 1
4
+ 1
2
= 34 . Therefore F is
continuous at 1.
We have F (2−) = limh↓0 F (2 − h) = limh↓0 [ 12 + 2−h
4
] = 1 and F (2) = 1. Therefore F is continuous
at 2.
34

Only discontinuity of F is at the point 0. In particular, P(X = 0) = F (0) − F (0−) = 41 . At all

other points F is continuous and hence P(X = x) = 0, ∀x ̸= 0.
Observe that P(0 ≤ X < 1) = F (1−) − F (0−) = 43 . Again, P( 32 < X ≤ 2) = F (2) − F ( 23 ) =
1 − [ 12 + 38 ] = 18 .

We now discuss special classes of RVs defined on a given probability space (Ω, F, P). Recall
that PX and FX denote the law/distribution and the distribution function (DF) of an RV X,
respectively.

Definition 4.7 (Discrete RV). An RV X is said to be a discrete RV if there exists a finite or

countably infinite set S ⊊ R such that
X X
1 = PX (S) = P(X ∈ S) = PX ({x}) = P(X = x)
x∈S x∈S

and P(X = x) > 0, ∀x ∈ S. In this situation, we refer to the set S as the support of the discrete
RV X.

Remark 4.8. Let X be a discrete RV with DF FX and support S. Then we have the following
observations.
(a) PX (S c ) = 1 − PX (S) = 0. In particular, for any x ∈ S c , 0 ≤ P(X = x) = PX ({x}) ≤
PX (S c ) = 0 and hence P(X = x) = 0, ∀x ∈ S c .
(b) Since PX (S) = 1, for any A ⊆ R, we have PX (A) = PX (A ∩ S) (see problem set 1).
Moreover,
X
PX (A) = P(X ∈ A) = PX (A ∩ S) = P(X = x).
x∈A∩S

(c) Recall that FX is right continuous, i.e. FX (x+) = FX (x), ∀x ∈ R. Moreover, FX (x) −
FX (x−) = P(X = x). From the discussion above, we conclude that

0, if x ∈ S,

>

FX (x) − FX (x−) = P(X = x)
= 0, if x ∈ S c .



Hence, the set of discontinuities of FX is exactly the support S.

 35

(d) Note that

X X
1= P(X = x) = [FX (x) − FX (x−)].
x∈S x∈S
Hence, the sum of the jumps of FX is exactly 1.

Example 4.9. Consider the DF F : R → [0, 1] considered in Example 4.6 given by





 0, if x < 0,



1 + x, if 0 ≤ x ≤ 1,


4 2
F (x) =
1
+ x4 , if 1 < x < 2,



2




1, if x ≥ 2.



As discussed earlier, F only has a discontinuity at the point 0. If an RV X has this F as the DF,
then
X 1
P(X = x) = P(X = 0) = ̸= 1,
x∈D 4
with D = {0} as the set of discontinuities of F . This RV X is not discrete.

Example 4.10. Let X denote the number of heads in tossing a fair coin twice independently. As
computed earlier in Example 3.23, the DF FX is given by




 0, if x < 0,



1, if 0 ≤ x < 1,


4
FX (x) =
3
4, if 1 ≤ x < 2,







1, if x ≥ 2.



Clearly, the set D of discontinuities of FX is {0, 1, 2} with


1



 4
− 0 = 14 , if x = 0,


P(X = x) = FX (x) − FX (x−) =  43 − 4
1
= 12 , if x = 1,



1 − 3

= 14 , if x = 2.
4
P
Since x∈D P(X = x) = 1, the RV X is discrete with support D.
36

Definition 4.11 (Probability Mass Function (p.m.f.)). Let X be a discrete RV with DF FX and
support S. Consider the function fX : R → R defined by

X (x) − FX (x−) = P(X = x), if x ∈ S,

F

fX (x) := 
0,
 if x ∈ S c .

This function fX is called the probability mass function (p.m.f.) of X.

Example 4.12. Continuing with the Example 4.10, the p.m.f. fX is given by

1



 4
, if x = 0,



1, if x = 1,


2
fX (x) =
1
, if x = 2.,





 4


0, otherwise.



Remark 4.13. Let X be a discrete RV with DF FX , p.m.f. fX and support S. Then we have the
following observations.

(a) Continuing the discussion from Remark 4.8, we have for all A ⊆ R,
X
PX (A) = P(X ∈ A) = fX (x).
x∈A∩S

(b) As a special case of the previous observation, note that for A = (−∞, x], x ∈ R, we obtain
X
FX (x) = P(X ≤ x) = P(X ∈ (−∞, x]) = fX (t).
t∈(−∞,x]∩S

Therefore, the p.m.f. fX is uniquely determined by the DF FX and vice versa.

(c) To study a discrete RV X, we may study any one of the following three quantities, viz. the
law/distribution PX , the DF FX or the p.m.f. fX . Given any one of these quantities, the
other two can be obtained using the relations described above.
 37

(d) By Definition 4.7 and Definition 4.11, we have that the p.m.f. fX : R → R is a function
such that

fX (x) = 0, ∀x ∈ S c ,
X
fX (x) > 0, ∀x ∈ S, fX (x) = 1.
x∈S

Remark 4.14. Let ∅ =

̸ S ⊂ R be a finite or countably infinite set and let f : R → R be such that

f (x) = 0, ∀x ∈ S c ,
X
f (x) > 0, ∀x ∈ S, f (x) = 1.
x∈S

Then by an argument similar to Proposition 1.44, we conclude that P as defined below is a prob-
ability function/measure on B, where B denotes the power set of R. For all A ⊆ R, consider
X
P(A) := f (x).
x∈A∩S

By an argument similar to Theorem 4.1, we can then show that the function F : R → R defined
by F (x) := P((−∞, x]), ∀x ∈ R is non-decreasing, right continuous with limx→−∞ F (x) = 0 and
limx→∞ F (x) = 1. By Theorem 4.2, this F is the DF of some RV Y , i.e. FY = F and by
construction, Y must be discrete with support S and p.m.f. fY = f .

Example 4.15. Take S to be the set of natural numbers {1, 2, · · · } and consider the function
f : R → R defined by 
1 , if x ∈ S,


2x
f (x) :=
0, if x ∈ S c .


P P∞ 1
Then f takes non-negative values with x∈S f (x) = n=1 2n = 1. Therefore f is the p.m.f. of
some RV X with DF FX given by
X
FX (x) = P(X ≤ x) = fX (t)
t∈(−∞,x]∩S
 
0, if x < 1, 0, if x < 1,

 

= P =
m 1 1 − 1m ,

 n=1 2n , if x ∈ [m, m + 1), m ∈ S. 
2
if x ∈ [m, m + 1), m ∈ S.
38

Definition 4.16 (Continuous RV and its Probability Density Function (p.d.f.)). An RV X is said
to be a continuous RV if there exists an integrable function f : R → [0, ∞) such that
Z x
FX (x) = P(X ≤ x) = f (t) dt, ∀x ∈ R.
−∞

The function f is called the probability density function (p.d.f.) of X.

Remark 4.17. Let X be a continuous RV with DF FX and p.d.f. fX . Then we have the following
observations.
Rx
(a) Since fX is integrable, from the relation FX (x) = −∞ fX (t) dt, ∀x ∈ R, we have FX is
continuous on R. In particular, FX is absolutely continuous. Moreover, for all a < b, we
have Z b Z a Z b
FX (b) − FX (a) = fX (t) dt − fX (t) dt = fX (t) dt.
−∞ −∞ a
(b) Since FX is continuous, we have
(i) FX (x−) = FX (x) = FX (x+), ∀x ∈ R.
(ii) P(X = x) = PX ({x}) = FX (x) − FX (x−) = 0, ∀x ∈ R.
(iii) P(X < x) = FX (x−) = FX (x) = P(X ≤ x), ∀x ∈ R.
(iv) For all a < b,

P(a < X < b) = P(a < X ≤ b) = P(a ≤ X < b) = P(a ≤ X ≤ b)

Z b
= FX (b) − FX (a) = fX (t) dt.
a

(c) If A ⊂ R is finite or countably infinite, then by the finite/countable additivity of PX , we

have
X
P(X ∈ A) = PX (A) = PX ({x}) = 0.
x∈A

(d) By definition, we have fX (x) ≥ 0, ∀x ∈ R and

Z x Z ∞
1 = lim FX (x) = lim fX (t) dt = fX (t) dt.
x→∞ x→∞ −∞ −∞
R∞
Remark 4.18. Let f : R → [0, ∞) be an integrable function with −∞ f (t) dt = 1. Then the
Rx
function F : R → [0, 1] defined by F (x) := −∞ f (t) dt, ∀x ∈ R is non-decreasing and continuous
 39

with limx→−∞ F (x) = 0 and limx→∞ F (x) = 1. By Theorem 4.2, this F is the DF of some RV Y ,
i.e. FY = F and by construction, Y must be continuous with p.d.f. fY = f .

Example 4.19. Let X be an RV with the DF FX : R → R as discussed in Example 4.4. Here,





 0, if x < 0,


FX (x) := x, if 0 ≤ x < 1,




1, if x ≥ 1.



Then the function f : R → [0, ∞) defined by


1, if 0 ≤ x ≤ 1,


f (x) :=
0, otherwise



Rx
is an integrable function with FX (x) = −∞ f (t) dt, ∀x ∈ R. Therefore, X is a continuous RV with
p.d.f. f .

Example 4.20. Consider the DF F : R → [0, 1] considered in Example 4.6 given by





 0, if x < 0,



1 + x, if 0 ≤ x ≤ 1,


4 2
F (x) =
1
+ x4 , if 1 < x < 2,





 2


1, if x ≥ 2.



As discussed earlier, F has a discontinuity at the point 0. Therefore, an RV X with DF F is not

a continuous RV.

Note 4.21. Given a continuous RV X with p.d.f. fX , the DF FX is computed by the formula
Rx
FX (x) = −∞ fX (t) dt, ∀x ∈ R.
40

Example 4.22. Consider a function f : R → R of the form





 αx, if x ∈ [−1, 0),


f (x) = x2
 8
, if x ∈ [0, 2],



0, otherwise



for some α ∈ R. For this f to be a p.d.f. of a continuous RV, two conditions need to be satisfied,
R∞
viz. f (x) ≥ 0, ∀x ∈ R and −∞ f (x) dx = 1.
The first condition is satisfied on (−∞, −1) ∪ [0, ∞). For x ∈ [−1, 0), we must have αx ≥ 0, which
implies α ≤ 0.
R0 R 2 x2
From the second condition, we have −1 αx dx + 0 8 dx = 1. This yields α = − 43 , which satisfies
α ≤ 0.
Therefore, for f to be a p.d.f. we must have α = − 43 .

In what follows, we consider the question of computing fX from the DF FX .

Remark 4.23 (Is the p.d.f. of a continuous RV unique?). Let X be a continuous RV with DF FX
and p.d.f. fX . Fix any finite or countably infinite set A ⊂ R and fix c ≥ 0. Consider the function
g : R → [0, ∞) defined by 
X (x), if x ∈ Ac ,

f

g(x) :=
c, if x ∈ A.


Rx
Then g is integrable and FX (x) = −∞ g(t) dt, ∀x ∈ R. Hence, g is also a p.d.f. for X. Therefore,
the RV X with DF FX is a continuous RV with p.d.f. f (or g). For example,

1, if 0 < x < 1,


g(x) :=
0, otherwise



is a p.d.f. for X as in Example 4.19. More generally, we may also consider


X (x), if x ∈ Ac ,

f

g(x) :=
 cx , if x ∈ A


 41

as a p.d.f., where cx ≥ 0, ∀x ∈ A.

Note 4.24. In fact, a p.d.f. fX for a continuous RV X is determined uniquely on the complement
of sets of ‘length 0’, such as sets which are finite or countably infinite. We do not make a precise
statement – this is beyond the scope of this course. However, we consider the deduction of p.d.f.s
from the DFs.

The next result is stated without proof.

Theorem 4.25. Let X be an RV with DF FX .

R∞
(a) If FX is differentiable on R with −∞ FX′ (t) dt = 1, then X is a continuous RV with p.d.f.
FX′ .
(b) If FX is differentiable everywhere except on a finite or a countably infinite set A ⊂ R with
R∞
−∞ FX′ (t) dt = 1, then X is a continuous RV with p.d.f. f given by

F ′ X (x), if x ∈ Ac ,


f (x) :=
0, if x ∈ A.



Note 4.26. Continuing the discussion from Note 4.21, the DF FX of a continuous RV X may be
used to compute the p.d.f. fX . In Example 4.19, the DF FX is given by




 0, if x < 0,


FX (x) := x, if 0 ≤ x < 1,



1, if x ≥ 1.


It is differentiable everywhere except at the points 0 and 1. Using Theorem 4.25, we have the p.d.f.
given by 
1, if 0 < x < 1,


f (x) := 
0,
 otherwise.
Note 4.27. To study a continuous RV X, we may study any one of the following three quantities,
viz. the law/distribution PX , the DF FX or the p.d.f. fX . Given any one of these quantities, the
other two can be obtained using the relations described above.
42

Remark 4.28 (Identifying discrete/continuous RVs from their DFs). Suppose that the distribution
of an RV X is specified by a given DF FX . In order to check if X is a discrete/continuous RV, we
use the following steps.
(a) Identify the set D = {x ∈ R : FX (x−) < FX (x+)} = {x ∈ R : P(X = x) > 0} of
discontinuities of FX . Recall that D is a finite or a countably infinite set.
(b) If D is empty, then FX is continuous on R. By verifying the hypothesis of Theorem 4.25
or otherwise, check if there exists a p.d.f.. If a p.d.f. exists, then X is a continuous RV.
Otherwise, X is not a continuous RV.
(c) If FX has at least one discontinuity, then FX is not continuous on R and hence X cannot
be a continuous RV. For X to be a discrete RV X, we must have
X X
[FX (x+) − FX (x−)] = P(X = x) = 1.
x∈D x∈D

If the above condition is satisfied, X is a discrete RV. Otherwise, X is not a discrete RV.

Note 4.29. Consider the DF F : R → [0, 1] considered in Example 4.6 given by


0, if x < 0,






1 + x, if 0 ≤ x ≤ 1,


4 2
F (x) =
1
+ x4 , if 1 < x < 2,





 2


1, if x ≥ 2.



As discussed in Example 4.9 and Example 4.20, an RV with DF F is neither discrete nor continuous.