Lecture 03

Lecture 3: Conditional Probability and Bayes’
Theorem
MSU-STT-351-sum-19B
(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 1 / 36

Contents
In this lecture, we will discuss:
1 Conditional Probability
2 Bayes’ Theorem
3 Independent Events
4 Parallel and Series Systems
5 Exercise
6 Homework

Conditional Probability
Conditional Probability: Let A , B be two events and assume P (B ) > 0.

The conditional probability of A given B (that is, B has occurred), denoted
by P (A |B ), is defined as
P (A ∩ B )
P (A |B ) = P (A and B )/P (B ) = .
P (B )
The above realtion leads to:
P (A and B ) = P (A ∩ B ) = P (A |B )P (B );
P (A and B ) = P (A ∩ B ) = P (B |A )P (A ).
If A and B are disjoint, then P (A |B ) = 0.

Example 1 (Ex. 58): Show that for events A , B and C with P (C ) > 0,
P (A ∪ B |C ) = P (A |C ) + P (B |C ) − P (A ∩ B |C ).
Proof: Note

P (A ∪ B ) ∩ C P (A ∩ C ) ∪ (B ∩ C )
P (A ∪ B |C ) = =
P (C ) P (C )
P (A ∩ C ) + P (B ∩ C ) − P (A ∩ B ∩ C )
=
P (C )
=P (A |C ) + P (B |C ) − P (A ∩ B |C ).
Note if A and B are disjoint, then
P (A ∪ B |C ) = P (A |C ) + P (B |C ).

It can be seen that
P (A c |B ) =1 − P (A |B )
P (A |B )+P (A c |B ) = 1
P (B |B ) =1.
Also, the following facts holds in general:
P (A |B c ) , 1 − P (A |B );
P (A c |B c ) , 1 − P (A |B );
P (A |B ) + P (A c |B c ) , 1.

Example 2: A box contains 4 red and 2 green balls. Draw successively

two balls without replacement and observe the color.
Define the following events: G1 = green on the first draw, G2 = green on

the second draw, R1 = red on the first draw, R2 = red on the second draw.
For this experiment, the sample space S is
S = {G1 G2 , G1 R2 , R1 G2 , R1 R2 }
First, we compute the probabilities of simple events. Note it involves

conditional probabilities.

The probabilities of the simple events G1 G2 , G1 R2 , R1 G2 and R1 R2 are:

2 1 2
(i) P (G1 G2 ) = P (G1 )P (G2 |G1 ) = × =
6 5 30
4 3 12
(ii) P (R1 R2 ) = P (R1 )P (R2 |R1 ) = × =
6 5 30
2 4 8
(iii) P (G1 R2 ) = P (G1 )P (R2 |G1 ) = × =
6 5 30
4 2 8
(iv) P (R1 G2 ) = P (R1 )P (G2 |R1 ) = × =
6 5 30
The probability that the second ball is green is
8 2 10
P (G2 ) = P (R1 G2 ) + P (G1 G2 ) = + = .
30 30 30

The tree diagram is

G1 G2
G1
G1 R2
R 1 G2
R1
R1 R2

Suppose that green ball was observed in the second draw. What is the
conditional probability that the first ball was also green?
It is given by
2
P (G1 G2 ) 30
P ( G1 | G2 ) = = 10
= 0.2.
P ( G2 ) 30
Exercise 1. Find also
(a): the probability that exactly one ball selected is green.
(b): the probability that at least one ball selected is green.
(Do it yourself!).

Example 3: Given is a contingency table of 100 students cross-classified

by their school goal and gender.
Goals
Grades Popular Sports Total
Boy 24 10 13 47
Gender Girl 27 19 7 53
Total 51 29 20 100
A student is selected at random. Let G = {a girl is selected} and

S = {wants to excel at sports}.
(i ) Find P (G ) and P(S).

Note
P (G ) = (53/100) = 0.53; P (S ) = (20/100) = 0.2.

(ii ) Find the probability that “a girl is selected and she wants to excel at
sports.”
It is given by
P (G ∩ S ) = (7/100) = 0.07
(iii ) Find the probability that “a student wants to excel at sports, given that
a girl is selected.”
It is given by
P (GS ) (7/100) 7
P ( S |G ) = = = .
P (G ) (53/100) 53

Multiplication Rule:
From
P (A ∩ B )
P ( A |B ) = ,
P (B )
we get
P (A ∩ B ) = P (A |B )P (B )
= P (B |A )P (A ).

Exercise 2. In a lot of 10 elements, there are three defectives. Two

elements are selected at random and without replacement, one after
another.
(i ) What is the probability that the first is good, and the second is
defective?
(ii ) What is the probability that the second is defective?
(iii ) What is the probability that the second is defective given the first one
is good?
(iv ) What is the probability that the first one is good given the second is
defective?

The Law of Total Probability:
Let A1 , . . . , Ak be mutually exclusive (Ai ∩ Aj = φ for i , j ) and exhaustive

(∪ki=1 Ai = S ) events. Then for any other event B,
P (B ) =P (B |A1 )P (A1 ) + . . . + P (B |Ak )P (Ak )

k
X
= P (B |Ai )P (Ai )
i =1
Note the event B is a subset of sample space S.

Proof:
Since Ai ’s are mutually exclusive and exhaustive, if B occurs it must be in
conjunction with exactly one of the Ai ’s. That is,
B = (B ∩ A1 ) ∪ ... ∪ (B ∩ Ak ),
where the events (B ∩ Ai ) are mutually exclusive (why?). Thus,
k
X k
X
P (B ) = P (B ∩ Ai ) = P (B |Ai )P (Ai ),
i =1 i =1
using the multiplication rule.

We next discuss the Bayes formula which is very useful to compute
certain conditional probabilities. Suppose A and B are any two events with
P (B ) > 0. Given that P (A ), P (B |A ), P (B |A c ), how to find P (A |B )?
Solution: Note first that
P (A ∩ B ) P ( B |A ) P ( A )
P (A |B ) = = .
P (B ) P (B )
But P (B ) is not given. However,
P (B ) =P (B ∩ A ) + P (B ∩ A c )
=P (B |A )P (A ) + P (B |A c )P (A c )
Hence, the Bayes formula for two events is
P (B |A )P (A )
P (A |B ) = .
P (B |A )P (A ) + P (B |A c )P (A c )

Bayes’ Theorem
In the above result, P (A ) is called prior probability and P (A |B ) is called

the posterior probability of A , given that B has occurred. The above result
can be extended to k events.
Bayes’ Theorem
Let A1 , . . . , Ak be a collection of k mutually exclusive and exhaustive

events with prior probabilities P (Ai ), i = 1, . . . , k . Let B be any event with
P (B ) > 0. Then the posterior probability of Ai given that B has occurred is
T
P (Ai B )
P (Ai |B ) =
P (B )
P (B |Ai )P (Ai )
= Pk ,
i =1 P (B |Ai )P (Ai )
for i = 1, . . . , k .

Bayes’ Theorem
Example 3. Suppose a box contains 4 red and 6 green balls. Two balls
are drawn at random (and without replacement), one after another. Then
the tree diagram for the outcomes, with associated probabilities, is
12
3
R1 R2 90
9
R1
4
6 24
10
9
R1 G2 90
24
G1 R 2 90
4
9
6
10
G1
5 30
9 G1 G2 90

Bayes’ Theorem
In the tree diagram, Ri (Gi ) denotes the event of having a red (green) ball
at the i-th draw, i = 1, 2. Note the second set of branches denotes the set
S of all possible outcomes of the experiment. That is, given by
S = {R1 R2 , R1 G2 , G1 R2 , G1 G2 }.
Note, for example,
P (G1 R2 ) = P (G1 )P (R2 |G1 )

6 4
= ×
10 9
4
= .
15

Bayes’ Theorem
Similarly, we find
12 24 30
P (R1 R2 ) = ; P (R1 G2 ) = ; P ( G1 G2 ) = .
90 90 90
Note: The sum of the probabilities of all the branches (at the end) add up
to 1.
Also, P (R2 ) can be found as follows:
P (R2 ) = P (R1 R2 ) + P (G1 R2 )

12 + 24 36 2
= = = .
90 90 5

Bayes’ Theorem
Reverse conditioning: How to find P (R1 |R2 )? This is called reverse in

conditioning.
Note this probability is not shown in the tree, but can be computed as
follows.
Note as seen earlier,

2 12 2
P (R2 ) = ; P (R1 R2 ) = = .
5 90 15
Hence,
P (R1 R2 ) 2×5 1
P (R1 |R2 ) = = = .
P (R2 ) 15 × 2 3

Independent Events
We now give a formal definition of independence of two events.
Definition 1 (Independence)
Two events A and B are said to be independent if P (B |A ) = P (B ) or
P (AB ) = P (A )P (B ).
Two events A and B are dependent if P (B |A ) , P (B ) or
P (AB ) , P (A )P (B ).
Example 5
Let P (A ) = 0.4 and P (B ) = 0.5. If A and B are independent, find P (AB ).
Solution: Note P (AB ) = P (A )P (B ) = 0.4 × 0.5 = 0.2.

Independent Events
Example 6
Given P (A ) = 0.6, P (B ) = 0.5, P (A ∪ B ) = 0.8, are A and B
independent?
Solution: Note
P (AB ) = P (A ) + P (B ) − P (A ∪ B )
= (0.6 + 0.5 − 0.8)
= 0.30
= P (A )P (B ).
Hence, A and B are independent.

Bayes’ Theorem
Example 7 (Benefits): A survey shows 56% of all American workers have

a workplace retirement plan, 68% have health insurance, and 49% have
both benefits. We select a worker at random.
(a ) What is the probability he has neither health insurance nor a
retirement plan?
(b ) What is the probability he has health insurance, if he has retirement
plan?
(c ) Are having health insurance and a retirement plan independent
events? Are these two benefits mutually exclusive?

Bayes’ Theorem
Solution:
Let R = {retirement plan}; H = {health insurance}. Given
P (R ) = 0.56; P (H ) = 0.68; P (RH ) = 0.49.
(a) By Demorgan’s rule
P (R c ∩ H c ) = P (R ∪ H )c
= 1 − P (R ∪ H ).
Now,
P (R ∪ H ) = P (R ) + P (H ) − P (R ∩ H )
= 0.56 + 0.68 − 0.49 = 0.75.
Therefore,
P (R c ∩ H c ) = 1 − (0.75) = 0.25.
Bayes’ Theorem
(b) Also,
P (H ∩ R )
P (H |R ) =
P (R )
0.49 7
= = .
0.56 8
7
(c) Since P (H |R ) = , P (H ) = 0.68, the events H and R are not
8
independent.
(d) Are H and R mutually exclusive?
Since P (HR ) = 0.49 > 0, the events H and R are not mutually exclusive.

Independent Events
Example 8 (Ex 71): An oil exploration company currently has two active
projects, one in Asia and the other in Europe. Let A be the event that the
Asian project is successful and B be the event that the European project is
successful. Suppose that A and B are independent events with
P (A ) = 0.4 and P (B ) = 0.7.
(a) If the Asian project is not successful, what is the probability that the
European project is also not successful? Explain your reasoning.
(b) What is the probability that at least one of the two projects will be
successful?
(c) Given that at least one of the two projects is successful, what is the
probability that only the Asian project is successful?

Independent Events
Solution:
(a) Since the events A and B are independent, then A c and B c are also
independent (Prove this fact!).
Hence,
P (B c |A c ) = P (B c ) = 1 − 0.7 = 0.30.
(b) P (A ∪ B ) = P (A ) + P (B ) − P (A )P (B ) = 0.4 + 0.7 − (0.4 × 0.7) = 0.82.
P (AB c ∩ (A ∪ B )) P (AB c ) 0.12

(c) P (AB c |A ∪ B ) = = = = 0.146.
P (A ∪ B ) P (A ∪ B ) 0.82

Independent Events
Exercise 4. Items coming off a production line are categorized as good

(G), slightly blemished (B), and defective (D), and the percentages are
80%, 15% for good and for slightly blemished, respectively. Suppose that
two items will be selected randomly for inspection and the selections are
independent.
(a) Find the probability that at least one of the items is slightly blemished.
(b) Find the probability that neither of the items is good.
(c) Suppose that one of the selected items is good, what is the
(conditional) probability that both of them are good?
(d) Now suppose that 3 items are selected. What is the probability that at
least one is not good?

Parallel and Series Systems
Parallel and Series Systems: Suppose a system consists of three
components, say, C1 , C2 and C3 , which function independently and
correctly with probabilities pi = P (Ci = 1), for 1 ≤ i ≤ 3 (probability that a
component works is often referred to as the reliability of the component).
Series systems: A series system n components functions if all the
components function. Hence,the probability that the series system of n
components functions is p1 p2 . . . pn .
A series system of n independent components fails if at least one of the
components fails. Let Fi = {Ci = 0} denote the failure of the i-th
component so that P (Fi ) = qi = 1 − pi , i = 1, 2, . . . , n. Then the probability
that the series system fails is
P (F1 ∪ F2 ∪ . . . Fn ) = 1 − P (F1 ∪ F2 ∪ . . . ∪ Fn )c
= 1 − P (F1c F2c . . . Fnc )
= 1 − p1 p2 . . . pn ,
since the components are independent. This is expected.
Parallel Systems:
A parallel system PS3 of three components functions if at least one of the
components functions. Let pi = P (Ci = 1) that component Ci works.
Then P (PS3 = 1) = 1 − P (none of the components functions) =
1 − (1 − p1 )(1 − p2 )(1 − p3 ).
Some Complex Systems
C1 C2 C3 C1 C2 C3
C4 C5 C6 C4 C5 C6
(a ) Series parallel system (b ) Parallel series sytem

Example 9 (Exercise 80) Consider the system of components connected

as in the figure given below. Components C1 and C2 are connected in
parallel, so that subsystem S1 works if either C1 or C2 works; since C3 and
C4 are connected in series, that subsystem S2 works if both C3 and C4
work. If components work independently of one another, with
P (component works) = 0.9, calculate P (system works).
C1
C2
C3 C4

Solution:
Let S1 = parallel subsystem of components C1 and C2 ; S2 = series
subsystems of C3 and C4 . Then, S = S1 ∪ S2 (since S1 and S2 are in
parallel). Also, given P (Ci = 1) = 0.9, for 1 ≤ i ≤ 4.
Therefore,
P (system works) = P (S ) = P (S1 ∪ S2 )

= P (S1 ) + P (S2 ) − P (S1 )P (S2 )
= (0.9 + 0.9 − 0.81) + (0.9 × 0.9)
− (0.9 + 0.9 − 0.81)(0.9)(0.9)
= 0.99 + 0.81 − 0.8019 = 0.9981

Exercise
Exercise 5 Consider a system of components connected as shown below.

If all components work independently, and the probability that a given
component works correctly is 0.9 for each, what is the probability that the
entire system works correctly?
C3 C4
C2
C1

Exercise
Exercise 6. On each of its two wings a plane has 2 engines. We assume

that the engines operate independently and P (engine fails) = p = 0.2. A
plane will not crash if at least one engine operates on each wing.
(a) What is the probability that it will not crash?
(b) How many engines should be installed on each wing to have the
probability of not crashing at least 0.99?
(c) The plane has not crashed. What is the chance that all four engines
are in a good shape?

Homework
Sect 2.4: 45, 51, 55, 60, 67, 69
Sect 2.5: 71, 73, 81, 89
Supplementary: 93, 98, 99, 104, 114.

Lecture 03

Uploaded by

Lecture 03

Uploaded by

Lecture 3: Conditional Probability and Bayes’

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 1 / 36

In this lecture, we will discuss:

4 Parallel and Series Systems

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 2 / 36

Conditional Probability: Let A , B be two events and assume P (B ) > 0.

The above realtion leads to:

If A and B are disjoint, then P (A |B ) = 0.

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 3 / 36

Note if A and B are disjoint, then

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 4 / 36

It can be seen that

Also, the following facts holds in general:

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 5 / 36

Example 2: A box contains 4 red and 2 green balls. Draw successively

Define the following events: G1 = green on the first draw, G2 = green on

For this experiment, the sample space S is

First, we compute the probabilities of simple events. Note it involves

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 6 / 36

The probabilities of the simple events G1 G2 , G1 R2 , R1 G2 and R1 R2 are:

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 7 / 36

The tree diagram is

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 8 / 36

Exercise 1. Find also

(a): the probability that exactly one ball selected is green.

(b): the probability that at least one ball selected is green.

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 9 / 36

Example 3: Given is a contingency table of 100 students cross-classified

A student is selected at random. Let G = {a girl is selected} and

(i ) Find P (G ) and P(S).

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 10 / 36

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 11 / 36

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 12 / 36

Exercise 2. In a lot of 10 elements, there are three defectives. Two

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 13 / 36

The Law of Total Probability:

Let A1 , . . . , Ak be mutually exclusive (Ai ∩ Aj = φ for i , j ) and exhaustive

P (B ) =P (B |A1 )P (A1 ) + . . . + P (B |Ak )P (Ak )

Note the event B is a subset of sample space S.

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 14 / 36

where the events (B ∩ Ai ) are mutually exclusive (why?). Thus,

using the multiplication rule.

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 15 / 36

Hence, the Bayes formula for two events is

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 16 / 36

In the above result, P (A ) is called prior probability and P (A |B ) is called

Let A1 , . . . , Ak be a collection of k mutually exclusive and exhaustive

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 17 / 36

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 18 / 36

Note, for example,

P (G1 R2 ) = P (G1 )P (R2 |G1 )

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 19 / 36

Also, P (R2 ) can be found as follows:

P (R2 ) = P (R1 R2 ) + P (G1 R2 )

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 20 / 36

Reverse conditioning: How to find P (R1 |R2 )? This is called reverse in

Note as seen earlier,

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 21 / 36

We now give a formal definition of independence of two events.

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 22 / 36

Hence, A and B are independent.

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 23 / 36

Example 7 (Benefits): A survey shows 56% of all American workers have

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 24 / 36

P (R ) = 0.56; P (H ) = 0.68; P (RH ) = 0.49.

(a) By Demorgan’s rule

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 26 / 36

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 27 / 36

(b) P (A ∪ B ) = P (A ) + P (B ) − P (A )P (B ) = 0.4 + 0.7 − (0.4 × 0.7) = 0.82.

P (AB c ∩ (A ∪ B )) P (AB c ) 0.12

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 28 / 36

Exercise 4. Items coming off a production line are categorized as good

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 29 / 36

Some Complex Systems

(a ) Series parallel system (b ) Parallel series sytem

(P. Vellaisamy: MSU-STT-351-Sum-19B) Probability & Statistics for Engineers 31 / 36