STAT – 835

PROBABILITY AND
STATISTICS

Basics of
Probability III
(mopping up the basics)

Muhammad Ijaz
December 20th, 2016
 Set Theory

Typical 2-set problems involving numbers

A B A B

x x

A x A
B B

For such problems, Morgan’s Law is used.

STAT 835 - Probability and Statistics 2
 Set Theory

De Morgan’s Law for 2-set problems

De Morgan's laws are rules relating the logical operators "and" and "or"
in terms of each other

A B

n(A  B) = n(A) + n(B) – n(A  B)

STAT 835 - Probability and Statistics 3

 Set Theory

Example 1

n(A  B) = 20
3 x 5 Find x

Answer:
n(A  B) = n(A) + n(B) – n(A  B)
20 = (3+x) + (x+5) – x
=> x = 12
STAT 835 - Probability and Statistics 4
 Set Theory

Example 2

P
n(P  Q) = 100
Q
x
n(P) = 70
n(Q) = 43
Find x
Answer:
n(P  Q) = n(P) + n(Q) – n(P  Q)
100 = 70 + 43 – x
=> x = 13

STAT 835 - Probability and Statistics 5

 Set Theory

Example 3
P Q
x 4 7
U = 25
5 n(P  Q)’ = 5
Find x
Answer:
Obviously, n(P  Q) = U – n(P  Q)’ = 20
Then from Morgan’s Law,
n(P  Q) = n(P) + n(Q) – n(P  Q)
=> 20 = (4 + x) + 11 – 4
=> x = 9
STAT 835 - Probability and Statistics 6
 Set Theory

Morgan’s Law for 3-set problems

A B

n(A  B  C) = n(A) + n(B) + n(C)

- n(A  B) – n(A  C) – n(B
 C)
+STAT
n(A  B  C)
835 - Probability and Statistics 7
Points to remember … Set Theory

For Two Events A and B:

Union: “A or B” = A  B [“either A or B or both” implies union]

Intersection: “A and B” = A  B [“and” implies intersection]

Complement: A´ Mutually Exclusive: No outcomes in common

B
A B A
B A B
S S S A S
Venn diagram of A&B AB: intersection of A&B AB: union of A&B A & B mutually exclusive

STAT 835 - Probability and Statistics 8

 Set Theory

APPLICATION OF SET THEORY TO

PROBABILITY
Remember Morgan‟s Law?
U
A B

n(A B)  n(A)  n(B)  n(A

B)
Dividing through the above equation by n(U) yields:

n( A  B) n( A) n(B) n( A  B)
  
n(U ) n(U ) n(U ) n(U )

p(A B)  p(A)  p(B)  p(A

STAT 835 - Probability and Statistics 9
B)
 Set Theory

Application of Set Theory to Probability

(cont‟d)
p(A B)  p(A)  p(B)  p(A B)
i.e.,
p(A happens or B happens or both happen at same
time) = p(A happens) + p(B happens)
- p(both A and B happen at same time)

Note that in the special cases where A and B cannot occur

at the same time, (i.e., for disjoint events), p( A∩ B) = 0

p(A B)  p(A)  p(B)

STAT 835 - Probability and Statistics 10

 Set Theory

Application of Set Theory to Probability(cont‟d)

Similar results can be obtained for probability of

three events (recall the 3-set problem)

U
A B

STAT 835 - Probability and Statistics 11

 Set Theory

Application of Set Theory to Probability(cont‟d)

p(A B  C)  p(A)  p(B)  p(C)

 p(A B)  p(A C)  p(B
 C)

 p(A  B  C)
If A, B and C are disjoint (i.e., they cannot occur simultaneously;
they are mutually exclusive), then

p(A B  C)  p(A)  p(B)  p(C)

STAT 835 - Probability and Statistics 12
 Set Theory

Application of Set Theory to Probability(cont‟d)

U
A n(A)  n(A')  n(U
)
Dividing through by n(U) gives:
n( A) n( A') n(U )
 
n(U ) n(U )
n(U )
p(A)  P( A')  1
The sum of the probabilities of complementary events is always = 1
STAT 835 - Probability and Statistics 13
 Set Theory

So far…
p( A) 
n( A) n(U
)
p(A B)  p(A)  p(B)  p(A B)

p(A B  C)  p(A)  p(B)  p(C)

 p(A B)  p(A C)  p(B 
C)
 p(A  B  C)

p(A)  1 P(A')
STAT 835 - Probability and Statistics 14
Work Examples on Mutually Exclusive Events

(1) Mr. „X‟estimates that the probabilities of his getting grades

A to F are as follows:
A- 0.28, B- 0.41, C- 0.18, D- 0.12, F- 0.01
(a) Find the probability that he gets either A or B.
(b) What is the probability that he does not get A or B?

Answer:
p(A or B) = p(A u B) = p(A) + p(B) – p(A ∩ B)
But A and B are mutually exclusive, so p(A ∩ B) = 0
Therefore
p(A or B) = p(A) + p(B) = 0.28 + 0.41 = 0.69
p(A u B)‟ = 1 – p(A u B) = 1 – 0.69 = 0.31

STAT 835 - Probability and Statistics 15

Work Examples on Mutually Exclusive Events
(2) In the year 2013, it is estimated that the probabilities that Pakistan‟s NHS
will be, on the average, excellent, good and poor condition is as follows:
Excellent- 0.22, Good- 0.38, Poor- 0.40
(a) Find the probability that in 2013 the NHS is in either excellent or good
condition.
(b) What is the probability that in 2013, the NHS will not be in excellent
condition?

Answer:
p(E or G) = p(E u G) = p(E) + p(G) – p(E ∩ G)
But E and G are mutually exclusive, so p(E ∩ G) = 0
Therefore
p(E or G) = p(E) + p(G) = 0.22 + 0.38 = 0.60 p(E‟)
= 1 – p(E) = 1 – 0.22 = 0.78

STAT 835 - Probability and Statistics 16

Work Example on Probability and Set Theory
Example:
Assume the probability of having pizza for lunch is 40%, the probability of having
pizza for dinner is 70%, and the probability of having pizza for lunch and dinner
is 30%. Also, this person always skips breakfast. We can recast this example
using:
P(A)= probability of having pizza for lunch =40%
P(B)= probability of having pizza for dinner =
70% P(AB)=30% (pizza for lunch and dinner)
1) What is the probability that pizza is eaten at least once a
day?
The key words are “at least once”, this means we want the union of
A& B
P(AB)=P(A)+P(B)-P(AB) = 0.7+0.4-0.3 =0.8
2) What is the probability that pizza is not eaten on a given
day?
Not eating pizza (Z´) is the complement of eating pizza (Z) so P(Z)+P(Z´)=1
P(Z´)=1-P(Z)
The non-overlapping =1-0.8
blue area = 0.2
is pizza for lunch, no pizza
for dinner. pizza for lunch
The non-overlapping red area is pizza for dinner, no pizza for

lunch.

pizza for dinner

 In a typical probability question, how do we know when
two events are sequential?

Hint: Look for the key words:

“and”
“as well as”
“then”
Also recognize that intersecting events (overlaps) in a
Venn diagram indicate that two or more events are
sequential.

STAT 835 - Probability and Statistics 18

How to calculate the probability of 2
sequential events
(a) If the 2 events are statistically independent p(A
happens and B happens)
= p(A and B)
= p(A ∩ B)
= p(A) * p(B)

p(A  B)  p(A)* p(B)

STAT 835 - Probability and Statistics 19

How to calculate the probability of 2
sequential events (cont’d)
(b) If the 2 events are statistically dependent
p(A happens, and then B happens given that A has
happened)
= p(A and B)
= p(A ∩ B)
= p(A) * p(B/A)

p(A B)  p(A)* p(B / A)

STAT 835 - Probability and Statistics 20

 General Probability Formula for the following
events

Compound
Simple Events p(A u B) = p(A) + p(B) – p(A ∩ B)

Not applicable

Mutually Exclusive Sequential

p(A u B) = p(A) + p(B) p(A u B) = p(A) + p(B) - p(A ∩ B)

Statistically Statistically
Independent Dependent
p(A ∩ B) = p(A) * p(B) p(A ∩ B) = p(A) * p(B/A)

STAT 835 - Probability and Statistics

 SUMMAR
Y p(S)
Probability Calculation Formula:
= n(S)/n(U)
Application of Set Theory to
Probability:
p(A u B) = p(A) + p(B) – p(A ∩ B)
Mutually Exclusive Events p(A
u B) = p(A) + p(B)
Non-mutually Exclusive Events
p(A u B) = p(A) + p(B) – p(A ∩
B)
where p(A ∩
B) =
p(A)*
p(B)
Statistically Independent Events
p(A ∩ B) = p(A) * p(B)
STAT 835 - Probability and Statistics 22
Statistically Dependent Events p(A
 Mutually Exclusive vs. Independent events
 It is not uncommon for people to confuse the concepts of mutually exclusive events, exhaustive events
and independent events.
 Mutually Exclusive
– Two events are mutually exclusive if they cannot occur at the same time (i.e., they have no
common outcomes). An example is tossing a coin, which can result in either heads or tails, but
not both. Both outcomes cannot happen simultaneously
– If event A happens, then event B cannot, or vice-versa. The two events "it rained on Monday"
and "it did not rain on Monday" are mutually exclusive events.
– When calculating the probabilities for exclusive events you add the probabilities.
– Another term for mutually exclusive is "disjoint."

23
 Mutually Exclusive vs. Independent
 Independent events
– The outcome of event A, has no effect on the outcome of event B. Such as "It rained on
Monday" and "My chair broke at work".
– When calculating the probabilities for independent events you multiply the
probabilities.
– You are effectively saying what is the chance of both events happening bearing in mind
that the two were unrelated.

– So, if A and B are mutually exclusive, they cannot be independent. If A and B are
independent, they cannot be mutually exclusive.

STAT 835 - Probability and Statistics 24

 Conditional Probability
 Conditional probability
– Knowing that an event F occurred might affect the
probability that another event E also occurred
– Reduce the effective sample space from S to F
– Definition (assuming P(F)  0):

 E and F are independent if P(E  F) = P(E) P(F)

– Implies P(E|F) = P(E) and P(F|E) = P(F),
– i.e., knowing that one event occurs tells you nothing about
the other

STAT 835 - Probability and Statistics 25

Example on Conditional Probability
The probability that an engineering system collapses is
0.007. The probability that the systems a collapsed due
to development of a fault is 0.002.

Last Monday the system collapsed. Find the

probability that it developed a fault.

Answer
Let F denote the event that the system developed a fault Let C denote
the event that the system collapses
We seek p(F/C)
P(F/C)= p(C∩ F)/P(C) = 0.002 / 0.007 = 0.286

STAT 835 - Probability and Statistics 26

 Tree Diagrams
• A tree diagram is simply a way of representing a
sequence of events. Tree diagrams are
particularly useful in probability since they
record all possible outcomes in a clear and
uncomplicated manner.
• Probability trees are useful for calculating
combined probabilities. It helps you to map out
the probabilities of many possibilities
graphically, without the use of complicated
probability equations.

- Probability and Statistics 28

 Why Use a probability tree?
Sometimes you don’t know whether to multiply or
add probabilities. A probability tree makes it easier to
figure out when to add and when to multiply. Plus,
seeing a graph of your problem, as opposed to a
bunch of equations and numbers on a sheet of paper,
can help you see the problem more clearly.
 Parts of a probability tree.
A probability tree has two main parts: the branches
and the ends. The probability of each branch is
generally written on the branches, while the outcome
is written on the ends of the branches.
 Multiplication and Addition
Probability Trees make the question of whether to
multiply or add probabilities simple: multiply along
the branches and add probabilities down the columns.
In the following example (courtesy of Yale University
), you can see how adding the far right column adds up
to 1, which is what we would expect the sum total of
all probabilities to be:
.9860+ 0.0040 + 0.0001 + 0.0099 = 1
Explanation
 How to Use a Tree Diagram

Sometimes, you’ll be faced with a probability question

that just doesn’t have a simple solution. Drawing a
probability tree (or tree diagram) is a way for you to
visually see all of the possible choices, and to avoid
making mathematical errors. This how to will show
you the step-by-step process of using a decision tree.
How to Use a Probability Tree: Steps
Sample question: “An airplane manufacturer has
three factories A B and C which produce 50%, 25%,
and 25%, respectively, of a particular airplane.
Seventy percent of the airplanes produced in factory A
are passenger airplanes, 25% of those produced in
factory B are passenger airplanes, and 25% of the
airplanes produced in factory C are passenger
airplanes. If an airplane produced by the manufacturer
is selected at random, calculate the probability the
airplane will be a passenger plane.”
 Steps
Step 1:Draw lines to represent the first set of options
in the question (in our case, 3 factories). Label them
(our question list A B and C so that is what we’ll use
here).

Step 2: Convert the percentages to decimals, and place

those on the appropriate branch in the diagram. For our
example, 50% = 0.5, and 25% = 0.25.
Step 3: Draw the next set of branches. In our case, we
were told that 70% of factory A’s output was
passenger. Converting to decimals, we have 0.7 P (“P”
is just my own shorthand here for “Passenger”) and
0.3 NP (“NP” = “Not Passenger”).
Step 4:Repeat step 3 for as many branches as you
are given.
Step 5: Multiply the probabilities of the first branch
that produces the desired result together. In our
case, we want to know about the production of
passenger places, so we choose the first branch that
leads to P.
Step 6: Multiply the remaining branches that
produce the desired result. In our example there are
two more branches that can lead to P.
Step 6: Add up all of the probabilities you
calculated in steps 5 and 6. In our example, we had:
.35 + .0625 + .0625 = .475
That’s it!
 Tree Diagram Example
 Mr. „X‟ „s Closet
3 Shirts

2 Pants

2 Pairs of Shoes

STAT 835 - Probability and Statistics 41

 List all of Mr. X’s outfits
1

42
 List all of Mr. X’s outfits
1

2
3

43
 List all of Mr. X’s outfits
1

2
3

44
 List all of Mr. X’s
outfits 1

2
3

6
7

45
 1

2
3

6
7

9
10

46
 1

2
3

6
7

9
10

11
36
 Basics of Probability III

PERMUTATIONS AND COMBINATIONS

In some cases involving engineering systems and

operations, we need to know:
- the total number of possibilities through
which an event could happen, or

- the total number of ways system

components can be arranged, or

- the total number of ways some

system
operations or events can be scheduled

In all these cases, what we are actually finding is the total

number of elements in the sample space, and we generally
apply the Multiplication Rule
STAT 835 - Probability and Statistics 48
Permutations and Combinations(cont‟d)
Basics of Probability III

In this regard, there are 2 kinds of problems:

- Finding the number of ways a given

number of objects (or events) can be
arranged or can occur (Permutation)

- Finding the total number of ways a given

number of objects can be arranged or can
occur regardless of the order in which they
occur (Combination)

STAT 835 - Probability and Statistics 49

Permutations and Combinations(cont‟d)
Basics of Probability III

Permutation
To find the total number of ways an event can
occur, we apply the Multiplication Rule.

Example:
At 1:00 pm tomorrow, I wish to have a sandwich
for Lunch at a fast food. I can go to BK,
McDonalds, Subway or Wendy's. At each of these
joints there are three different kinds of sandwiches:
chicken, beef, and veggie.
What is the total number of choices available to me?

STAT 835 - Probability and Statistics 50

Permutations and Combinations(cont‟d)
Answer:
I can eat at 4 different joints
At each joint, I can have 3 choices
Therefore, there are 12 choices available to me (i.e., 4*3)

Note: The tree for this problem is as follows:

Chicken
Beef
veggie
BK
chicken

Beef
MD veggie
chicken
SW Beef

veggie
WY chicken
Beef
veggie

1st Event 2nd Event 51

Permutations and Combinations(cont‟d)
Basics of Probability III

MULTIPLICATION RULE

If an event can occur in n1 ways, and

for each of these ways, a 2nd event can occur in
n2 ways,
Then the two events can occur in n1* n2
ways together

i.e., the total number of possibilities = n1* n2

STAT 835 - Probability and Statistics 52

Permutations and Combinations(cont‟d)
Basics of Probability III

What if there are several kinds of cheese for

each type of sandwich at each of these joints?
e.g., a chicken could come with either cheddar
cheese, spiced cheese, or Italian cheese.

Also, there could be many kinds of cheddar

cheese, many kinds of spiced cheese, and
many kinds of Italian cheese, etc.

STAT 835 - Probability and Statistics 53

Permutations and Combinations(cont‟d)
Basics of Probability III

In cases involving a very large

number of events, we use the
General Multiplication Rule

1st Event 2nd Event 3rd Event kth Event

44
n1 = 4 n2 = 3 n3 = 3
Permutations and Combinations(cont‟d)
Basics of Probability III

The General Multiplication Rule

If an event can occur in n1 ways, and for each of

these a 2nd event can occur in n2 ways, and for
each of the 2nd event, a 3rd event can occur in n3
ways, etc…
Then the total number of possibilities is :
n1 * n2 * n3 *…* nk

Where k is the number of events

STAT 835 - Probability and Statistics 55

Permutations and Combinations(cont‟d)
Basics of Probability III

So, what exactly is permutation?

Is an arrangement of all or part of a set of

objects (events) taking a certain number of the
objects at a time

Makes use of the general multiplication

rule

STAT 835 - Probability and Statistics 56

 Basics of Probability III

Going back to the “Fast Food Breakfast” example

…

If there are 4 kinds of cheese for each chicken,

beef, or veggie sandwich, what are the
total number of ways in which I could have
breakfast that morning?

Total nr. of possibilities = n1 * n2 * n3

= 4 * 3 * 4 = 48
(i.e., 4 kinds of restaurants, 3 kinds of “meat” and 4 kinds of
cheese.)
STAT 835 - Probability and Statistics 57
 Permutations and Combinations(cont‟d)

Example 2
In how many ways can the letters a, b, and c
be arranged?
Answer:
ab c c
b
ac b c b
b ac a
a c
b ca b c
ca b a
c a
cba b
b a

n1 = 3 n2 = 2 n3 = 1
STAT 835 - Probability and Statistics 58
Permutations and Combinations(cont‟d)
Basics of Probability III

It can be seen from the list and tree

diagram that a, b, and c can be arranged
in 3*2*1 = 6 ways
Therefore,
3 distinct objects can be arranged in 3*2*1 ways
Similarly,
4 distinct objects can be arranged in
4*3*2*1 ways
The number of permutations of n distinct
objects, taking one at a time, is n!

STAT 835 - Probability and Statistics 59

Permutations and Combinations(cont‟d)
Basics of Probability III

Example 3
In how many ways can the letters a,
b, c and d be arranged?

Answer
n=4
Number of permutations = n! = 4! =
4*3*2*1 = 24 ways.

What if we took 2 letters at a time?

STAT 835 - Probability and Statistics 60
Permutations and Combinations(cont‟d)
Basics of Probability III

For example, in how many ways can we arrange

the letters a,b,c, d, taking two letters at a time
Answer
ab, ac, ad, ba, bc, bd, ca, cb, cd, da, db,

dc b
c
a d
a
c
b d
c a
b
d
d
a
b
c
n1 = 4 n2 = 3 51
STAT 835 - Probability and Statistics
Permutations and Combinations(cont‟d)
Basics of Probability III

Using the Multiplication Rule, it is seen that a,b,c

and d, taking 2 letters at a time, can
be arranged in 4*3 = 12 ways
i.e., 4 objects, taking 2 at a time, can be
arranged in 4*3 ways

It can be shown that generally, the number of

permutations of n objects, taking r at a time,
is as follows:
n
P
(n !
STAT 835 r)!
- Probability and Statistics 62
 Permutations
 Sometimes we are concerned with how
many ways a group of objects
can be _a_r_r_a_n_gd__.
e

•How many ways to arrange books on a shelf

•How many ways a group of people can stand in line

•How many ways to scramble a word‟s letters

STAT 835 - Probability and Statistics 63