Module 6 – Probability

by: Mr. Arnel A. Cueto

MSCS-Candidate, BSCS, Certified in Professional Teaching
IT Manager, IT Consultant, IT Auditor, Professor
Six Sigma Certified(Yellow, Green and Black Belt)
Management Certifications(Project Management, Information System
Auditing, Controls and Assurance)
Programming NCIV and CHS NCII Assessor
 Phenomena

Non-deterministic
Deterministic
Deterministic Phenomena
• There exists a mathematical model that allows “perfect”
prediction the phenomena’s outcome.
• Many examples exist in Physics, Chemistry (the exact
sciences).
Non-deterministic Phenomena
• No mathematical model exists that allows “perfect”
prediction the phenomena’s outcome.
Non-deterministic Phenomena
• may be divided into two groups.
1. Random phenomena
– Unable to predict the outcomes, but in the long-
run, the outcomes exhibit statistical regularity.

2. Haphazard phenomena
– unpredictable outcomes, but no long-run,
exhibition of statistical regularity in the
outcomes.
 Phenomena

Non-deterministic

Deterministic
Haphazard

Random
Haphazard phenomena
– unpredictable outcomes, but no long-run,
exhibition of statistical regularity in the
outcomes.
– Do such phenomena exist?
– Will any non-deterministic phenomena exhibit
long-run statistical regularity eventually?
Random phenomena
– Unable to predict the outcomes, but in the long-
run, the outcomes exhibit statistical regularity.

Examples
1. Tossing a coin – outcomes S ={Head, Tail}
Unable to predict on each toss whether is Head or
Tail.
In the long run can predict that 50% of the time
heads will occur and 50% of the time tails will occur
2. Rolling a die – outcomes
S ={ , , , , , }

Unable to predict outcome but in the long run can

one can determine that each outcome will occur 1/6
of the time.
Use symmetry. Each side is the same. One side
should not occur more frequently than another side
in the long run. If the die is not balanced this may
not be true.
Definitions
 PROBABILITY

• Probability can be conceptualized as finding the chance of occurrence of

an event. ... Mathematically, it is the study of random processes and their
outcomes.
 Four concepts basic to probability
• Experiment: An action (e.g., rolling a die) that produces an
observable result.
• Outcome: A possible result of the experiment. For instance, if we
roll a die, then the outcome is the number that comes up.
• Sample Space: The set, S, of all possible outcomes of some
experiment. For instance if we roll a die, then S={1,2,3,4,5,6}.
• Event: A subset of the sample space, usually described in words
that have a meaning to us, though this is not necessary. For
instance in the case above we could consider the event {2,4,6},
which we can describe as “we roll an even number.” On the other
hand {1,2,5} is a perfectly good event, even though it seems to
have no simple description.
 Four concepts basic to probability

• Formally a probability measure P on a sample space S is a

function that assigns to every event in S a number (probability)
that satisfies several conditions, the first of which is that every
such probability must be between 0 and 1. We think of events
with probabilities near 0 of being unlikely to happen and with
probabilities near 1 of being likely to happen.
 An Introduction to Discrete Probability
• Probability theory dates back to 1526 when the Italian mathematician, physician, and gambler
Girolamo Cardano wrote the first known systematic treatment of the subject in his book Liber
de Ludo Aleae (Book on Games of Chance).
• (This book was not published until 1663, which may have held back the development of
probability theory.)
• In the seventeenth century the French mathematician Blaise Pascal determined the odds of
winning some popular bets based on the outcome when a pair of dice is repeatedly rolled.
• In the eighteenth century, the French mathematician Laplace, who also studied gambling,
defined the probability of an event as the number of successful outcomes divided by the
number of possible outcomes.
• For instance, the probability that a die comes up an odd number when it is rolled is the number
of successful outcomes—namely, the number of ways it can come up odd—divided by the
number of possible outcomes—namely, the number of different ways the die can come up.
There are a total of six possible outcomes—namely, 1, 2, 3, 4, 5, and 6—and exactly three of
these are successful outcomes—namely, 1, 3, and 5. Hence, the probability that the die comes
up an odd number is 3/6 = 1/2. (Note that it has been assumed that all possible outcomes are
equally likely, or, in other words, that the die is fair.
 The sample Space, S
The sample space, S, for a random phenomena is the set of all
possible outcomes.
Examples
1. Tossing a coin – outcomes S ={Head, Tail}

2. Rolling a die – outcomes

S ={ , , , , , }

={1, 2, 3, 4, 5, 6}
 An Event , E
The event, E, is any subset of the sample space, S. i.e. any set
of outcomes (not necessarily all outcomes) of the random
phenomena
Venn
S diagram
E
The event, E, is said to have occurred if after the outcome has
been observed the outcome lies in E.

S
E
Examples

1. Rolling a die – outcomes

S ={ , , , , , }
={1, 2, 3, 4, 5, 6}

E = the event that an even number is

rolled
= {2, 4, 6}
={ , , }
 Special Events
The Null Event, The empty event - f

f = { } = the event that contains no outcomes

The Entire Event, The Sample Space - S
S = the event that contains all outcomes
The empty event, f , never occurs.
The entire event, S, always occurs.
 Set operations on Events
Union

Let A and B be two events, then the union of A

and B is the event (denoted by AB) defined by:
A  B = {e| e belongs to A or e belongs to B}

AB

A B
The event A  B occurs if the event A occurs or
the event and B occurs .

AB

A B
Intersection

Let A and B be two events, then the intersection

of A and B is the event (denoted by AB) defined
by:
A  B = {e| e belongs to A and e belongs to B}

AB

A B
The event A  B occurs if the event A occurs and
the event and B occurs .

AB

A B
Complement

Let A be any event, then the complement of A

(denoted by A ) defined by:

A = {e| e does not belongs to A}

A
A
The event A occurs if the event A does not
occur

A
A
In problems you will recognize that you are working with:

1. Union if you see the word or,

2. Intersection if you see the word and,
3. Complement if you see the word not.
 Two events A and B are called mutually
exclusive if:
Definition: mutually exclusive

A B 

A B
If two events A and B are are mutually
exclusive then:
1. They have no outcomes in common.
They can’t occur at the same time. The outcome
of the random experiment can not belong to both
A and B.

A B
 Finite Probability
• An experiment is a procedure that yields one of a given set of possible
outcomes. The sample space of the experiment is the set of possible
outcomes. An event is a subset of the sample space. Laplace’s definition of
the probability of an event with finitely many possible outcomes will now be
stated.
 Example 1: Balls-and-Urn
• An urn contains four blue balls and
five red balls. What is the probability
that a ball chosen at random from
the urn is blue?
• Solution: To calculate the
probability, note that there are nine
possible outcomes, and four of
b1
these possible outcomes produce a b2
b7 b9
blue ball. Hence, the probability b3 b5
b8
that a blue ball is chosen is 4/9. b4 b6
• Solution: There are a total of 36 equally likely possible outcomes when two
dice are rolled. (The product rule can be used to see this; because each die
has six possible outcomes, the total number of outcomes when two dice are
rolled is 62 = 36.) There are six successful outcomes, namely, (1, 6), (2, 5), (3,
4), (4, 3), (5, 2), and (6, 1), where the values of the first and second dice are
represented by an ordered pair. Hence, the probability that a seven comes
up when two fair dice are rolled is 6/36 = 1/6.
• EXAMPLE 3: In a lottery, players win a large prize when they pick four digits
that match, in the correct order, four digits selected by a random
mechanical process. A smaller prize is won if only three digits are matched.
What is the probability that a player wins the large prize? What is the
probability that a player wins the small prize?
Solution: There is only one way to choose all four digits correctly. By the product rule, there are 10 4 =
10,000 ways to choose four digits. Hence, the probability that a player wins the large prize is 1/10,000 =
0.0001. Players win the smaller prize when they correctly choose exactly three of the four digits.
Exactly one digit must be wrong to get three digits correct, but not all four correct. By the sum rule, to
find the number of ways to choose exactly three digits correctly, we add the number of ways to choose
four digits matching the digits picked in all but the ith position, for i = 1, 2, 3, 4. To count the number of
successes with the first digit incorrect, note that there are nine possible choices for the first digit (all
but the one correct digit), and one choice for each of the other digits, namely, the correct digits for
these slots. Hence, there are nine ways to choose four digits where the first digit is incorrect, but the
last three are correct. Similarly, there are nine ways to choose four digits where the second digit is
incorrect, nine with the third digit incorrect, and nine with the fourth digit incorrect. Hence, there is a
total of 36 ways to choose four digits with exactly three of the four digits correct. Thus, the probability
that a player wins the smaller prize is 36/10,000 = 9/2500 = 0.0036.
Probabilities of Complements and Unions of
Events
 Probabilistic Reasoning

Common problem is determining which of two events is more likely. Analyzing

the probabilities of such events can be tricky. The next example describes a
problem of this type. It discusses a famous problem originating with the
television game show Let’s Make a Deal and named after the host of the show,
Monty Hall.
 Example
The Monty Hall Three-Door Puzzle Suppose you are a game show contestant.
You have a chance to win a large prize. You are asked to select one of three
doors to open; the large prize is behind one of the three doors and the other
two doors are losers. Once you select a door, the game show host, who knows
what is behind each door, does the following. First, whether or not you
selected the winning door, he opens one of the other two doors that he knows
is a losing door (selecting at random if both are losing doors). Then he asks you
whether you would like to switch doors. Which strategy should you use?
Should you change doors or keep your original selection, or does it not
matter?
 Solution
The probability you select the correct door (before the host opens a door and asks
you whether you want to change) is 1/3, because the three doors are equally likely to
be the correct door.
The probability this is the correct door does not change once the game show host
opens one of the other doors, because he will always open a door that the prize is
not behind. The probability that you selected incorrectly is the probability the prize
is behind one of the two doors you did not select.
Consequently, the probability that you selected incorrectly is 2/3. If you selected
incorrectly, when the game show host opens a door to show you that the prize is not
behind it, the prize is behind the other door. You will always win if your initial choice
was incorrect and you change doors. So, by changing doors, the probability you win
is 2/3. In other words, you should always change doors when given the chance to do
so by the game show host. This doubles the probability that you will win.
 Conditional Probability
• Suppose that we flip a coin three times, and all eight possibilities are equally likely.
Moreover, suppose we know that the eventF, that the first flip comes up tails, occurs.
• Given this information, what is the probability of the event E, that an odd number of
tails appears? Because the first flip comes up tails, there are only four possible
outcomes: TTT, TTH, THT, and THH, where H and T represent heads and tails,
respectively. An odd number of tails appears only for the outcomes TTT and THH.
• Because the eight outcomes have equal probability, each of the four possible
outcomes, given that F occurs, should also have an equal probability of 1/4. This
suggests that we should assign the probability of 2/4 = 1/2 to E, given that F occurs.
This probability is called the conditional probability of E given F. In general, to find
the conditional probability of E given F, we use F as the sample space. For an
outcome from E to occur, this outcome must also belong to E ∩ F.
Conditional Probability
 Example 1

A bit string of length four is generated at random so that each of the 16 bit
strings of length four is equally likely. What is the probability that it contains at
least two consecutive 0s, given that its first bit is a 0? (We assume that 0 bits
and 1 bits are equally likely.)
 Solution
Let E be the event that a bit string of length four contains at least two consecutive
0s, and let F be the event that the first bit of a bit string of length four is a 0.
The probability that a bit string of length four has at least two consecutive 0s,
given that its first bit is a 0, equals p(E | F ) = p(E ∩ F ) p(F )
Because E ∩ F = {0000, 0001, 0010, 0011, 0100}, we see that p(E ∩ F ) = 5/16.
Because there are eight bit strings of length four that start with a 0, we have p(F )
= 8/16 = 1/2. Consequently,
 Example 2

What is the conditional probability that a family with two children has two
boys, given they have at least one boy? Assume that each of the possibilities
BB, BG, GB, and GG is equally likely, where B represents a boy and G
represents a girl. (Note that BG represents a family with an older boy and a
younger girl while GB represents a family with an older girl and a younger boy.)
 Solution

• Let E be the event that a family with two children has two boys, and let F be
the event that a family with two children has at least one boy. It follows that
E = {BB}, F = {BB, BG, GB}, and E ∩ F = {BB}. Because the four possibilities
are equally likely, it follows that p(F ) = 3/4 and p(E ∩ F ) = 1/4. We conclude
that
 Independence
Suppose a coin is flipped three times, as described in the introduction to our
discussion of conditional probability. Does knowing that the first flip comes up
tails (event F) alter the probability that tails comes up an odd number of times
(event E)? In other words, is it the case that p(E | F ) = p(E)? This equality is valid
for the events E and F, because p(E | F ) = 1/2 and p(E) = 1/2. Because this equality
holds, we say that E and F are independent events. When two events are
independent, the occurrence of one of the events gives no information about
the probability that the other event occurs. Because p(E | F ) = p(E ∩ F )/p(F ),
asking whether p(E | F ) = p(E) is the same as asking whether p(E ∩ F ) = p(E)p(F ).
The events E and F are independent if and only if p(E ∩ F ) = p(E)p(F ).
 Example

Suppose E is the event that a randomly generated bit string of length four
begins with a 1 and F is the event that this bit string contains an even number
of 1s. Are E and F independent, if the 16 bit strings of length four are equally
likely?
 Solution
There are eight bit strings of length four that begin with a one: 1000, 1001,
1010, 1011, 1100, 1101, 1110, and 1111. There are also eight bit strings of length
four that contain an even number of ones: 0000, 0011, 0101, 0110, 1001, 1010,
1100, 1111. Because there are 16 bit strings of length four, it follows that p(E) =
p(F ) = 8/16 = 1/2. Because E ∩ F = {1111, 1100, 1010, 1001}, we see that p(E ∩
F ) = 4/16 = 1/4.
Because p(E ∩ F ) = 1/4 = (1/2)(1/2) = p(E)p(F ), we conclude that E and F are
independent.
Other Probabilities
 Example
• A fair coin is flipped 3 times. Let S be the sample space of 8
possible outcomes, and let X be a random variable that
assignees to an outcome the number of heads in this outcome.

• Random variable X is a function X:S → X(S),

where X(S)={0, 1, 2, 3} is the range of X, which is the number of
heads, and
S={ (TTT), (TTH), (THH), (HTT), (HHT), (HHH), (THT), (HTH) }

• X(TTT) = 0
X(TTH) = X(HTT) = X(THT) = 1
X(HHT) = X(THH) = X(HTH) = 2
X(HHH) = 3

• The probability distribution (pdf) of random variable X

is given by
P(X=3) = 1/8, P(X=2) = 3/8, P(X=1) = 3/8, P(X=0) = 1/8.
 Probabilities

• We write P(A) as “the fraction of possible worlds in which A is true”

 Visualizing A

Event space of
all possible
Worlds in which P(A) = Area of
worlds A is true reddish oval

Its area is 1
Worlds in which A is False
 Example

• What is the probability that, if we flip a coin three times, that we get
an odd number of tails?
(TTT), (TTH), (THH), (HTT), (HHT), (HHH), (THT), (HTH)
Each outcome has probability 1/8,
p(odd number of tails) = 1/8+1/8+1/8+1/8 = ½
 Visualizing
Sample Space

• 1. Listing
• S = {Head, Tail}
• 2. Venn Diagram
• 3. Contingency Table
• 4. Decision Tree Diagram
 Venn Diagram
Experiment: Toss 2 Coins. Note Faces.

Tail Event
TH
HH HT
Outcome
TT
S

S = {HH, HT, TH, TT} Sample Space

 Contingency Table
Experiment: Toss 2 Coins. Note Faces.
nd
2 Coin
st Head Tail
1 Coin Total Outcome
Simple
Event Head HH HT HH, HT
(Head on
Tail TH TT TH, TT
1st Coin)
Total HH, TH HT, TT S

S = {HH, HT, TH, TT} Sample Space

 Tree Diagram

Experiment: Toss 2 Coins. Note Faces.

H HH
H
T HT
Outcome

H TH
T
T TT
S = {HH, HT, TH, TT} Sample Space
May the light of wisdom be with you!

Keep your Faith and be Safe

Thank you!