Chapter I
Chapter I
ECON 2042
Addis Ababa University
Department of Economics
May, 2022
CHAPTER ONE
OVERVIEW OF BASIC PROBABILITY
THEORY
Introduction
There are large numbers of happenings in nature and in the
realm of human activity that are associated with uncertainties.
Uncertainty plays an important role in our daily lives and
activities as well as in business.
It is often necessary to "guess" about the outcome of an event
in order to make a decision.
In our daily lives we are faced with a lot of decision-making
situations that involve uncertainty.
For example:
In a business context: investment, stock prices
Game Wins, Lottery wins, Election win, Traffic
jam
Market Conditions: effectiveness of an ad
campaign or the eventual success of a new product.
Weather forecast: appearance of clouds in the sky
next morning is not as certain.
The sex of a baby to be born some months hence is
again not known for certain.
In generally, we often do not know an outcome with certainty,
instead, we are forced to guess, to estimate, to hedge our bets.
In each of such happenings there are two or more outcomes
leading to uncertainty.
If an experiment is repeated under essentially similar conditions,
we generally come across two types of situation: namely
1) the result or the outcome is unique or certain - known as
deterministic,
2) the result is not unique rather may be one of the several
possible outcomes- known as unpredictable or non-
deterministic or probabilistic phenomenon.
The experiment (models) related to non-deterministic phenomenon
is called random experiment or non-deterministic model.
In this course we will deal with the random experiments- or non -
deterministic models.
Examples:
Exp1: Toss a die and observe the number that shows up.
Exp2: Toss a coin n time and observe the total number of
heads obtained
Exp3: Manufacture items on a production line and count the
number of defective items produced during a 24 hr. period.
Exp4: testing the life length of a manufactured light bulb
by inserting it into a socket and the time elapsed (in hrs.) until it
burns out recorded.
Exp5: Measuring blood pressure of a group of individuals,
Exp6: Checking an automobile’s petrol mileage,
Exp7 : Measuring daily rainfall, and so on.
So what is probability theory ?
Probability is the science of uncertainty.
It provides precise mathematical rules for understanding and
analyzing our own ignorance.
It does not tell us tomorrow’s weather or next week’s stock prices;
rather, it gives us a framework for working with our limited
knowledge and for making sensible decisions based on what we
do and do not know.
Is a measure of uncertainty with values between zero and one,
inclusive, describing the relative possibility(chance or likelihood)
of occurrence of an event.
In sum, probability theory provides us with a precise
understanding of uncertainty; and this understanding can help us
make predictions, make better decisions, assess risks, and even
make money.
1.1 Sample Space, Sample Points, Events and Event
Space
i) Experiment: An activity or measurement that results in an
outcome.
ii) Sample space: is the set of all possible outcomes of an experiment
denoted by S or Ω.
iii) Sample points: Each element of the sample space S is called a
sample point and usually denoted by ω𝑖 .
iv) Events: One or more of the possible outcomes; a subsets of the
sample space, S.
v) Event space: is a class or collection of all events associated with a
given experiment OR sample space. We use E to denote event
space.
EXAMPLE: Let the experiment be rolling of a well balanced single
die one times.
Given this:
The sample space of the experiment is S = {1, 2, 3, 4, 5, 6}
which contains all the possible outcomes of the experiment.
Each outcome is a sample point, i.e., 𝝎𝒊 ∈ S.
Let A = {2, 4, 6}.
Then A is a subset of S and defines the event of obtaining an
even outcome.
Let B = {4}.
Then B defines the event of obtaining a number 4.
Both A and B are subsets of the sample space and thus are
events.
We may be interested on the following events
a) the outcome is the number 1
b) the outcome is even but less than 3
c) the outcomes is not even and so on.
When an event contains only one element of S, like B above, it is
called simple(elementary) event.
If we collect all subsets of S in one set and denote it by E this new set
is called Event Space.
Types of Events
We defined have already defined Event as any subset of
outcomes of an experiment.
1) Mutually Exclusive Events: If two or more events cannot occur
simultaneously in a single trial of an experiment, then such events
are called mutually exclusive events or disjoint events.
In other words, two events are mutually exclusive if the occurrence
of one of them prevents or rules out the occurrence of the other.
For example, the numbers 2 and 3 cannot occur simultaneously on the
roll of a dice.
Symbolically, a set of events {𝑨𝟏 , 𝑨𝟐 , . . ., 𝑨𝒏 } is mutually
exclusive if 𝑨𝒊 ∩ 𝑨𝒋 = ∅ for i ≠ j.
This means the intersection of two events is a null set (∅); it is
impossible to observe an event that is common in both 𝑨𝒊 and 𝑨𝒋 .
C) Subjective Definition
If there is little or no past experience on which to base probability it may be
guessed subjectively.
Essentially, this means evaluating the available information and then estimating
the probability of an event.
The subjective approach is always based on the degree of beliefs, convictions, and
experience concerning the likelihood of occurrence of a random event.
Example: Estimating:
i) The probability that a new product of a firm will be successful in the market.
ii) The probability that a student will score an A in the course, and
iii) When a person says that the probability of rain tomorrow is, say 70% - he is
expressing his personal degree of belief.
1.3. Axioms of Probability: The Rule of probability
Before giving axiomatic definition of probability let’s first see, the summary of what
we know about set theory which are relevant to our interest.
De Morgan’s Law
I) (𝐀 ∪ 𝑩)= 𝑨 ∩ 𝑩
II) (𝐀 ∩ 𝑩)= 𝑨 ∪ 𝑩
NOTE: Answers for questions iii to v provide us a proof for the third
axiom.
Rules(Properties) of Probability
Other properties that we associate with a probability can be derived from the axioms.
1) The Complement Rule:
Sometimes it is easier to calculate the probability of an event happening by determining
the probability of it not happening and subtracting the result from 1.
Because A and its complement 𝐴𝐶 = {ω; ω ∉ A} are mutually exclusive and
collectively exhaustive, the probabilities of A and 𝐴𝐶 sum to 1.
P(A) + P(𝑨𝑪 ) = P(A ∪ 𝑨𝑪 ) = P(Ω) = 1
or P(𝑨𝑪 ) = 1 − P(A).
Example_1: if we toss a biased coin, we may want to say that P{heads} = p where p is not
necessarily equal to 1/2.
By necessity, P{tails} = 1 − p.
Example_2: Toss a coin 4 times.
5 𝟏𝟏
P{fewer than 3 heads} = 1 − P{at least 3 heads} = 1 − =
16 𝟏𝟔
2) The Difference Rule:
We write B\A to denote the outcomes that are in B but not in A.
If A ⊂ B, then
P(B \ A) = P(B) − P(A)
(The symbol ⊂ denotes “contains in”. A and B\A are mutually exclusive and their
union is B.
Thus, P(B) = P(A) + P(B\A).
Example_1: In A single die roll: let
A: an event of odd number
B: a number less than or equal to five
Solution:
A = {1, 3, 5} B = {1, 2, 3, 4, 5}
B/A = {2, 4}
P(B) = P(A) + P(B\A)
Exercises: The events X and Y are mutually exclusive. Suppose P(X) = .05
and P(Y) = .02.
i) What is the probability of either X or Y occurring?
ii) What is the probability that neither X nor Y will happen?
Rules of Multiplication
5) The Special Rule of Multiplication
The special rule of multiplication requires that two events A and B are
independent.
Two events are independent if the occurrence of one event does not alter
the probability of the occurrence of the other event.
Example: suppose two coins are tossed. The outcome of a coin toss (head or
tail) is unaffected by the outcome of any other prior coin toss (head or tail).
For two independent events A and B, the probability that A and B will both
occur is found by multiplying the two probabilities.
This is the special rule of multiplication and is written symbolically as:
𝑷(𝑨 𝒂𝒏𝒅 𝑩) = 𝑷(𝑨)𝑷(𝑩)
For three independent events, A, B, and C, the special rule of multiplication
used to determine the probability that all three events will occur is:
𝑷(𝑨 𝒂𝒏𝒅 𝑩 𝒂𝒏𝒅 𝑪) = 𝑷(𝑨)𝑷(𝑩)𝑷(𝑪)
Example_1: A survey by the American Automobile Association (AAA)
revealed 60% of its members made airline reservations last year. Two members
are selected at random. What is the probability both made airline reservations
last year?
Solution:
The probability the first member made an airline reservation last year is .60,
written P(𝑅1 ) = 0.60, where 𝑅1 refers to the fact that the first member made a
reservation. The probability that the second member selected made a
reservation is also .60, so P(𝑅2) = 0.60.
Because the number of AAA members is very large, you may assume that 𝑹𝟏
and 𝑹𝟐 are independent. Consequently, using the previous formula, the
probability they both make a reservation is 0.36, found by:
P(𝑅1 and 𝑅2) = P(𝑅1 )P(𝑅2 = (.60)(.60) = 0.36
Furthermore, recall R indicates that a reservation is made and 𝑅 indicates
no reservation is made.
The complement rule is applied to compute the probability that a member
does not make a reservation, P(𝑹) = 0.40.
Using this information, the probability that neither member makes a
reservation, [P( 𝑅1 ) P 𝑅2)] = (.40)(.40) = 0.16.
3) Combination Rule
Sometimes the ordering or arrangement of objects is not important, but
only the objects that are chosen. For example, we may not care in what
order the books are placed on the shelf, but only which books you are able
to shelve. In addition, when a five-person committee is chosen from a
group of 10 students, the order of choice is not important because all 5
students will be equal members of committee.
This counting rule for combinations allows us to select r (say) number of
outcomes from a collection of n distinct outcomes without caring in what
order they are arranged. This rule is denoted by
𝑛 𝒏!
nCr 𝒐𝒓 𝑟 =
(𝒏 − 𝒓)! 𝒓!
Example 1: From eight persons, how many committees of 3 members
may be chosen?
Solution: Since two committees are the same if they are made up of the same
members we have,
8 𝟖! 𝟖!
C
8 3 𝒐𝒓 = = = 𝟓𝟔 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 𝑐𝑜𝑚𝑚𝑖𝑡𝑒𝑠
𝟑 (𝟖−𝟑)!𝟑! 𝟓!𝟑!
If one engineer is selected at random from the company, find the probability that,
A) he has only a bachelor’s degree.
B) he has a master’s degree, given that he is over 40.
C) he is under 30, given that he has only a bachelor’s degree.
1.6 Bayes' Theorem
Thomas Bayes was the 18𝑡ℎ century British minister and clergyman.
He was interested in the question: Does God really exist?
To answer this question, he attempted to develop a formula to
determine the probability that God does exist based on evidence that
was available to him on earth.
Later Laplace refined Bayes’ work and gave it the name Bayes’
Theorem.
The Bayes’ theorem is useful in revising the original probability
estimates of known outcomes as we gain additional information about
these outcomes. The prior probabilities, when changed in the light of
new information, are called revised or posterior probabilities.
It is a method to compute posterior probabilities (conditional
probabilities under statistical dependence)(is based on conditional
probabilities).
Suppose 𝑨𝟏 , 𝑨𝟐 , . . . , 𝑨𝒏 represent n mutually exclusive and collectively
exhaustive events with prior marginal probabilities 𝐏(𝑨𝟏 ), 𝐏(𝑨𝟐 ). . . ,𝐏(𝑨𝒏 ). Let B
be an arbitrary event with 𝑷(𝑩) ≠ 𝟎 for which conditional probabilities 𝐏(𝐁|𝑨𝟏 ),
𝐏(𝐁|𝑨𝟐 ), . . . , 𝐏(𝐁|𝑨𝒏 ), are also known.
Given the information that outcome B has occurred, the revised (or posterior)
probabilities 𝐏(𝑨𝒊 |𝑩), are determined with the help of Bayes’ theorem as:
𝑷(𝑨𝒊 ∩ 𝑩)
𝐏 𝑨𝒊 𝑩 = … … … … … … … … … … … … … … … … … … . . . (𝒂)
𝑷(𝑩)
Where 𝐏 𝑨𝒊 𝑩 is read as the posterior probability of events 𝑨𝒊 given event B (thus,
conditional probability).
Since events 𝑨𝟏 , 𝑨𝟐 , . . . , 𝑨𝒏 are mutually exclusive and collectively exhaustive, the
event B is bound to occur with either 𝑨𝟏 , 𝑨𝟏 , . . . ,𝑨𝒏 . That is,
𝑩 = 𝑨𝟏 ∩ 𝑩 ∪ 𝑨𝟐 ∩ 𝑩 ∪ ⋯ ∪ (𝑨𝒏 ∩ 𝑩)
Since 𝑨𝟏 ∩ 𝑩 , 𝑨𝟐 ∩ 𝑩 ,…(𝑨𝒏 ∩ 𝑩) are mutually exclusive,
𝑷 𝑩 = 𝑷 𝑨𝟏 ∩ 𝑩 + 𝑷 𝑨𝟐 ∩ 𝑩 + ⋯ + 𝑷(𝑨𝒏 ∩ 𝑩)
= 𝒏𝒊=𝟏 𝑷(𝑨𝒊 ∩ 𝑩)
= 𝑷 𝑨𝟏 𝑷(𝑩|𝑨𝟏 ) + 𝑷 𝑨𝟐 𝑷(𝑩|𝑨𝟐 ) + ⋯ + 𝑷 𝑨𝒏 𝑷(𝑩|𝑨𝒏 )
𝒏
𝑷 𝑩 = 𝑷 𝑨𝒊 𝑷(𝑩|𝑨𝒊 )
𝒊=𝟏
From formula (a) given in the previous, for a fixed i, we have,
𝑷(𝑨𝒊 ∩ 𝑩) 𝑷 𝑨𝒊 𝑷(𝑩|𝑨𝒊 )
𝐏 𝑨𝒊 𝑩 = =
𝑷(𝑩) 𝑷 𝑨𝟏 𝑷(𝑩|𝑨𝟏 ) + 𝑷 𝑨𝟐 𝑷(𝑩|𝑨𝟐 ) + ⋯ + 𝑷 𝑨𝒏 𝑷(𝑩|𝑨𝒏 )
The steps in this probability revision process are shown in figure below.
Illustration
Example 1: Suppose 5 percent of the population of X, fictional Third
World country, have a disease that is peculiar to that country. We will let 𝑨𝟏
refer to the event “has the disease” and 𝑨𝟐 refer to the event “does not have
the disease.”
Thus, we know that if we select a person from the X at random, the
probability the individual chosen has the disease is 0.05, or 𝐏 𝑨𝟏 =
𝟎. 𝟎𝟓.
This probability, 𝐏 𝑨𝟏 = 𝑷 𝒉𝒂𝒔 𝒕𝒉𝒆 𝒅𝒊𝒔𝒆𝒂𝒔𝒆 = 𝟎. 𝟎𝟓, is called the
Prior Probability. It is given this name because the probability is
assigned before any empirical data are obtained.
Prior Probability is the initial probability based on the present level of
information.
Thus, the prior probability a person is not afflicted with the disease is
0.95, or 𝐏 𝑨𝟐 = 𝟎. 𝟗𝟓.
Additional Information
There is a diagnostic technique to detect the disease, but it is not very
accurate. Let B denote the event “test shows the disease is present.”
Assume that historical evidence shows that if a person actually has the disease,
the probability that the test will indicate the presence of the disease is 0.90.
Using the conditional probability definitions developed, this statement is written
as:
𝐏 𝑩|𝑨𝟏 = 𝟎. 𝟗𝟎
Assume the probability is 0.15 that for a person who actually does not have the
disease the test will indicate the disease is present.
𝐏 𝑩|𝑨𝟐 = 𝟎. 𝟏𝟓
Let’s randomly select a person from X and perform the test. The test results indicate
the disease is present. What is the probability the person actually has the
disease?
That is 𝑷(𝒉𝒂𝒔 𝒕𝒉𝒆 𝒅𝒊𝒔𝒆𝒂𝒔𝒆|𝒕𝒉𝒆 𝒕𝒆𝒔𝒕 𝒓𝒆𝒔𝒖𝒍𝒕𝒔 𝒂𝒓𝒆 𝒑𝒐𝒔𝒊𝒕𝒊𝒗𝒆)
The probability 𝐏 𝑨𝟏 |𝑩 is called a Posterior Probability.
Posterior Probability is a revised probability based on additional
information.
Now, the remaining step will be to determine the posterior probability, and
with the help of Bayes’ theorem we can determine the posterior probability as
follow:
𝑷 𝑨𝟏 𝑷(𝑩|𝑨𝟏 )
𝐏 𝑨𝟏 𝑩 =
𝑷 𝑨𝟏 𝑷(𝑩|𝑨𝟏 ) + 𝑷 𝑨𝟐 𝑷(𝑩|𝑨𝟐
𝟎. 𝟎𝟓 ∗ 𝟎. 𝟗𝟎
𝐏 𝑨𝟏 𝑩 = = 𝟎. 𝟐𝟒
𝟎. 𝟎𝟓 ∗ 𝟎. 𝟗𝟎 + 𝟎. 𝟗𝟓 ∗ 𝟎. 𝟏𝟓
So the probability that a person has the disease, given that he or she tested
positive, is 0.24.
How is the result interpreted? If a person is selected at random
from the population, the probability that he or she has the disease is 0.05.
If the person is tested and the test result is positive, the probability that
the person actually has the disease is increased about fivefold, from 0.05
to 0.24.
Example 2: Suppose an item is manufacture by three machines X, Y, and Z having
equal capacity and operational rate. It is known that the percentages of defective items
produced by X, Y, and Z are 2, 7, and 12 percent, respectively. All the items produced by
X, Y, and Z are put into one bin. From this bin, one item is drawn at random and is found
to be defective. What is the probability that this item was produced on Y?
Solution: Let A be the defective item. We know the prior probability of defective
items produced on X, Y, and Z, that is, 𝑷(𝑿) = 𝟏/𝟑; 𝑷(𝒀) = 𝟏/𝟑 and 𝑷(𝒁) = 𝟏/𝟑.
We also know that 𝑃(𝐴|𝑋) = 0.02, 𝑷(𝑨|𝒀) = 𝟎. 𝟎𝟕, 𝑷(𝑨|𝒁) = 𝟎. 𝟏𝟐.
𝑷 𝒀 𝑨 = ???
Then,
𝑷 𝒀 𝑷(𝑨|𝒀)
𝐏 𝒀𝑨 =
𝑷 𝑿 𝑷(𝑨|𝑿) + 𝑷 𝒀 𝑷(𝑨|𝒀) + 𝑷 𝒁 𝑷(𝑨|𝒁)
1
0.07 ∗ ( ) 0.07 0.07
= 3 = =
1 1 1
∗ 0.02 + ∗ 0.07 + ∗ (0.12) 0.02 + 0.07 + 0.12 0.21
3 3 3
= 𝟎. 𝟑𝟑𝟑 or 33.3% they it will be from machine Y.
Exercise 1: A factory produces certain types of output by three machines.
Daily production for Machine 𝑨 = 3000 units; Machine 𝑩 = 2500 units;
and Machine 𝑪 = 4500 units, respectively. Past experience shows that 1%,
1.2% and 2% of the outputs from machines A, B and C, respectively, are
defective. An item is drawn at random from the day’s production and is found
to be defective. What is probability that it comes from the output of
a) Machine A?
b) Machine B? and
c) Machine C ?
Exercise 2: In a town, 10% of all adults over 50 years have diabetes. If a
doctor correctly diagnoses 90% of all persons with diabetes as having the
disease and incorrectly diagnoses 2% of all persons without diabetes as having
the disease, what is the probability of an adult over 50 is diagnosed by this
doctor as having diabetes and actually has the disease.
END !