Probability
• A probability provides a quantitative description of the likely
occurrence of a particular event.
– Mathematical Study of randomness
– Quantitative measure of chance.
• It is conventionally expressed on a scale from 0 to 1;
– A rare event has a probability close to 0, a very common event has a
probability close to 1.
– 0: Absolutely no chance
– 1: Absolute certainty
• Alternative Approaches:
– The probability of an event has been defined as its long-run relative
frequency.
– It has also been thought of as a personal degree of belief that a
particular event will occur
Some basic concepts/terminologies in probability
• Some terminologies:
– Trial and Experiment
– Sample Space
• It is an exhaustive list of all the possible outcomes of an
experiment.
– Event
• Elementary Event
– an event which consists of exactly one outcome
• Composite Event
– event that contains more than one outcome
– a composite event includes more than one elementary events.
– Example: Toss two coins.
• Possible outcomes {HH, HT, TH, TT}
• The event “get two heads” is an elementary event
• The event “get two of the same side” is a composite event.
Some basic concepts/terminologies in probability
• Mutually exclusive events:
– cant occur simultaneously
– no element in common
• Non-exclusive events:
– have one or more elementary outcomes common between them
– they can occur together.
• Exhaustive events:
– Two or more events are exhaustive if their union is the sample space.
• All elements in the sample space are mutually exclusive and collectively
exhaustive.
exhaustive
– Composite events need not be mutually exclusive.
• Sum of probabilities of events that are mutually exclusive and
collectively exhaustive is always 1.0.
1.0
• Complementary event: The event in question does not occur.
• Complement of event A is designated A’.
– Clearly P(A’) = 1-P(A).
Composite Events
• Assumption: All elements in the sample space are equally
likely to occur
• Experiment: Rolling a die:
• What is the chance of each elementary event occurring?
– Clearly 1 in 6 or 1/6 = 0.1667.
• Can we now determine probabilities of composite events?
• Event A: outcome of the toss is an even number.
– Which elementary outcomes are favorable to event A?
• 2, 4, 6.
– Since each of these has a 1 in 6 chance, P(A) = 3/6 = 0.5
• Event B: outcome is divisible by 3
– Favorable outcomes: 3, 6.
– P(B) = 2/6 = 0.333.
• Odds: ratio of favorable to unfavorable outcomes.
– What are the odds that the outcome is divisible by 3? 2:4
• If the odds of an event are a:b, then its probability is a/(a+b).
Probability of union of events
• P(A or B) or P(A B) =?
• If A and B are mutually exclusive events,
– P(A or B) = P(A B) = P(A) + P(B)
Example:
4
• A: Outcome of die toss is less then 3 1 5
• B: Outcome is divisible by 3 2 6 3
• P(A or B) = 2/6 + 2/6 = 4/6 = 0.667
A B
• What if they are non-exclusive?
Example:
• A: Even number 1
4 5
• B: Divisible by 3 6 3
2
P(A B) = P(A)+P(B)-P(A B)
A B
– General rule of addition
• P(A B) = 3/6+2/6-1/6 = 4/6
Independent events
• Draw a card from a deck.
• Put it aside.
• Now draw another card.
– What is the probability that first card is a King?
– What is the probability that second card is a King?
– Does it depend of the first draw? Why?
– What if the first card is put back in the deck before the second
draw? Does it still depend on the first draw? Why?
• Two events are independent if the occurrence of non-
occurrence of one event has no effect on the probability
of the other event.
• They are dependent when the occurrence of one event
does affect the probability of the other event.
Conditional Probability
• Event A: A King on the first draw.
• Event B: A King on the second draw.
– P(A) = 4/52
• What about of P(B)?
– It depends on whether first draw was a King or not.
– If A occurred, probability of B = 3/51
– If A did not occur, probability of B = 4/51.
• Notice that there is a condition attached to the probability in
each case.
– P(B|A) = 3/51
– P(B|A’) = 4/51.
• Why is the probability different in the two cases?
– Dependence!
• Conditional Probability:
– If A occurred, probability of B:
P(B|A): probability of B, when A has happened.
Joint Probability
• For two events A and B, what is the probability that both events take place.
– Joint probability: P(A and B) or P(A B) or P(AB)
• If A and B are independent events, P(A and B) = P(A).P(B)
• What if they are dependent?
• View this as a two step process:
– A occurs.
– Given that A has occurred, B occurs.
P(A and B) = P(A). P(B|A)
• A die toss example:
– A: outcome is less than 5 New Sample Space
– B: outcome is a multiple of 3
• P(A) = 4/6, P(B) = 2/6, P(B|A) = ? 4 5
3 6
• P(B|A) = 1/4 A 2 1
B
• P(A and B) = (4/6).(1/4) = 1/6
P(B|A) = P(A and B) / P(A)
Summary
P(A B) = P(A) + P(B) Where events A and B
are mutually exclusive
P(A B) = P(A)+P(B)-P(A B) A and B are non-exclusive
Type of Probability Formula under Formula under
statistical statistical
independence dependence
Marginal
P(A) P(A) Total Probability
Joint
P(AB) P(A) × P(B) P(A|B) × P(B)
P(BA) P(B) × P(A) P(B|A) × P(A)
Conditional
P(B|A) P(B) P(BA)/P(A)
P(A|B) P(A) P(AB)/P(B)
