Probability theory

Random variables
In an experiment a number is often attached to each outcome.
S
s X(s) R

Definition:
A random variable X is a function defined on S, which takes
values on the real axis
X: S → R
Sample space Real numbers

1 lecture 2
 Probability theory
Random variables
Example:
Random variable Type
Number of eyes when rolling a dice discrete 
The sum of eyes when rolling two dice discrete 
discrete 
counting
Number of children in a family

Age of first-time mother discrete 
Time of running 5 km continuous 

Amount of sugar in a coke continuous  measure
Height of males continuous 


Discrete: can take a finite number of values or an

infinite but countable number of values.
Continuous: takes values from the set of real numbers.
2 lecture 2
 Discrete random variable
Probability function

Definition:
Let X : S → R be a discrete random variable.

The function f(x) is a probabilty function for X, if

1. f(x) ≥ 0 for all x
2. ∑ f ( x ) = 1
x

3. P(X = x) = f(x),
where P(X=x) is the probability for the outcomes s∈S : X(s) = x.

3 lecture 2
 Discrete random variable
Probability function
Example: Flip three coins X : # heads X : S → {0,1,2,3}

Outcome Value of X Probability fuinction

TTT X=0 f(0) = P(X=0) = 1/8
HTT, TTH, THT X=1 f(1) = P(X=1) = 3/8
HHT, HTH, THH X=2 f(2) = P(X=2) = 3/8
HHH X=3 f(3) = P(X=3) = 1/8

Notice! The definition of a probability function is fulfilled:

1. f(x) ≥0
2. ∑ f(x) = 1
3. P(X=x) = f(x)

4 lecture 2
 Discrete random variable
Cumulative distribution function

Definition:
Let X : S → R be a discrete random variable with
probability function f(x).

The cumulative distribution function for X, F(x), is

defined by

F(x) = P(X ≤ x) = ∑ f (t)

t ≤x
for - ∞ < x < ∞

5 lecture 2
 Discrete random variable
Cumulative distribution function
Example: Flip three coins X : # heads X : S → {0,1,2,3}
Outcome Value of X Probability function Cumulative dist. Func.
TTT X=0 f(0) = P(X=0) = 1/8 F(0) = P(X < 0) = 1/8
HTT, TTH, THT X=1 f(1) = P(X=1) = 3/8 F(1) = P(X < 1) = 4/8
HHT, HTH, THH X=2 f(2) = P(X=2) = 3/8 F(2) = P(X < 2) = 7/8
HHH X=3 f(3) = P(X=3) = 1/8 F(3) = P(X < 3) = 1

Probability function: Cumulative distribution function:

f(x) F(x) 1.0
0.4 0.8
0.3 0.6
0.2 0.4
0.1 0.2

0 1 2 3 x 0 1 2 3 x
6 lecture 2
 Continuous random variable

A continuous random variable X has probability 0 for all

outcomes!!

Mathematically: P(X = x) = f(x) = 0 for all x

Hence, we cannot represent the probability function f(x) by a

table or bar chart as in the case of discrtete random variabes.

Instead we use a continuous function – a density function.

7 lecture 2
 Continuous random variable
Density function

Definition:
Let X: S → R be a continuous random variables.

A probability density function f(x) for X is defined by:

1. f(x) ≥ 0 for all x

∞
2. ∫ f ( x ) dx = 1
−∞
b
3. P( a < X < b ) = ∫ f(x) dx
a
Note!! Continuity: P(a < X < b) = P(a < X < b) = P(a < X < b) = P(a < X < b)

8 lecture 2
 Continuous random variable
Density function
Example: X: service life of car battery in years (contiuous)
Density function: 0,5

0,4
− 0.16 + 0.16 x for 1 ≤ x ≤ 3.5

0,3

f ( x) =  0.96 − 0.16 x for 3.5 < x ≤ 6 0,2

 0,1

 0 otherwise 0
1 3 3,5 6

Probability of a service life longer than 3 years:

∞
P( X > 3) = ∫ f ( x) dx
3
3.5 6
= ∫ (−0.16 + 0.16 x) dx + ∫ (0.96 − 0.16 x) dx
3 3.5

=  = 0.68
9 lecture 2
 Continuous random variable
Density function
Alternativ måde:
0,5

− 0.16 + 0.16 x for 1 ≤ x ≤ 3.5 0,4

 0,3
f ( x) =  0.96 − 0.16 x for 3.5 < x ≤ 6 0,2


 0 othwerwise 0,1

0
1 3 3,5 6

Probability of a service life longer than 3 years:

P( X > 3) = 1 − P( X ≤ 3)
3
= 1− ∫ f ( x) dx
−∞
3
= 1 − ∫ (−0.16 + 0.16 x) dx
1

=  = 1 − 0.32 = 0.68
10 lecture 2
 Continuous random variable
Density function
Definition:
Let X : S → R be a contiguous random variable with density
function f(x).

The cumulative distribution function for X, F(x), er defined by

x
F(x) = P(X ≤ x) = ∫ f ( t ) dt
−∞
for - ∞ < x < ∞
Note: F´(x) = f(x)
F(3) = P( X ≤ 3) 0,5

3 0,4

= ∫ f ( x ) dx 0,3

−∞ 0,2

3 0,1

= ∫ − 0.16 + 0.16 x dx 0
1 3 3,5 6
1

11 =  = 0.32 lecture 2
 Continuous random variable
Density function
From the definition of the cumulative distribution funct. we get:

P( a < X < b ) = P( a < X < b ) = P( a < X < b ) = P( a < X < b )

= P(X < b ) – P(X < a )
= F(b) – F(a)

a b

12 lecture 2
 Continuous random variable
Density function

Discrete random variable Contiuous random variable

• Sample space is finite or has • The sample space contains

countable many outcomes infinitely many outcomes

• Probability function f(x) • Density function f(x) is a

Is often given by table Continuous function

• Calculation of probabilities • Calculation of probabilities

P( a < X < b) = ∑ f(t) b
a<t<b P( a < X < b ) = ∫ f(t) dt
a

13 lecture 2
 Joint distribution
Joint probability function

Definition:
Let X and Y be two discrete random variables.
The joint probability function f(x,y) for X and Y
Is defined by
1. f(x,y) ≥ 0 for all x og y

2. ∑ ∑ f ( x, y ) = 1
x y

3. P(X = x ,Y = y) = f(x,y) (the probability that both X = x and Y=y)

For a set A in the xy plane: P(( X, Y ) ∈ A ) = ∑ ∑ f ( x, y )

( x ,y ) ∈A

14 lecture 2
 Joint distribution
Marginal probability function

Definition:
Let X and Y be two discrete random variables with joint
probability function f(x,y).
The marginal probability function for X is given by
g(x) = Σy f(x,y) for all x

The marginal probability function for Y is given by

h(y) = Σ f(x,y) for all y
x

15 lecture 2
 Joint distribution
Marginal probability function

Example 3.14 (modified):

The joint probability function f(x,y) for X and Y is given by

x 0 1 2
y
• h(1) = P(Y =1) = 3/14+3/14+0 = 3/7
0 3/28 9/28 3/28

1 3/14 3/14 0 • g(2) = P(X= 2) = 3/28+0+0 = 3/28

2 1/28 0 0
• P(X+Y < 2) = 3/28+9/28+3/14 = 18/28 = 9/14

16 lecture 2
 Joint distribution
Joint density function

Definition:
Let X og Y be two continuous random variables.
The joint density function f(x,y) for X and Y is
defined by
1. f(x,y) ≥ 0 for all x
∞ ∞
2. ∫−∞ ∫−∞ f ( x, y ) dx dy = 1
3. P(a < X< b, c<Y< d) = ∫c ∫ f ( x, y ) dx dy
d b

For a region A in the xy-plane: P[(X,Y)∈A] = ∫ ∫ f(x,y) dxdy

A
17 lecture 2
 Joint distribution
Marginal density function

Definition:
Let X and Y be two continuous random variables with
joint density function f(x,y).
The marginal ∞
density function for X is given by
g ( x) = ∫ f ( x, y ) dy for all x
−∞
The marginal density function for Y is given by
∞
h( y ) = ∫ f ( x, y) dx
−∞
for all y

18 lecture 2
 Joint distribution
Marginal density function

Example 3.15 + 3.17 (modified):

Joint density f(x,y) for X and Y:
2

2
 (2x + 3y) 0 ≤ x ≤ 1, 0 ≤ y ≤ 1 1.5

f(x, y) =  5

1

0 otherwise 0.5
1

Marginal density
∞
function for X: 0
1
0.8
0.5

g( x ) = ∫ f ( x, y ) dy
0.6
0.4
0.2 0
0

−∞
1

= ∫ 2 (2x + 3 y ) dy
0 5

19
= [ 2
5
1
2xy + 3 y
5
2
]1
0
=
4
5
x+
3
5 lecture 2
 Joint distribution
Conditional density and probability
functions
Definition:
Let X and Y be random variables (continuous or discrete)
with joint density/probability function f(x,y). Then the
Conditional denisty/probability function for Y given X=x is
f(y|x) = f(x,y) / g(x) g(x) ≠ 0
where g(x) is the marginal density/probability function for X, and
the conditional density/probability function for X given Y=y is
f(x|y) = f(x,y) / h(y) h(y) ≠ 0
where h(y) is the marginal density/probability function for Y.

20 lecture 2
 Joint distribution
Conditional probability function
Examples 3.16 + 3.18 (modified):
Joint probability function
f(x,y) for X and Y is given by:
10 for x=0
x  28
y 0 1 2 • marginal pf. g( x ) = 15 for x =1
28
3
0 3/28 9/28 3/28  28 for x=2
1 3/14 3/14 0

2 1/28 0 0 • P(Y=1 | X=1 ) = f(1|1)

= f(1,1) / g(1)
= (3/14) / (15/28)
= 6/15
21 lecture 2
 Joint distribution
Independence

Definition:
Two random variables X and Y (continuous or discrete) with
joint density/probability functions f(x,y) and marginal
density/probability functions g(x) and h(y), respectively, are
said to be independent if and only if
f(x,y) = g(x) h(y) for all x,y

or if f(x|y) = g(x) (x indep. of y) or f(y|x)=h(y) (y indep. af x)

22 lecture 2