Lecture 8

Defining Econometrics, Economic Models, and Econometric Models

Introduction to Econometrics
BSc Eco 2023, Spring 2025
Instructor: Sunaina Dhingra
Email-id: [email protected]
Lecture Date: March 3rd 2025
Assumptions and Unbiasedness
Property of OLS Estimators
 SLR.1: Linear in parameters

Source: Wooldridge, Chapter 2

• We need linearity in parameters, i.e., the β’s should have power 1 but our equation
may not be linear in variables y and x
 SLR.2: Random Sampling

Source: Wooldridge, Chapter 2

• As the primary goal is to draw conclusions about the population, the sample that
we used must be drawn at random from the population

• If the sample is not drawn at random, it may not be representative of the

population and the conclusions that we draw from it may be biased
The Simple Regression Model
• Discussion of random sampling: Wage and education
• The population consists, for example, of all workers of country A
• In the population, there is a linear relationship between wages and years
of education.
• Draw completely randomly a worker from the population
• The wage and the years of education of the worker drawn are random
because one does not know beforehand which worker is drawn.
• Throw that worker back into the population and repeat the random draw n
times.
• The wages and years of education of the sampled workers are used to
estimate the linear relationship between wages and education.

6
The Simple Regression Model

7
 SLR.3: Sample Variation in the explanatory variable
Source: Wooldridge, Chapter 2

• This assumption requires that our x

needs to vary in some way

• Assumption SLR.3 fails if the sample

standard deviation of xi is zero;
otherwise, it holds.
• If this is zero, we cannot find beta1hat,
or beta0hat
Source: Author’s estimation using Wage1 dataset in STATA, refer Do file for command
 SLR.4: Zero Conditional Mean

Source: Wooldridge, Chapter 2

• It states that the average value of the disturbance, conditional on x, equals zero for all
possible values of the explanatory variable.

• It also implies the cloud of data centers on a straight line at every possible value of x
 Statistical Property 1: Unbiasedness
• Bias means our estimator’s expected value does not equal the true value in the population

• In some ways, biased estimators aren't even "correct on average.“

• The estimators are unbiased if

መ = 𝛽-------(1)
E(𝛽)

• Unbiasedness does not imply that the 𝛽መ for every possible sample is equal to its
population value

• It only means that on an average 𝛽መ are not too large or small in comparison to the
population value
 Theorem1: Unbiasedness of OLS Estimates
Source: Wooldridge, Chapter 2

• If assumptions hold, we have a very important theorem that states that under assumptions SLR.1
through SLR.4, OLS estimates of 𝛽 ෢0 and 𝛽෢1 are unbiased, i.,e they are equal to the population 𝛽
෢0 and 𝛽
෢1
, on average.
• In real world, simple linear regression estimators will most commonly be biased
• Unbiased SLR parameters (Theorem 1) only works if E(u|x)=0 i.e. SLR.4 is true

Interpretation of unbiasedness
• The estimated coefficients may be smaller or larger, depending on the sample that is the result of a random draw.
• However, on average, they will be equal to the values that characterize the true relationship between y and x in the
population.
• “On average” means if sampling was repeated, i.e. if drawing the random sample and doing the estimation was
repeated many times.
• In a given sample, estimates may differ considerably from true values.
 Failure of SLR.4 may lead to biased Estimates

• Usually SL.4 fails to be true due to:

❑ Reverse causality

❑ Wrong functional form

❑ Expected values of y conditional on x do not really fall on a straight line

❑ Error in measurement of x variable

❑ Omitted variables
Assumptions of Homoskedasticity
and variance of OLS Estimators
 Introduction
መ =𝛽
• The unbiasedness property states that E(𝛽)
• It is also important to know how far we can expect β ෢1 to be away from true β1 on an
average
• This allows us to choose the best estimator among all
• The measure of spread to work with is the variance or its square root, the standard
deviation
• The variance of the OLS estimators can be computed if assumptions SLR.1-5 are met

• Variances of the OLS estimators

• Depending on the sample, the estimates will be nearer or farther away
from the true population values.
• How far can we expect our estimates to be away from the true population
values on average (= sampling variability)?
• Sampling variability is measured by the estimator‘s variances
 SLR.5 Homoskedasticity
Source: Wooldridge, Chapter 2

• Homoskedasticity assumption states that the variance of error term conditional on x, is

constant

• This is also known as the “constant variance” assumption

• SLR.4 and SLR.5 are different from each other.

• SLR4 involves the expected value of errors conditional on x
• SLR.5 involves the variance of error conditional on x

• SLR.5 plays no role in showing that OLS estimators are unbiased

The Simple Regression Model
• Graphical illustration of homoskedasticity

16
The Simple Regression Model
• An example for heteroskedasticity: Wage and education

17
 Conditional Variance of the Error term
𝑉𝑎𝑟 μ|𝑥 = E[ (μ −μ) | x] 2 -----------(1)

=E[μ2|x]-[E[μ|x]]2

E[μ|𝑥] = 0, 𝐸[μ2|x] = 𝜎 2

𝜎 2 = 𝑉𝑎𝑟 μ −−−−−−−−−−−(2)
• σ2 is the unconditional variance of error u, and so σ2 is often called the error variance or
disturbance variance

• The square root of 𝜎 2 , σ, is the standard deviation of the error

• A larger σ means that the distribution of the unobserved errors affecting 𝑦 is more
spread out.
 Conditional Variance of the Error term(contd)

Recall,
𝐸[𝑦|𝑥] = 𝛽0 + 𝛽1 𝑥 −−−− −(3)

𝑉𝑎𝑟[μ|𝑥] = 𝜎 2 −−−− −(4)

• The conditional expectation of y given x is linear in x, but the variance of y given x is
constant 𝛽0 > 0 and 𝛽1 > 0 Figure 1: The simple regression model under homoskedasticity

Source: Wooldridge, Chapter 2, Figure 2.8

 Heteroskedasticity

• When 𝑉𝑎𝑟 μ|𝑥 depends on x, the error term is said to exhibit heteroskedasticity (or non
constant variance)

𝑉𝑎𝑟[μ|𝑥]=𝜎𝑖2 −−− −(5)

 Sampling variances of the OLS estimators
Theorem 2.2 (Variances of the OLS estimators)

• Under the assumptions SLR.1 through SLR.5

෢1 ) = 𝜎2 𝜎2
𝑉𝑎𝑟 (𝛽 𝑛 2
= ---------(6)
෌𝑖=1(𝑥𝑖 −𝑥)ҧ 𝑆𝑆𝑇𝑥

𝑛
𝜎 2 𝑛−1 ෌𝑖=1( 𝑥𝑖 )2
෢0 ) =
𝑉𝑎𝑟 (𝛽 𝑛 ---------(7)
෌𝑖=1(𝑥𝑖 −𝑥)ҧ 2

• Where these are conditional on the sample values {𝑥1 , 𝑥2 … 𝑥𝑛 }

• Conclusion:
• The sampling variability of the estimated regression coefficients will be higher, the
larger the variability of the unobserved factors, and lower with higher variation in
the explanatory variable.
 Sampling variances of the OLS estimators(contd)

෢1 ) = 𝜎2 𝜎2
𝑉𝑎𝑟 (𝛽 𝑛 2
= ---------(6)
ҧ
෌𝑖=1(𝑥𝑖 −𝑥) 𝑆𝑆𝑇𝑥

• Key things to note about the slope parameter:

෢1 )
❑ The larger the error variance, the larger is 𝑉𝑎𝑟 (𝛽

෢1
❑ The higher the variance of 𝑥𝑖 , the lower will be the variance of 𝛽

❑ The larger the sample size, larger the variance in x, Therefore, a larger sample size
෢1
results in a smaller variance for 𝛽
 Estimating Error Variance and
Precision Property of OLS Estimators
 Introduction
• Just like an estimator's unbiasedness is the number 1 quality we seek. Similarly, good
precision, or a low variance, is the "number 2 quality" we look for in an estimator

෢1 ) is given below:
• The variance of the slope estimator (𝛽
෢ 𝜎2
𝑉𝑎𝑟 (𝛽1 ) = 𝑛 ----------(1)
෌𝑖=1(𝑥𝑖 −𝑥)ҧ 2
• Problem:

𝜎 2 , which although constant by SLR 5 is unobserved as we never observed the errors

• Solution:

෢1 )
Use the data to estimate 𝜎 2 , which then allows us to estimate 𝑉𝑎𝑟 (𝛽
Difference between population error and sample residuals

• The equation of a population model in terms of a randomly sampled observations

𝑦𝑖 =𝛽0 + 𝛽1 𝑥𝑖 + μ𝑖 ----------(2)

where μ𝑖 is the error for observation 𝑖

• 𝑦𝑖 in terms of its fitted value and residual

෢0 + 𝛽
𝑦𝑖 = 𝛽 ෢1 𝑥𝑖 +𝑢ෝ𝑖 ----------(3)

μ, error in eq 2: Never observed

μො , residuals in eq3: computed from the data

 Estimating Error Variance
𝑦𝑖 =𝛽0 + 𝛽1 𝑥𝑖 + μ𝑖 ----------(2)

where μ𝑖 is the error for observation 𝑖

෢0 + 𝛽
𝑦𝑖 = 𝛽 ෢1 𝑥𝑖 +𝑢ෝ𝑖 ----------(3)

• Equating (2) and (3) we can write the sample residuals as a function of population errors
෢0 - 𝛽
μෝ𝑖 =𝑦𝑖 - 𝛽 ෢1 𝑥𝑖 −−−−−−− −(4)

෢0 - 𝛽
μෝ𝑖 = (𝛽0 + 𝛽1 𝑥𝑖 + μ𝑖 ) - 𝛽 ෢1 𝑥𝑖 ----------(5)

෢0 -𝛽0 ) – (𝛽
μෝ𝑖 = μ𝑖 - (𝛽 ෢1 - 𝛽1 ) 𝑥𝑖 ----------(6)
 Estimating Error Variance(contd.)
෢0 -𝛽0 ) – (𝛽
μෝ𝑖 = μ𝑖 - (𝛽 ෢1 - 𝛽1 ) 𝑥𝑖 ----------(6)

• μෝi is not the same as μi but expected value of the sample residuals is same as that the
expected value of errors

𝜎 2 = 𝐸[μ2 ] ----------(7)

• Unbiased estimator of 𝜎 2 is given below

𝑛
𝜎2 = −1
𝑛 ෌𝑖=1( μ𝑖 )2 ----------(8)
 Estimating Error Variance(contd.)
𝑛
𝜎 2 = 𝑛−1 ෌𝑖=1( μ𝑖 )2 ----------(8)

• Replacing the errors with the estimated values, i.e. the residuals in the sample

𝑛 𝑆𝑆𝑅
𝜎2 = 𝑛 ෌𝑖=1(μෝ𝑖 )2
−1 = ----------(9)
𝑛

• The above is biased in small sample and the bias can be removed by making adjustments
for the degrees of freedom

2 1 𝑛 𝑆𝑆𝑅
𝜎ො = 𝑛−2 ෌𝑖=1(μෝ𝑖 )2 =𝑛−2 ----------(10)
The Simple Regression Model
• Estimating the error variance

29
 Precision
• Standard error of the regression: 𝜎ො = 𝜎ො 2

• It gives an idea of how precise the estimator is

𝑛
𝜎 2 𝑛−1 ෌𝑖=1( 𝑥𝑖 )2
෢0 ) =
𝑉𝑎𝑟 (𝛽 𝑛
෌𝑖=1(𝑥𝑖 −𝑥)ҧ 2

෢1 ) = 𝜎2
𝑉𝑎𝑟 (𝛽 𝑛
෌𝑖=1(𝑥𝑖 −𝑥)ҧ 2

• σ
ෝ is used to to estimate the standard deviations of β ෢0 and β෢1 . It would just be the
෢0 ) and Var β
square root of the variances of the estimators Var (β ෢1

• The lower the standard error, the more precise estimates of a population parameter
The Simple Regression Model
• Theorem 2.3 (Unbiasedness of the error variance)

• Calculation of standard errors for regression coefficients

The estimated standard deviations of the regression coefficients are called “standard errors.”
They measure how precisely the regression coefficients are estimated.

31
Covariance and Correlation
Coefficient of determination and correlation coefficient
WRAP -UP