ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 1) M.G.

Abbott

Regression Analysis: Terminology and Notation

Consider the generic version of the simple (two-variable) linear regression model.

It is represented by the following population regression equation (called the PRE

for short):

Yi = f (X i ) + u i = β 0 + β1X i + u i

• The PRF (population regression function):

f ( X i ) = β 0 + β1X i
= the i-th value of the population regression function (PRF).

• Observable Variables:

Yi ≡ the i-th value of the dependent variable Y

Xi ≡ the i-th value of the independent variable X

• Unobservable Variable:
ui ≡ the random error term for the i-th member of the population

• Unknown Parameters: the regression coefficients

β0 = the intercept coefficient
β1 = the slope coefficient on Xi

The true population values of the regression coefficients β0 and β1 are unknown.

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 1 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 2) M.G. Abbott

Variables and Parameters

PRE: Yi = f ( X i ) + u i = β 0 + β1X i + u i

• The variables of the regression model are Yi, Xi, and ui.

Yi and Xi are the observable variables; their values can be observed or measured.

Yi is called any of the following: (1) the dependent variable

(2) the regressand
(3) the explained variable.

Xi is called any of the following: (1) the independent variable

(2) the regressor
(3) the explanatory variable.

• ui is an unobservable random variable; its value cannot be observed or

measured. It is called a random error term.

• β0 and β1 are the parameters of the regression model, together with any unknown
parameters of the probability distribution of the random error term ui.

β0 and β1 are called regression coefficients; in particular,

β0 ≡ the intercept coefficient,

and
β1 ≡ the slope coefficient of X.

The true population values of the regression coefficients β0 and β1 are unknown.

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 2 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 3) M.G. Abbott

Simple Regression versus Multiple Regression

• A simple regression model has only two observable variables:

(1) one dependent variable or regressand Yi;

(2) one independent variable or regressor Xi.

• A multiple regression model has three or more observable variables:

(1) one dependent variable or regressand Yi;

(2) two or more independent variables or regressors X1i, X2i, ..., Xki, where

Xji ≡ the i-th value of the j-th regressor Xj (j = 1, 2, …, k).

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 3 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 4) M.G. Abbott

The Simple Linear Regression Model

• The PRE (population regression equation) for the simple linear regression
model:

Yi = f (X i ) + u i = β0 + β1X i + u i (1a)
↑ ↑
PRF random error
f ( X i ) = β 0 + β1X i
= the PRF (population regression function) for the i-th population member
u i = Yi − f ( X i ) = Yi − β 0 − β1X i
= the random error for the i-th population member
β 0 , β1 = the unknown regression coefficients β0 and β1
Number of regression coefficients = K = 2.
Number of slope coefficients = K − 1 = 2 − 1 = 1.

• Sample Data: A random sample of N members of the population for which the
observed values of Y and X are measured. Each sample observation is of the form

(Yi, Xi), i = 1, ..., N

• The SRE (sample regression equation) for the simple linear regression model:

Yi = f̂ (X i ) + û i = Ŷi + û i = βˆ 0 + βˆ 1X i + û i (1b)
↑ ↑
SRF residual

f̂ (X i ) = Ŷi = βˆ 0 + βˆ 1X i
= the SRF (sample regression function) for sample observation i
û i = Yi − f̂ (X i ) = Yi − Ŷi = Yi − βˆ 0 − βˆ 1X i
= the residual for sample observation i
β 0 , β1 = estimators or estimates of the regression coefficients β0 and β1

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 4 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 5) M.G. Abbott

The Multiple Linear Regression Model

• The PRE (population regression equation) for the multiple linear regression
model is:

Yi = f (X1i , X 2i , K, X ki ) + u i = β0 + β1X1i + β2 X 2i + L + βk X ki + u i (2a)

↑ ↑
PRF random error
f ( X1i , X 2i , K , X ki ) = β 0 + β1X1i + β 2 X 2i + L + β k X ki
= the PRF (population regression function) for the i-th population member
u i = Yi − f ( X1i , X 2i , K, X ki ) = Yi − β 0 − β1X1i − β 2 X 2i − L − β k X ki
= the random error for the i-th population member
β 0 , β1 , β 2 , K , β k = the unknown regression coefficients β0, β1, β2, …, βk
Number of regression coefficients = K.
Number of slope coefficients = k = K − 1.

• Sample Data: A random sample of N members of the population for which the
observed values of Y and X1, X2, …, Xk are measured. Each sample observation
is of the form

(Yi, X1i, X2i, …, Xki), i = 1, ..., N

• The SRE (sample regression equation) for the multiple linear regression model:

Yi = f̂ (X1i , X 2i , K , X ki ) + û i = Ŷi + û i = βˆ 0 + βˆ 1X1i + βˆ 2 X 2i + L + βˆ k X ki + û i (2b)

↑ ↑
SRF residual
f̂ (X1i , X 2i , K, X ki ) = Ŷi = βˆ 0 + βˆ 1X1i + βˆ 2 X 2i + L + βˆ k X ki
= the SRF (sample regression function) for sample observation i
û i = Yi − f̂ (X1i , X 2i , K, X ki ) = Yi − Ŷi = Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i − L − βˆ k X ki
= the residual for sample observation i
βˆ 0 , βˆ 1 , βˆ 2 , K, βˆ k = estimators or estimates of the regression coefficients

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 5 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 6) M.G. Abbott

• Examples of multiple regression models

♦ A three-variable linear regression model has two regressors; its PRE is written
as

Yi = β 0 + β1X1i + β 2 X 2i + u i .

Total number of regression coefficients = K = 3

Number of slope coefficients = k = K − 1 = 3 − 1 = 2

♦ A four-variable linear regression model has three regressors; its PRE is written
as

Yi = β 0 + β1X1i + β 2 X 2i + β3 X 3i + u i .

Total number of regression coefficients = K = 4

Number of slope coefficients = k = K − 1 = 4 − 1 = 3

♦ The general multiple linear regression model has K − 1 regressors; its PRE is
written as

Yi = β 0 + β1X1i + β 2 X 2i + L + β k X ki + u i .

Total number of regression coefficients = K.

Number of slope coefficients = k = K − 1.

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 6 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 7) M.G. Abbott

Regression Analysis: A Hypothetical Numerical Example

Reference: D. Gujarati (1995), Chapter 2, pp. 32-36.

Purpose: To illustrate some of the basic ideas of linear regression analysis.

The Model: A simple consumption function representing the relationship between

Yi ≡ the weekly consumption expenditure of family i ($ per week);

Xi ≡ the weekly disposable (after-tax) income of family i ($ per week);

The PRE (population regression equation) for this model can be written as

Yi = β 0 + β1X i + u i (1)

The Population: consists entirely of 60 families.

We assume that the weekly disposable incomes of these families take only 10
distinct values -- i.e., X takes only the 10 distinct values

Xi = 80, 100, 120, 140, 160, 180, 200, 220, 240, 260.

We further assume that we can observe the entire population of 60 families.

The data for the complete population is given in Table 2.1.

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 7 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 8) M.G. Abbott

Table 2.1: Population data points (Yi, Xi) for the population of 60 families.

Xi values → 80 100 120 140 160 180 200 220 240 260
Yi values ↓ 55 65 79 80 102 110 120 135 137 150
60 70 84 93 107 115 136 137 145 152
65 74 90 95 110 120 140 140 155 175
70 80 94 103 116 130 144 152 165 178
75 85 98 108 118 135 145 157 175 180
-- 88 -- 113 125 140 -- 160 189 185
-- -- -- 115 -- -- -- 162 -- 191
Sum Yi values 325 462 445 707 678 750 685 1043 966 1211
Number of Yi 5 6 5 7 6 6 5 7 6 7

• Interpretation of Table 2.1:

Each column of Table 2.1 represents the population conditional distribution of

Y (families’ weekly consumption expenditure) for the corresponding value of X
(families’ weekly disposable income).

♦ The first column gives the conditional distribution of Y for Xi = 80; five
families in the population have weekly disposable income equal to 80 dollars.

♦ The fifth column gives the conditional distribution of Y for Xi = 160; six
families in the population have weekly disposable income equal to 160 dollars.

♦ The tenth (last) column gives the conditional distribution of Y for Xi = 260;
seven families in the population have weekly disposable income equal to 260
dollars.

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 8 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 9) M.G. Abbott

Table 2.2: Population conditional probabilities of Y for each population value

of X.

• Notation:

p( Y X i ) = p(Yj X i ) = the conditional probability of Y for X = Xi

= the probability that the random variable Y takes the numerical

value Yj given that the variable X is equal to the numerical value Xi.

Conditional probabilities p( Y X i ) for the population data in Table 2.1

Xi values → 80 100 120 140 160 180 200 220 240 260
p( Y X i ) ↓ 1/5 1/6 1/5 1/7 1/6 1/6 1/5 1/7 1/6 1/7
1/5 1/6 1/5 1/7 1/6 1/6 1/5 1/7 1/6 1/7
1/5 1/6 1/5 1/7 1/6 1/6 1/5 1/7 1/6 1/7
1/5 1/6 1/5 1/7 1/6 1/6 1/5 1/7 1/6 1/7
1/5 1/6 1/5 1/7 1/6 1/6 1/5 1/7 1/6 1/7
-- 1/6 -- 1/7 1/6 1/6 -- 1/7 1/6 1/7
-- -- -- 1/7 -- -- -- 1/7 -- 1/7
Sum Yi values 325 462 445 707 678 750 685 1043 966 1211
Number of Yi 5 6 5 7 6 6 5 7 6 7
E( Y X i ) 65 77 89 101 113 125 137 149 161 173

• Interpretation of Table 2.2:

Each column of Table 2.2 contains the population conditional probabilities of

Y (families’ weekly consumption expenditure) for the corresponding value of X
(families’ weekly disposable income).

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 9 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 10) M.G. Abbott

Examples: Computing the Conditional Probabilities of Indivdual Y Values

1. Consider the column corresponding to Xi = 80.

There are five different values of Y for Xi = 80: .

Y | Xi = 80: 55, 60, 65, 70, 75.

The probability of observing any one family whose weekly disposable income
is Xi = 80 equals 1/5: e.g.,
1
p(Y = 55 | Xi = 80) = .
5
1
p(Y = 60 | Xi = 80) = .
5
1
p(Y = 65 | Xi = 80) = .
5
1
p(Y = 70 | Xi = 80) = .
5
1
p(Y = 75 | Xi = 80) = .
5

2. Consider the column corresponding to Xi = 160.

There are six different values of Y for Xi = 160: .

Y | Xi = 160: 102, 107, 110, 116, 118, 125.

The probability of observing any one family whose weekly disposable income
is Xi = 160 equals 1/6: e.g.,

1
p(Y = 102 | Xi = 160) = .
6
1
p(Y = 110 | Xi = 160) = .
6

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 10 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 11) M.G. Abbott

Population Conditional Means of Y

For each of the 10 population values of Xi, we can compute from Tables 2.1 and 2.2
the corresponding conditional mean value of the population values of Y.

For each of the values Xi of X, the population mean value of Y is called

(1) the population conditional mean of Y

or
(2) the population conditional expectation of Y.

• Notation:

E( Y X i ) = E( Y X = X i )
= the population conditional mean of Y for X = Xi
= the “expected value of Y given that X takes the specific value Xi"

• Definition:

E ( Y X i ) = E ( Y X = X i ) = ∑ p( Y X i ) Y
X =Xi

where

p( Y X i ) = the conditional probability of Y when X = Xi;

p( Y X i ) Y = the product of each population value of Y and its

corresponding conditional probability for X = Xi.

In words, the above formula for E( Y X i ) = E( Y X = X i ) says that for the value
Xi of X,

(1) multiply each population value of Y by its associated conditional probability

p( Y X i ) to get the product p( Y X i ) Y
(2) then sum these products p( Y X i ) Y over all the population values of Y
corresponding to X = Xi.

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 11 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 12) M.G. Abbott

• Illustrative Calculations of E( Y X i ) :

1. For Xi = 80, p( Y X i ) = 1/5:

E ( Y X i = 80 ) =
1 1 1 1 1
55 + 60 + 65 + 70 + 75
5 5 5 5 5
55 + 60 + 65 + 70 + 75
=
5
325
=
5
= 65

2. For Xi = 160, p( Y X i ) = 1/6:

E( Y X i = 160 ) =
1 1 1 1 1 1
102 + 107 + 110 + 116 + 118 + 125
6 6 6 6 6 6
102 + 107 + 110 + 116 + 118 + 125
=
6
678
=
6
= 113

3. For Xi = 260, p( Y X i ) = 1/7:

E( Y X i = 260 ) =
1 1 1 1 1 1 1
150 + 152 + 175 + 178 + 180 + 185 + 191
7 7 7 7 7 7 7
150 + 152 + 175 + 178 + 180 + 185 + 191
=
7
1211
=
7
= 173

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 12 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 13) M.G. Abbott

Table 2.3: Population Conditional Means of Y

Table 2.3
Xi E( Y X i )
80 65
100 77
120 89
140 101
160 113
180 125
200 137
220 149
240 161
260 173

• Interpretation of Table 2.3:

Table 2.3 tabulates the relationship between E( Y X i ) and X i for this particular
population of 60 families.

This population relationship between E( Y X i ) and X i is called either

(1) the population regression function, or PRF.

or
(2) the population conditional mean function, or population CMF

So Table 2.3 is a tabular representation of the PRF for the population of 60

families.

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 13 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 14) M.G. Abbott

Properties of the Population Regression Function, or PRF:

Table 2.3
Xi E( Y X i )
80 65
100 77
120 89
140 101
160 113
180 125
200 137
220 149
240 161
260 173

1. E( Y X i ) is a function of Xi: i.e., E( Y X i ) = f (X i ) .

2. E( Y X i ) is an increasing function of Xi: i.e.,

∆X i > 0 ⇒ ∆E( Y X i ) > 0 and ∆X i < 0 ⇒ ∆E( Y X i ) < 0 .

3. E( Y X i ) is a linear function of Xi: i.e.,

• A plot of the 10 points in Table 2.3 lie on a straight line.

• Each 20-dollar increase in X induces a constant 12-dollar increase in

E( Y X i ): i.e.,

∆E( Y X i ) 12
∆X i = 20 ⇒ ∆E( Y X i ) = 12 ⇒ = = 0.60.
∆X i 20

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 14 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 15) M.G. Abbott

4. The population regression function (PRF) -- also called the population

conditional mean function -- takes the general linear form

E( Y X i ) = β0 + β1X i .

5. The population values of the regression coefficients β1 and β2 for this

hypothetical population of 60 families are:

β 0 = 17 and β1 = 0.60 .

6. The population regression function, or PRF, for this particular population of

60 families is therefore

E( Y X i ) = β0 + β1X i = 17 + 0.60 X i .

‰ Summary -- The Population Regression Function (PRF)

The PRF, or population regression function, for this hypothetical population of

60 families is a linear function of the population values Xi of the regressor X; it
takes the form

f (X i ) = E( Y X i ) = β0 + β1X i = 17 + 0.60 X i .

where

β0 = 17 is the population value of the intercept coefficient

β1 = 0.60 is the population value of the slope coefficient of Xi.

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 15 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 16) M.G. Abbott

• Figure 2.1 Plot of Population Data Points, Conditional Means E(Y|X), and
the Population Regression Function PRF

Y Fitted values

200

PRF =
Weekly consumption expenditure, $

175
E(Y|X)
150

125

100

75

50

60 80 100 120 140 160 180 200 220 240 260

W eekly income, $

1. The small dots in Figure 2.1 constitute a scatterplot of the population values
of Y and X for the population of 60 families:
Each small dot corresponds to a single population data point of the form
(Yi, Xi) i = 1, 2, ..., 60.

2. The solid line in Figure 2.1 is the population regression line for the
population of 60 families.
Each pair of population values of ( E (Y | X i ), X i ) , is represented by a large
square dot in Figure 2.1.
This population regression line is the locus of the 10 points in Table 2.3 -- i.e.,
it connects the 10 points of the form ( E (Y | X i ), X i ) , i = 1, ..., 10.

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 16 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 17) M.G. Abbott

The Random Error Terms

• Definition: The unobservable random error term for the i-th population
member is denoted as ui and defined as

u i = Yi − E( Y X i ) ∀ i.

For each population member -- for each of the 60 families in our hypothetical
population -- the random error term ui equals the deviation of that population
member's individual Yi value from the population conditional mean value of Y for
the corresponding value Xi of X.
Terminology: The random error term ui is also known as the stochastic error term,
the random disturbance term, or the stochastic disturbance term

• Implication 1: By simple re-arrangement of the above definition of ui, it is

obvious that each individual population value Yi of Y can be written as

Yi = E( Y X i ) + u i
= β 0 + β1X i + u i since E( Y X i ) = β0 + β1X i .

This equation is called the population regression equation, or PRE.

Interpretation: The PRE indicates that each population value Yi of Y can be

expressed as the sum of two components:

(1) E( Y X i ) = β0 + β1X i
= the population conditional mean of Y for X = Xi
= the mean weekly consumption expenditure for all families in
the population who have weekly disposable income X = Xi.

(2) u i = the random error term for the i-th population member
= Yi − E( Y X i )
= the deviation of family i’s weekly consumption expenditure Yi
from the population mean value E(Y | Xi) of all families in the
population that have the same weekly disposable income X = Xi.

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 17 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 18) M.G. Abbott

Implication 2: The population conditional mean value of the random error terms for
each population value Xi of X equals 0 -- i.e.,

E( u i X i ) = 0 ∀ i.

Proof:

1. Take the conditional expectation for X = Xi of both sides of the PRE:

E(Yi X i ) = E[E(Y X i )] + E(u i X i )

= E(Y X i ) + E(u i X i ) since E(Y X i ) is a constant.

2. Since E( Yi X i ) = E( Y X i ) , the above equation implies that E( u i X i ) = 0 .

• What do the Random Error Terms ui Represent?

The random error terms represent all the unknown and unobservable
variables other than X that determine the individual population values Yi of
the dependent variable Y.

They arise from the following factors:

1. Omitted variables that determine the population Yi values

2. Intrinsic randomness in individual behaviour

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 18 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 19) M.G. Abbott

• Random Errors for Hypothetical Population of 60 Families

Random Error Terms for Xi = 100

Yi E ( Y | X i = 100) u i = Yi − E ( Y | X i = 100)
65 77 −12
70 77 −7
74 77 −3
80 77 3
85 77 8
88 77 11
Sum = 462 Sum = 0
Mean = 462/6 = 77 Mean = 0

Random Error Terms for Xi = 180

Yi E ( Y | X i = 180) u i = Yi − E (Y | X i = 180)
110 125 −15
115 125 −10
120 125 −5
130 125 5
135 125 10
140 125 15
Sum = 750 Sum = 0
Mean = 750/6 = 125 Mean = 0

Random Error Terms for Xi = 240

Yi E ( Y | X i = 240) u i = Yi − E ( Y | X i = 240)
137 161 −24
145 161 −16
155 161 −6
165 161 4
175 161 14
189 161 28
Sum = 966 Sum = 0
Mean = 966/6 = 161 Mean = 0

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 19 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 20) M.G. Abbott

The Sample Regression Function

• Important Point 1: Since in practice we do not observe the entire relevent

population, and never know the true PRF, we must estimate the PRF from
sample data.

• Objective of Regression Analysis: To estimate the PRF (population regression

function) from sample data consisting of N randomly selected observations (Xi,
Yi), i = 1, ..., N taken from the population.

• Form of the Sample Regression Function (SRF): The sample regression

function, or SRF, takes the general form

Ŷi = βˆ 0 + βˆ 1X i (i = 1, ..., N)

where

$ = an estimate of the PRF, f (X ) = E (Y | X ) = β + β X ;

Yi i i i 0 1 i

β̂ 0 = an estimate of the intercept coefficient β0;

β$ 1 = an estimate of the slope coefficient β1.

• Nature of the Sample Data: A sample is a randomly-selected subset of

population members.

1. The sample observations {(Yi, Xi): i = 1, ..., N} are typically a small subset of
the parent population of all population data points (Yi, Xi).

Sample size N is much smaller than the number of population data points.

2. Each random sample from a given population yields one estimate of the PRF
-- i.e., one estimate of the numerical value of β0, and one estimate of the
numerical value of β1.

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 20 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 21) M.G. Abbott

• Important Point 2: Each random sample from the same population yields a
different SRF -- i.e., a different numerical value of β̂ 0 , and a different numerical
value of β$ .
1

Example: Consider two random samples of 10 observations from the

population of 60 families. Each sample consists of one family for each of the 10
different population values of X.

Tables 2.4 and 2.5

Sample 1 Sample 2
Xi Yi Xi Yi
80 70 80 55
100 65 100 88
120 90 120 90
140 95 140 80
160 110 160 118
180 115 180 120
200 120 200 145
220 140 220 135
240 155 240 145
260 150 260 175

Because the two samples contain different Yi values for the 10 Xi values, they
will yield different SRFs -- a different numerical value of β̂ 0 , and a different
numerical value of β$ .
1

• Sample 1 SRF (SRF1): $ = 24.46 + 0.5091X ,

Yi i

where the Sample 1 coefficient estimates are β̂ 0 (1) = 24.46 and β$ 1 (1) = 0.5091

• Sample 2 SRF (SRF2): $ = 17.17 + 0.5761X ,

Yi i

where the Sample 2 coefficient estimates are β̂ 0 (2) = 17.17 and β$ 1 (2) = 0.5761

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 21 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 22) M.G. Abbott

• Figure 2.2 Plot of Sample Data Points and Sample Regression Functions
for Random Samples 1 and 2

SRF1 is the SRF based on Sample 1: $ = 24.46 + 0.5091X

Yi i

SRF2 is the SRF based on Sample 2: $ = 17.17 + 0.5761X

Yi i

SRF1 is the flatter regression line, SRF2 is the steeper regression line.

Important Points:

(1) Neither of these SRFs is identical to the true PRF. Each is merely an
approximation to the true PRF.

(2) How good an approximation any SRF provides to the true PRF depends on
how the SRF is constructed from sample data -- i.e., on the properties of the
coefficient estimators β̂ 0 and β$ 1 .

Y1 Y2

200
Weekly consumption expenditure, $

175

150

125

100

75

50

60 80 100 120 140 160 180 200 220 240 260

W eekly income, $

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 22 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 23) M.G. Abbott

The Sample Regression Equation (SRE)

• The sample regression equation (SRE) is the sample counterpart of the

population regression equation (PRE)

Yi = f (X i ) + u i = E(Yi X i ) + u i = β0 + β1X i + u i ⇐ the PRE

• Form of the Sample Regression Equation (SRE): The sample regression

equation, or SRE, takes the general form

Yi = Ŷi + û i = βˆ 0 + βˆ 1X i + û i (i = 1, ..., N) ⇐ the SRE

where

Ŷi = βˆ 0 + βˆ 1X i = an estimate of the PRF, f (X i ) = E (Yi | X i ) = β 0 + β1X i ;

β̂ 0 = an estimate of the intercept coefficient β0;
β$ 1 = an estimate of the slope coefficient β1.
u$ i = the residual for sample observation i.

• Interpretation of the SRE: The SRE represents each sample value of Y -- each Yi
value -- as the sum of two components:

(1) the estimated (or predicted) value of Y for each sample value Xi of X, i.e.,

Ŷi = βˆ 0 + βˆ 1X i (i = 1, ..., N);

(2) the residual corresponding to the i-th sample observation, i.e.,

û i = Yi − Ŷi = Yi − βˆ 0 − βˆ 1X i (i = 1, ..., N).

û i = the residual for the i-th sample observation

= the observed Y-value ( Yi ) − the estimated Y-value ( Ŷi )

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 23 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 24) M.G. Abbott

Compare the Population and Sample Regression Equations: the PRE and SRE

• The PRE for Yi is:

Yi = f (X i ) + u i = E(Yi X i ) + u i = β0 + β1X i + u i

• The SRE for Yi is:

Yi = Ŷi + û i = βˆ 0 + βˆ 1X i + û i

• Figure 2.3: Comparison of Population and Sample Regression Lines

(Yi, Xi)
Yi • SRF

Ŷi PRF

E (Y | X i )

Xi X

♦ The population regression line is a plot of the PRF: E(Yi X i ) = β0 + β1X i .

♦ The sample regression line is a plot of the SRF: Ŷi = βˆ 0 + βˆ 1X i .

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 24 of 25

ECON 351* -- Section 1: Basic Concepts of Regression Analysis (Page 25) M.G. Abbott

• Figure 2.3: Comparison of Population and Sample Regression Lines

(Yi, Xi)
Yi • SRF

Ŷi PRF

E (Y | X i )

Xi X
At X = Xi:

♦ The population regression equation (PRE) represents the population value Yi of Y

as the sum of two parts:

Yi = E(Yi X i ) + u i = β 0 + β1X i + u i , where E(Yi X i ) = β0 + β1X i

u i = Yi − E(Yi X i ) = Yi − β0 − β1X i = distance between Yi and E(Yi X i )

♦ The sample regression equation (SRE) represents the population value Yi of Y as

the sum of two parts:

Yi = Ŷi + û i = βˆ 0 + βˆ 1X i + û i , where Ŷi = βˆ 0 + βˆ 1X i

û i = Yi − Ŷi = Yi − βˆ 0 − βˆ 1X i = distance between Yi and Ŷi .

ECON 351* -- Section 1: Fileid 351lec02.doc ... Page 25 of 25