Classical Regression Analysis – Simple and

Multiple Regression Models – Model

Assumptions, Estimation and Interpretation
of Results

N Senthil Kumar
Member of Faculty, RBSC, Chennai
Multiple Regression Analysis
• Multiple Regression Involves more than one independent variables.
Example Yi=α + β1 X1i + β2X2i + εi. β1 , β2 are called partial regression
coefficients. β1 identifies change in the mean value of Y for a unit change
of X1, while holding X2.
• We continue to hold the 8 CNRLM assumptions.
• We add one more assumptions, i.e., no collinearity (perfect linear
relationship) among the dependent variables.
• As in simple linear regression, Our objective here is also to estimate the
conditional mean of Y for a given set of X1,X2 etc.
• As in simple linear regression, we use OLS method to arrive at the
formulae for the estimators. This is done by minimizing the sum of
squares of estimated errors. OLS Estimators are also Best Linear
Unbiased Estimators.
Multiple Regression Analysis –
Interval Estimation and
Hypothesis Testing
• Under the assumptions that εi
follows normal distribution, we
will have the βi s also follow
normal distributions.
• As we can estimate the Standard
Errors of the estimators of the
partial regression coefficients, we
will be in a position to arrive at
the confidence intervals for βi s.
• We can use t-statistic for each βi
to test whether the βi is
statistically different from zero.
• A significant test implies that the
concerned independent variable
is statistically significant when
other variables are held constant.
Testing the Overall Significance of Regression -
Analysis of Variance Approach
• Testing whether all the regressors are
simultaneously zero (statistically). i.e., to test
Ho : β1 = β2 = β3 …. βp = 0. This is called test
of overall significance of Regression equation.
• Testing of series of individual hypothesis is
not equivalent to testing them jointly as
confidence intervals are not independent.
• The joint hypothesis or the overall
significance can be tested by Analysis of
Variance (ANOVA) technique. Study of the
component of TSS (Total Sum of Squares) is
known as ANOVA from Regression point of
view.
• The F Ratio provides a test null hypothesis.
We need to just calculate the F ratio and
corresponding p-value. We Reject the Null if F
is larger above the critical value.
• R2 = ESS/TSS, = 1 – RSS/TSS. Adjusted R2 =
1 – [RSS/(n-p)]/[TSS/(n-1)]
• R2 tend to increase as we include more
independent variables. Therefore the
Adjusted R2 is a better goodness of fit
indicator.
Example

• bank <- [Link] ("[Link]", head=T)

• bank <- subset (bank, Year >= 2012)
• names (bank)
• # Estimating the model
• m1 <- lm (intincr ~ oc + unitcostemployee + cf
+ crar + brratio + cdratio + log(ast)+ factor
(BankGroup), data=bank )
• summary (m1)
> summary (m1)
Call:
lm(formula = intincr ~ oc + unitcostemployee + cf + crar + brratio +
cdratio + log(ast) + factor(BankGroup), data = bank)
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 3.553e-02 8.424e-03 4.217 3.20e-05 ***
oc -4.689e-05 7.227e-05 -0.649 0.516965
unitcostemployee 6.694e-01 1.366e-01 4.899 1.51e-06 ***
cf 5.782e-01 4.278e-02 13.515 < 2e-16 ***
crar 3.132e-05 2.356e-05 1.329 0.184648
brratio -2.562e-02 1.901e-02 -1.348 0.178683
cdratio 2.188e-05 1.055e-05 2.074 0.038844 *
log(ast) 3.903e-05 4.494e-04 0.087 0.930844
factor(BankGroup)3 3.073e-03 1.127e-03 2.726 0.006749 **
factor(BankGroup)4 3.531e-03 1.232e-03 2.866 0.004427 **
factor(BankGroup)5 -6.230e-03 1.627e-03 -3.829 0.000154 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.005853 on 331 degrees of freedom

Multiple R-squared: 0.7682, Adjusted R-squared: 0.7612
F-statistic: 109.7 on 10 and 331 DF, p-value: < 2.2e-16
Multiple Regression – Related Issues
Variable selection (Extra Sum of Squares)
• Extra sum of square method to quantify the contribution of a single variable to
regression sum of square.
• The extra sum of square of one or set of variables identified by 𝛽2 𝑔𝑖𝑣𝑒𝑛 𝑡ℎ𝑎𝑡 𝛽1 is
already present in the model is given by 𝑆𝑆𝑅 𝛽2 /𝛽1 = 𝑆𝑆𝑅 𝛽1 , 𝛽2 − 𝑆𝑆𝑅(𝛽1 )
• We can test the F Ratio, viz., partial F Test for the extra sum of square to determine
whether the variables 𝛽2 is significant to the model. The F ratio is given by 𝐹 =
𝑆𝑆𝑅 𝛽2 /𝛽1 /𝑟
.
𝑀𝑆𝑅𝑒𝑠
• The anova table provides the extra sum of square for each explanatory variable in the
order specified in the model
Variable selection (Extra Sum of Squares)
> anova (m1)
Analysis of Variance Table

Response: intincr
Df Sum Sq Mean Sq F value Pr(>F)
oc 1 0.0052848 0.0052848 154.2489 < 2.2e-16 ***
unitcostemployee 1 0.0057016 0.0057016 166.4164 < 2.2e-16 ***
cf 1 0.0243746 0.0243746 711.4307 < 2.2e-16 ***
crar 1 0.0000164 0.0000164 0.4772 0.49016
brratio 1 0.0000046 0.0000046 0.1337 0.71488
cdratio 1 0.0001826 0.0001826 5.3290 0.02159 *
log(ast) 1 0.0001219 0.0001219 3.5585 0.06012 .
factor(BankGroup) 3 0.0018983 0.0006328 18.4689 4.21e-11 ***
Residuals 331 0.0113405 0.0000343
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Multiple Regression – Related Issues
Testing of restrictions:
• Some time, we would want to test whether certain relationship (mainly
linear) among a sub set of coefficients holds good empirically as imposed
by priori theory. For example we would want to test 𝛽1 + 𝛽2 = 1 in Cobb
Douglas function.
(𝑅𝑆𝑆𝑅 −𝑅𝑆𝑆𝑈𝑅)/𝑚
• The F ratio is given by 𝐹 = where m is the number of
𝑅𝑆𝑆𝑈𝑅 /(𝑛−𝑝)
restrictions and p is the number of parameters in the equation, n is the
total number of observations.
Multiple Regression – Related Issues
Testing of restrictions:
#Restriction Testing
library (car)
library (lrmest)
linearHypothesis(m1, "oc + unitcostemployee + cf=1")
linearHypothesis(m1, [Link] = c(0,1,1,1,0,0,0,0,0,0,0),rhs = 1)

> linearHypothesis(m1, [Link] = c(0,1,1,1,0,0,0,0,0,0,0),rhs = 1)

Linear hypothesis test
Hypothesis:
oc + unitcostemployee + cf = 1
Model 1: restricted model
Model 2: intincr ~ oc + unitcostemployee + cf + crar + brratio + cdratio +
log(ast) + factor(BankGroup)
[Link] RSS Df Sum of Sq F Pr(>F)
1 332 0.011442
2 331 0.011340 1 0.00010115 2.9524 0.08668 .
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Multiple Regression – Related Issues
Estimation of restricted least square
# Restricted Least Square Estiation
library (lrmest)
rls (m1, R= c(0,1,1,1,0,0,0,0,0,0,0),r = 1) # not
working in this version
Multiple Regression – Related Issues
Testing for Structural Break (Chow Test)
• The restricted model assumes that the coefficients before and after the structural break
point remains equal. Therefore, it is the residual of the single equation applied on the
full data.
• The unrestricted model does not put this restriction that the coefficients are equal and
hence treat them from two population and independent and total RSS of unrestricted
model is given by 𝑅𝑆𝑆𝑈𝑅 = 𝑅𝑆𝑆1 + 𝑅𝑆𝑆2 .
(𝑅𝑆𝑆𝑅 −𝑅𝑆𝑆𝑈𝑅 )/𝑝
𝐹 = 𝑅𝑆𝑆
𝑈𝑅 /(𝑛1 +𝑛2 −2𝑝)

• The Chow test assumes that we know when structural took place and does not say the
structural break took place on account of intercept or coefficients.
• To test for homoscedastic variance, we test the F ratio as ratio of Residual Mean Squares
of both populations (convention is to out the bigger one as the numerator). Under Ho,
homoscedastic assumption, if the if the null is rejected, we may not be able to use the
Chow test
𝜎ො12
𝐹𝑛1 −1,𝑛2 −1 = 2
𝜎ො
Testing for Structural Break (Chow Test)
# Estimate Restricted sum of Square
bank1 <- subset (bank, Year <= 2014)
bank2 <- subset (bank, Year > 2014)
[Link] <- lm (intincr ~ oc + unitcostemployee + cf + crar + brratio + cdratio + log(ast)+ factor
(BankGroup), data=bank )
t <- anova([Link])
rss <- t$`Sum Sq`[9]

[Link] <- lm (intincr ~ oc + unitcostemployee + cf + crar + brratio + cdratio + log(ast)+ factor

(BankGroup), data=bank1 )
t <- anova([Link])
mse1 <- t$`Mean Sq` [9]
rss1 <- t$`Sum Sq`[9]

[Link] <- lm (intincr ~ oc + unitcostemployee + cf + crar + brratio + cdratio + log(ast)+ factor

(BankGroup), data=bank2 )
t <- anova([Link])
mse2 <- t$`Mean Sq` [9]
rss2 <- t$`Sum Sq`[9]

#testing for homoskedasticty

F <- mse1/mse2
pf (F, df1 = 170, df2=170)
# did not reject homskeasticty
denom <- (rss1 + rss2) / (nrow (bank1) + nrow (bank2) - 2 * 11)
numer <- (rss - rss1 + rss2)/11
F <- numer/denom
pf (F, df1 = 11, df2= nrow (bank1) + nrow (bank2) - 2 * 11)
# F did not reject
Part 2
Regression in matrix notation
Regression in Matrix Notations
𝑦 = 𝑋𝛽 + 𝑢 , E 𝑢/𝑋 = 0, Var 𝑢 = 𝜎 2 𝐼𝑛

𝑦1 𝑋11 𝑋12 𝑋1𝑘 𝛽1 𝑢1

𝑦2 𝑋21 𝑋22 𝑋2𝑘 𝛽2 𝑢2
𝑦= . , 𝑋= . . . , 𝛽 = . ,𝑢 = . ,
.. .. .. .. .. ..
𝑦𝑛 𝑋2𝑛 𝑋𝑛 𝑋𝑛𝑘 𝛽𝑘 𝑢𝑛

′
𝑅𝑆𝑆 = 𝑒 𝑒 = 𝑦 − 𝑋 𝛽 𝑦 − 𝑋𝛽መ
′ መ
𝛽መ𝑂𝐿𝑆 = 𝑋 ′𝑋 −1𝑋 ′𝑌 = 𝛽መ𝑂𝐿𝑆 = β + 𝑋 ′𝑋 −1𝑋 ′𝑢
𝑣𝑎𝑟 (𝛽መ𝑂𝐿𝑆) = 𝜎 2 𝑋 ′𝑋 −1
𝑦ො = 𝑋𝛽መ𝑂𝐿𝑆 = 𝑋 𝑋 ′𝑋 −1𝑋 ′𝑌 = 𝑃𝑌
𝑒 = 𝑦 − 𝑦ො = 𝐼 − 𝑋 𝑋 ′𝑋 −1𝑋 ′ 𝑌 = 𝑃𝑌ത
𝑣𝑎𝑟 𝛽መ𝑂𝐿𝑆 = 𝑋 ′𝑋 −1(𝑋 ′Σ𝑋) 𝑋 ′𝑋 −1 𝑖𝑓 𝑣𝑎𝑟 𝑢 = Σ
Generalized Least Squares - Estimator
Regression equation is 𝑦 = 𝑋𝛽 + 𝑢 and Var 𝑢 = 𝜎 2 Ω with
Ω positive definite matrix is assumed to be known. Hence Ω = 𝜎12 𝜎12 𝜎13 ......𝜎1𝑛
′
𝑃𝑃 where P is non – singular. Hence 𝜎21 𝜎22 . ......𝜎2𝑛
𝑃 −1 𝑦 = 𝑃 −1 𝑋𝛽 + 𝑃 −1 𝑢, which we can write as some Ω= 𝜎31 0. .. ............ 0.
𝑦 ∗ = 𝑋 ∗ 𝛽′+ 𝑢∗ , now 𝐸 𝑢∗ = 0 𝑎𝑛𝑑 𝑣𝑎𝑟 𝑢∗ = ′ .
𝑃 −1 𝑣𝑎𝑟 𝑢 𝑃 −1 which result in 𝑣𝑎𝑟 𝑢 ∗ = 𝜎 2 𝑃−1 Ω 𝑃−1 = . . . ...... .
′ .. . . . . 𝜎 2
2 −1
𝜎 𝑃 𝑃𝑃 𝑃 ′ −1 = 𝜎 2 𝐼𝑛 𝜎
𝜎𝑛1 𝑛2 . 𝑛
Hence the general
−1
transformation
2
needed is multiplying the
equation by 𝑃 to get a 𝜎 𝐼𝑛 error matrix and hence a BLUE
estimator
Hence the OLS estimator of 𝑦 ∗ = 𝑋 ∗ 𝛽 + 𝑢∗ is BLUE and
accordingly
−1 ′ −1 ′
𝛽መ𝐵𝐿𝑈𝐸 = 𝛽መ𝐺𝐿𝑆 = 𝑋 ∗ ′𝑋 ∗ 𝑋 ∗ ′𝑦 ∗ = 𝑋 ′𝑃 −1 𝑃 −1𝑋 𝑋 ′𝑃 −1 𝑃 −1𝑦 i.e.
መ ′ −1 −1 ′ −1 መ 2 ∗ ′ ∗ −1
𝛽𝐺𝐿𝑆 = 𝑋 Ω 𝑋 𝑋 Ω 𝑦 𝑎𝑛𝑑 𝑣𝑎𝑟 𝛽𝐺𝐿𝑆 = 𝜎 𝑋 𝑋 =
2 ′
𝜎 𝑋Ω 𝑋 −1 −1

Replacing 𝜎 2Ω by Σ, we get
−𝟏 ′ −𝟏 ∗ ′ ∗ −𝟏 −𝟏
෡ ′ −𝟏
𝜷𝑮𝑳𝑺 = 𝑿 𝜮 𝑿 𝑿 𝜮 𝒚 , 𝒗𝒂𝒓 𝜷𝑮𝑳𝑺 = 𝝈 𝑿 𝑿෡ 𝟐
= 𝑿𝜮 𝑿′ −𝟏
Generalized Least Squares – Heteroskedasticity and OLS Estimator
With, regression equation 𝑦 = 𝑋𝛽 + 𝑢 and Var 𝑢 = 𝜎 2Ω, we relax homoskedasticity and serial correlation
assumptions.
Heteroskedasticity Impact on the OLS Estimator
1. 𝛽መ𝑂𝐿𝑆 continues to be unbiased and consistent since these properties depends on E(u/X) =0, and plim
(X’u/n) = 0 and do not depend on the error covariance matrix.
2. However, 𝑣𝑎𝑟 𝛽መ𝑂𝐿𝑆 − 𝑣𝑎𝑟 𝛽መ𝐺𝐿𝑆 = 𝑋 ′𝑋 −1 (𝑋 ′Σ𝑋) 𝑋 ′𝑋 −1 − 𝑋 ′Σ −1𝑋 −1 is positive definite. Hence
𝛽መ𝐺𝐿𝑆 has lower variance than 𝛽መ𝑂𝐿𝑆
′
3. The 𝜎ො = 𝑦 − 𝑋𝛽𝑂𝐿𝑆 𝑦 − 𝑋𝛽መ𝑂𝐿𝑆 /(𝑛 − 𝑘) is no longer valid while the valid formula is
2 መ ൫𝑦 ∗ −
′ ′ −𝟏
∗ መ ∗ ∗ መ ෡
𝑋 𝛽𝐺𝐿𝑆 ൯ 𝑦 − 𝑋 𝛽𝐺𝐿𝑆 / 𝑛 − 𝑘 to be 𝒚 − 𝑿𝜷𝑮𝑳𝑺 𝜴 𝒚 − 𝑿𝜷 ෡ 𝑮𝑳𝑺 /(𝒏 − 𝒌)
Heteroskedasticity Impact on Inference
Now we notice, 𝛽መ𝑂𝐿𝑆 = 𝑋 ′𝑋 −1𝑋 ′𝑦, 𝑤ℎ𝑖𝑐ℎ 𝑐𝑎𝑛 𝑏𝑒 𝑒𝑥𝑝𝑟𝑒𝑠𝑠𝑒𝑑 𝑎𝑠 𝛽መ𝑂𝐿𝑆 = β + 𝑋 ′𝑋 −1 𝑋 ′𝑢,
Now, the actual variance is no longer 𝜎 2 𝑋 ′𝑋 −1and is given by 𝑣𝑎𝑟 𝛽መ𝑂𝐿𝑆 = 𝑋 ′𝑋 −1 (𝑋 ′Σ𝑋) 𝑋 ′𝑋 −1. Also
𝜎ො 2 estimated with OLS residuals is also incorrect. Hence under heteroskedasticity, using 𝜎 2 𝑋 ′𝑋 −1 testing
hypothesis is incorrect.
For heteroskedasticity robust standard error, assuming only heteroskedasticity, White (1980) used Σ෠ =
𝑑𝑖𝑎𝑔 𝑢ො 𝑖2 , 𝑢ො 𝑖 𝑂𝐿𝑆 𝑒𝑟𝑟𝑜𝑟𝑠.
Transformation for Heteroskedasticity and Serial Correlation – Special Cases
• Only heteroskedasticity, i.e., when Ω = 𝑑𝑖𝑎𝑔 𝜎𝑖2 ,then the Only Heteroskedasticity
∗ ∗ 𝑋𝑖𝑗 .. 0
transformation is 𝑦𝑖 = 𝑦𝑖 /𝜎𝑖 and 𝑋𝑖𝑗 = 𝜎 , 𝑗 = 1 𝑡𝑜 𝑘. 𝜎12 0 .. 0 1Τ𝜎1
𝑖 0 𝜎22.. 0 0 1Τ𝜎2.. 0
. .... 0
• However, assuming homoscedasticity 𝜎𝑢2, but serially Ω = 0. 0. .... 0. 𝑃 −1 = 0.
. .. .
correlated errors with, AR (1) i.e., 𝑢𝑡 = 𝜌𝑢𝑡−1 + 𝜀𝑡 we will . . .. . . 0 .
2 2 2 𝑠 2 0 ..𝜎𝑛2 0 ..1Τ𝜎𝑛
have 𝜎𝑢 = 𝜎𝜀 (1 − 𝜌 ) and 𝑐𝑜𝑣 𝑢𝑡 , 𝑢𝑡−𝑠 = 𝜌 𝜎𝑢 ., then the 0
transformation is 𝑦1∗ = 1 − 𝜌2 𝑦1 𝑎𝑛𝑑𝑦𝑡∗ = 𝑦𝑡 − 𝜌𝑦𝑡−1 Only Serial Correlation AR(1)
for t= 2 to T with similar transformation for X. This will 1 𝜌 𝜌2 ......𝜌𝑇−1
have 𝑢∗ with covariance matrix 𝜎𝜀2 𝐼𝑇 . 𝜌 1 . ......𝜌𝑇−2
Ω = 𝜌.
2 0. . ............ 0
• Feasible GLS Estimator : When Ω is unknown, we replace Ω . .
. . . .. . . . . .
෡
with its consistent estimator Ω to get the Feasible GLS 𝑇−1 𝜌 𝑇−2 ......
𝜌 . 1
መ ′ −1 −1
Estimator. We had 𝛽𝐺𝐿𝑆 = 𝑋 Ω 𝑋 𝑋 Ω 𝑦 and ′ −1
−1 ′ −1
መ ′ ෡ −1
therefore 𝛽𝐹𝐺𝐿𝑆 = 𝑋 Ω 𝑋 𝑋 Ω ෡ 𝑦. But 𝛽መ𝐹𝐺𝐿𝑆 is no 1 − 𝜌2 0 𝜌2...... 0 0
−𝜌 1 . ...... 0 0
longer BLUE but consistent. 𝑃 −1 = 0. −𝜌 . ............ 0. 0
. . .
. . . ...... 1 0
0 0 . ......−𝜌1
Restricted Least Square
Restriction on coefficient Expressed as 𝑅𝛽 = 𝑟
Lagrangian function for minimization, where 𝜇 is Lagrange multiplier.
𝜓 𝛽, 𝜇 = 𝑦 − 𝑋𝛽 ′ 𝑦 − 𝑋𝛽 + 2𝜇′ (𝑅𝛽 − 𝑟)
Note 𝜇(𝑔 ×1) , 𝑅(𝑔 ×𝑘) , 𝑟(𝑔 ×1) where g is number of restrictions.
We get
𝛽መ𝑅𝐿𝑆 = 𝛽መ𝑂𝐿𝑆 − 𝑋 ′ 𝑋 −1 𝑅′ 𝑅 𝑋 ′ 𝑋 −1 𝑅 ′ −1 𝑅𝛽መ𝑂𝐿𝑆 − 𝑟
𝜇Ƹ = 𝑅 𝑋 ′𝑋 −1𝑅 ′ −1 𝑅𝛽መ𝑂𝐿𝑆 − 𝑟
Note here Restricted OLS is biased if the restriction 𝑅𝛽 = 𝑟 is not true.
LM Test
Under the null hypothesis 𝐻𝑜 ∶ 𝑅𝛽 = 𝑟,the Lagrange Multiplier
𝜇Ƹ ~ 𝑁 ( 0, 𝜎 2 𝑅 𝑋 ′𝑋 −1 𝑅′ −1) which gives the LM statistic
′
𝜇Ƹ 𝑅 𝑋 𝑋 𝑅 𝜇Ƹ Τ𝜎 = 𝑅𝛽𝑂𝐿𝑆 − 𝑟 𝑅 𝑋 ′𝑋 −1 𝑅′ −1 𝑅𝛽መ𝑂𝐿𝑆 − 𝑟 Τ𝜎 2 ~ 𝜒2 (𝑔)
′ ′ −1 ′ 2 መ
Testing Restrictions (Hypothesis)
Likelihood Ratio Test
If 𝑋1 , 𝑋3 , 𝑋3 ... 𝑋𝑛 𝑎𝑟𝑒 𝑓𝑟𝑜𝑚 𝑁 (𝜇, 𝜎 2 ) , the likelihood ratio test for 𝐻𝑜 ∶ 𝜇 = 𝜇𝑜 𝑣𝑠 𝐻1 : 𝜇 ≠ 𝜇1 is given
max 𝐿𝑜
by 𝐿𝑅 = −2 log 𝜆 where the likelihood ratio 𝜆 = max 𝐿 and max 𝐿𝑜 is the maximum value of
likelihood under the Ho and max 𝐿 is the 𝑛
maximum likelihood elsewhere which is nothing but the
MLE. In this case, max 𝐿𝑜 = 1Τ2 2𝜋 exp − σ𝑛𝑖=1 𝑥𝑖 − 𝜇𝑜 2 /(2𝜎 2 ) and max 𝐿 =
𝑛
1Τ2 2𝜋 exp − σ𝑛𝑖=1 𝑥𝑖 − 𝜇ො𝑀𝐿𝐸 2/(2𝜎 2 ) 𝑤ℎ𝑒𝑟𝑒 𝜇ො𝑀𝐿𝐸 = 𝑥.ҧ From this, we get − 2 log( 𝜆) =
𝑥ҧ − 𝜇𝑜 2 /(𝜎 2Τ𝑛) ~ 𝜒2 1 .
Likelihood Ratio Test for Testing Restrictions
max 𝐿
𝑅
For the linear model to test the restriction 𝐻𝑜 ∶ 𝑅𝛽 = 𝑟 we define 𝜆 = where max 𝐿𝑅 is the
max 𝐿𝑈𝑅
maximum value of likelihood under the Ho and max 𝐿𝑈𝑅 is the maximum likelihood if there is no
𝑛 ′
restriction. Accordingly we have max 𝐿𝑅 = 1 2 2𝜋 exp − 𝑦 − 𝑋𝛽𝑅𝑀𝐿𝐸 𝑦 − 𝑋𝛽መ𝑅𝑀𝐿𝐸 /(2𝜎 2)
Τ መ
𝑛 ′
with 𝛽𝑅𝑀𝐿𝐸 = 𝛽𝑅𝑂𝐿𝑆 and max 𝐿𝑈𝑅 = 1Τ2 2𝜋 exp − 𝑦 − 𝑋 𝛽መ𝑈𝑅𝑀𝐿𝐸 𝑦 − 𝑋𝛽መ𝑈𝑅𝑀𝐿𝐸 /(2𝜎2 ) with
መ መ
𝛽መ𝑈𝑅𝑀𝐿𝐸 = 𝛽መ𝑂𝐿𝑆 , from which we get
− 2 log( 𝜆) = (𝑅𝑅𝑆𝑆 − 𝑈𝑅𝑆𝑆)/𝜎 2 which is reduced to the same LM statistic since 𝑅𝑅𝑆𝑆 −
′
𝑈𝑅𝑆𝑆 = 𝑅𝛽𝑂𝐿𝑆 − 𝑟 𝑅 𝑋 ′ 𝑋 −1 𝑅′ −1 𝑅𝛽መ𝑂𝐿𝑆 − 𝑟 , which under the null hypothesis is 𝜒2 (𝑔)
መ
Testing Restrictions
In practice, as we do not know, 𝜎 2 , we replace the same with MSE (Mean
Square Errors), and we shall have the F ratio as below:

(𝑅𝑅𝑆𝑆 − 𝑈𝑅𝑆𝑆)/𝜎 2 ~ 𝜒2 (𝑔)

𝑈𝑅𝑆𝑆/𝜎 2 ~ 𝜒2 (𝑛 − 𝑘)

(𝑅𝑅𝑆𝑆 − 𝑈𝑅𝑆𝑆)/𝑔
𝐹𝑔,𝑛−𝑘 =
𝑈𝑅𝑆𝑆/(𝑛 − 𝑘)
Testing Regression Stability (Chow Test)
• For an arbitrary set of time points and segment of population, the population is
divided into two groups with 𝑛1 , 𝑛2 observations. Chow test performs testing of
stability of regression coefficients in these two segment of population.
• Test 𝐻𝑜 𝛽1𝑎 = 𝛽1𝑏 , 𝛽2𝑎 = 𝛽2𝑏 , 𝛽3𝑎 = 𝛽3𝑏 ,… 𝛽𝑘𝑎 = 𝛽𝑘𝑏 ,
• K restrictions
𝑦1 𝑋1 0 𝛽 𝑎 𝑢1
• URSS arrived as by 𝑦 = 𝑏 + 𝑢 , i.e. run two regression
2 0 𝑋2 𝛽 2
𝑦1 𝑋1 𝑎 𝑢1
• RRSS is arrived by 𝑦 = 𝛽 + 𝑢
2 𝑋2 2
(𝑅𝑅𝑆𝑆 −𝑈𝑅𝑆𝑆)/𝑘
• Chow F ratio is 𝐹𝑘,𝑛1+𝑛2 −2𝑘 =
𝑈𝑅𝑆𝑆/(𝑛1+𝑛2 −2𝑘)
• Chow Predictive Test: when 𝑛1 > 𝑛2 , 𝑛2 < k, RRSS is based on full set single
regression, URSS is based on 𝑛1 observation, F ratio will be as below
(𝑅𝑅𝑆𝑆 − 𝑈𝑅𝑆𝑆)/𝑛2
𝐹𝑛2 ,𝑛1−𝑘 =
𝑈𝑅𝑆𝑆/(𝑛1 − 𝑘)
Thanks