Scribd
Upload
34 views2 pages

Chapter 1 Simple Linear Regression Model

The document summarizes key concepts in simple linear regression modeling. It introduces the basic linear regression model as yi = β1 + β2xi + ei, where yi is the dependent variable, xi is the independent variable, and ei is the error term. It describes how to estimate the intercept (β1) and slope (β2) parameters using ordinary least squares. It also discusses evaluating goodness of fit using R-squared and root mean squared error, and performing analysis of variance to partition total sum of squares into explained and residual sums of squares. Finally, it briefly mentions extending the model to handle non-linear relationships.

Uploaded by

Nermine Limeme
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
Download as pdf or txt
34 views2 pages

Chapter 1 Simple Linear Regression Model

The document summarizes key concepts in simple linear regression modeling. It introduces the basic linear regression model as yi = β1 + β2xi + ei, where yi is the dependent variable, xi is the independent variable, and ei is the error term. It describes how to estimate the intercept (β1) and slope (β2) parameters using ordinary least squares. It also discusses evaluating goodness of fit using R-squared and root mean squared error, and performing analysis of variance to partition total sum of squares into explained and residual sums of squares. Finally, it briefly mentions extending the model to handle non-linear relationships.

Uploaded by

Nermine Limeme
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
Download as pdf or txt
Download as pdf or txt
You are on page 1/ 2

Chapter 1 Simple Linear Regression Model

The general Model


Linear Regression’s used when dependent variable is numeric. Independent variables can be numeric or categorical.
The assumed relationship in a linear regression model has the form:
𝑦𝑖 = β1 + β2𝑥𝑖 + 𝑒𝑖
𝑦𝑖 Is the dependent variable (or Response variable)
𝑥𝑖 Is the independent variable (or Explanatory Variables)
𝑒𝑖 Is the error term
2
σ Is the variance of the error term?
β1 Is the intercept parameter or coefficient
β2 Is the slope parameter or coefficient

𝑦𝑖 ℎ𝑎𝑠 𝑡𝑤𝑜 𝑐𝑜𝑚𝑝𝑜𝑛𝑒𝑛𝑡𝑠: β1 + β2𝑥𝑖 𝑤ℎ𝑖𝑐ℎ 𝑖𝑠 𝑛𝑜𝑛. 𝑟𝑎𝑛𝑑𝑜𝑚 𝑎𝑛𝑑 𝑒𝑖 𝑤ℎ𝑖𝑐ℎ 𝑖𝑠 𝑎 𝑟𝑎𝑛𝑑𝑜𝑚 𝑐𝑜𝑚𝑝𝑜𝑛𝑒𝑛𝑡 .

The results of estimation if β1 and β2 using Ordinary Least Squares (OLS) method are:
^ ^
β1 = 𝑦 − β2. 𝑥
𝑁
∑ (𝑥𝑖−𝑥)(𝑦𝑖−𝑦)
𝑖=1
β2 = 𝑁 2
∑ (𝑥𝑖−𝑥)
𝑖=1

Goodness of Fit
To assess the performance of the predictive regression model, we use:
2
● 𝑅 : representing the squared correlation between the observed and the predicted values by the model.
⇨ The higher the R-square, the better the model. The closer it is to 1, the better the fit.
● Root Mean Squared Error (𝑅𝑀𝑆𝐸) measures he model prediction error
⇨ The lower the RMSE the better the model.

Analysis of Variance (ANOVA) and Sums of Squares


𝑁 𝑁 2 𝑁
2 ^ ^2
∑ (𝑦𝑖 − 𝑦) = ∑ (𝑦𝑖 − 𝑦) + ∑ 𝑒𝑖
𝑖=1 𝑖=1 𝑖=1

𝑇𝑆𝑆 = 𝐸𝑆𝑆 + 𝑅𝑆𝑆


Total Sum of Squares (TSS) = Explained Sum of Squares (ESS) + Residual Sum of Squares (RSS)
2
𝑅
𝑁 2 𝑁
^ ^2
∑ (𝑦𝑖−𝑦) ∑ 𝑒𝑖
2 𝑖=1 𝑖=1
𝑅 = 𝑁 2
=1− 𝑁 2
∑ (𝑦𝑖−𝑦) ∑ (𝑦𝑖−𝑦)
𝑖=1 𝑖=1

2 𝐸𝑆𝑆 𝑅𝑆𝑆
𝑅 = 𝑇𝑆𝑆
=1− 𝑇𝑆𝑆

2
⇨ The Closer 𝑅 is to 1 the better the fit.
RMSE
- Randomly split your data into training set (called𝑁1) and test set (called𝑁2).
- Build your Regression model using the training set
- Make predictions using the test set and compute the model accuracy metrics.

𝑁2 2
(
∑ 𝑦𝑖−𝑦𝑖
^
)
𝑅𝑀𝑆𝐸 = ⎛ 𝑖=1
⎞
𝑁2
⎝ ⎠
⇨ The lower the RMSE, the better the model.

Non-Linear Relationships
2
𝑦𝑖 = β1 + β2𝑥𝑖 + 𝑒𝑖
𝑑𝑦
The Marginal Effect: 𝑚𝑒 = 𝑑𝑥
= 2β2𝑥

𝑑𝑦 𝑥 𝑥
The Elasticity: ϵ = 𝑑𝑥 𝑦
= 𝑚𝑒 𝑦

𝑙𝑜𝑔(𝑦𝑖) = β1 + β2𝑥𝑖 + 𝑒𝑖

You might also like