Scribd
Upload
147 views2 pages

Optimal Alpha for Ridge and Lasso Regression

The document discusses optimal alpha values for ridge and lasso regression models. For ridge, the optimal alpha value is 10, and for lasso it is 0.001. Doubling the alpha values changes the most important predictor variables only slightly. Lasso regression is chosen over ridge because it allows for greater feature reduction. When the top 5 predictor variables from lasso are unavailable, the next most important variables are listed. For a model to be robust and generalizable, training and test accuracies should be similar regardless of the training data, and the model should not be influenced by outliers. This improves the model's accuracy on unknown data.

Uploaded by

richard samuel
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
147 views2 pages

Optimal Alpha for Ridge and Lasso Regression

The document discusses optimal alpha values for ridge and lasso regression models. For ridge, the optimal alpha value is 10, and for lasso it is 0.001. Doubling the alpha values changes the most important predictor variables only slightly. Lasso regression is chosen over ridge because it allows for greater feature reduction. When the top 5 predictor variables from lasso are unavailable, the next most important variables are listed. For a model to be robust and generalizable, training and test accuracies should be similar regardless of the training data, and the model should not be influenced by outliers. This improves the model's accuracy on unknown data.

Uploaded by

richard samuel
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Question 1

What is the optimal value of alpha for ridge and lasso regression? What will be the changes in the
model if you choose double the value of alpha for both ridge and lasso? What will be the most
important predictor variables after the change is implemented?

Answer: -

The best alpha value for Ridge regression is – 10.

For Lasso regression, the best alpha value is –0.001.

Ridge's most relevant predictive variables at the moment are:

{'OverallQual': 0.099, 'GrLivArea': 0.081, 'GarageCars': 0.074, 'FullBath': 0.063, '2ndFlrSF': 0.054,

Lasso’s most relevant predictive variables at the moment are:

{'GrLivArea': 0.27, 'OverallQual': 0.204, 'GarageCars': 0.106, 'FireplaceQu': 0.039, 'OverallCond':


0.035,

Changing the Model

Adjusting the optimal value of alpha for Ridge as – 20

Most important predictor variables are:-

{'OverallQual': 0.076, 'GarageCars': 0.063, 'GrLivArea': 0.061, 'FullBath': 0.056, '2ndFlrSF': 0.045,

Adjusting the optimal value of alpha for Lasso as – 0.002

Most important predictor variables are: -

{'GrLivArea': 0.27, 'OverallQual': 0.204, 'GarageCars': 0.106, 'FireplaceQu': 0.039, 'OverallCond':


0.035,

Question 2

You have determined the optimal value of lambda for ridge and lasso regression during the
assignment. Now, which one will you choose to apply and why?

Answer: -

Ridge will be good in general if all of our predictor factors are equally important.

If only a handful of the predictor variables are meaningful, Lasso would be a decent choice.

In this scenario, I would choose Lasso over Ridge because it allows for greater feature reduction than
Ridge, resulting in a more promising Model.

Question 3

After building the model, you realised that the five most important predictor variables in the lasso
model are not available in the incoming data. You will now have to create another model excluding
the five most important predictor variables. Which are the five most important predictor variables
now?

Answer: -

The following are the next most important factors discovered for Lasso Regression after eliminating
the current most important variables.

{'FullBath': 0.122, 'Fireplaces': 0.091, '2ndFlrSF': 0.078, 'KitchenQual': 0.07, 'BsmtQual': 0.057,

Question 4

How can you make sure that a model is robust and generalisable? What are the implications of the
same for the accuracy of the model and why?

Answer:

When a model meets the majority of the following criteria, it is said to be robust:

1. When the training and test accuracies are almost identical.

2. Even when given a different set of training data, the model's behaviour remains constant.

3. It is not influenced by outliers

Because it must predict well on unobserved data, a model must be robust and generalizable. If the
model is not generalizable and robust, it will act strangely when fed with unknown data and/or
when data contains outliers. One thing to keep in mind is that we will thoroughly examine data
outliers and eliminate those that are irrelevant. This will aid us in improving the model's accuracy.
The confidence intervals will also be used to normalise the model's predictions.

You might also like