Scribd
Upload
14 views25 pages

Logistic Regression Guide & Concepts

Logistic regression is a machine learning classification algorithm that uses an activation function called the sigmoid function to model the relationship between a categorical dependent variable and one or more independent variables. It estimates the probabilities of possible outcomes by fitting data to a logistic curve. The objective is to minimize a cost function, typically through gradient descent optimization, to find the optimal parameters that predict class membership. Some key applications of logistic regression include fraud detection, sentiment analysis, and disease prediction.

Uploaded by

Prudhvi Raj
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
14 views25 pages

Logistic Regression Guide & Concepts

Logistic regression is a machine learning classification algorithm that uses an activation function called the sigmoid function to model the relationship between a categorical dependent variable and one or more independent variables. It estimates the probabilities of possible outcomes by fitting data to a logistic curve. The objective is to minimize a cost function, typically through gradient descent optimization, to find the optimal parameters that predict class membership. Some key applications of logistic regression include fraud detection, sentiment analysis, and disease prediction.

Uploaded by

Prudhvi Raj
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd

Logistic

Regression
Regression analysis is a statistical
method to model the relationship
What is between the dependent and
independent variables.
Regression
Regression estimates the relationship
between the target and the independent
variable

It is used to find the trends in data


Why is
It helps to predict real/continuous
values
Regression
used
For determining the most important
and least important factors
Loss function
Important 1
A function that is defined on a
single data instance is called

concepts of Loss function

Regression 2
Cost function
A function that is defined on an
entire data instance is called the
Cost function

y = a_0 + a_1 * x
Gradient Descent
3
Briefly elaborate on what you
want to discuss.
• A cost function is a measure of how wrong the
model is in terms of its ability to estimate the
relationship between dependent and
independent variables

• This is typically expressed as a difference or


distance between the predicted value and the Cost function
actual value

• The objective of a ML model, is to find


parameters, weights that minimises the cost
function
Types of cost functions

Regression Classification Multi - class


classification
• Mean Error • Cross Entropy
• Categorical cross Entropy
• Mean Square Error • Binary cross Entropy
• Mean Absolute Error
Minimizing the Cost function :
Gradient Descent

• Gradient descent enables a model to learn the gradient or direction that the model
should take in order to reduce errors.

• Gradient descent is an efficient optimization algorithm that attempts to find a local or


global minima of a function.

• Direction in the simple linear regression example refers to how the model parameters
a_0 and a_1 should be corrected to further reduce the cost function.

• As the model iterates, it gradually converges towards a minimum which is also called as
convergence.
Gradient Descent
Effect of learning rate in gradient descent
3D plot
What is
Logistic
Regression
Logistic regression is a supervised learning classification

Definition algorithm. It is a predictive analysis algorithm and based on the


concept of probability.
Assumptions

No
Target variable Variables Sample size
multi-collinearity
The target variable must We must add meaningful Choose a large sample
The independent
always be binary variables in our model size for logistic
variables must be
regression
independent of each
other
1 2

Target variable Curve


The Logistic regression is In Logistic regression, we
used to predict the categorical find the S-curve by which we
dependent variables can classify the samples
What makes it
different 3 4

Cost function Accuracy


A more complex cost Maximum likelihood
function is used estimation method is used for
estimation of accuracy
Types of Logistic Regression

• Binomial • Multinomial • Ordinal

The Dependent variable will The dependent variable The dependent variable
have only two possible can have 3 or more can have 3 or more
types either 1 and 0 possible unordered possible ordered types
types
• The activation function used in the Logistic
regression is sigmoid function.

Sigmoid • Sigmoid function maps any real value into

function
another value between 0 and 1.

• Sigmoid function is used to map predictions to


probabilities.
σ(x) = 1/(1+exp(-x))
In Linear
Regression
h(x) = β₀ + β₁X

In Logistic
Regression
σ(Z) = σ(β₀ + β₁X)
Cost function in Logistic
Regression

Gradient Descent
Decision Boundary
Implementation of Logistic Regression
Hyperparameter
Tuning
Applications of Logistic Regression

• Fraud detection • Object Detection


• Sentiment Analysis • Handwriting recognition
• Image segmentation • Disease Prediction
Thank you!

You might also like