Scribd
Upload
10 views

Machine Learning-Lecture 04

Uploaded by

Amna Arooj
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
10 views

Machine Learning-Lecture 04

Uploaded by

Amna Arooj
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 31

Machine Learning

LECTURE – 04
CLASSIFICATION—A TWO-STEP PROCESS
 Model construction: describing a set of predetermined
classes

 Each tuple/sample is assumed to belong to a predefined


class, as determined by the class label attribute

 The set of tuples used for model construction is Training


set

 The model is represented as classification rules, decision


trees, or mathematical formulae

2
CLASSIFICATION—A TWO-STEP PROCESS
 Model usage:
 For classifying future or unknown objects
 Estimate accuracy of the model
 The known label of test sample is compared with the

classified result from the model


 Accuracy rate is the percentage of test set samples that are

correctly classified by the model


 Test set is independent of training set (otherwise over-

fitting)
 If the accuracy is acceptable, use the model to classify new
data
3
PROCESS (1): MODEL CONSTRUCTION

Classification
Algorithms
Training
Data

NAME RANK YEARS TENURED Classifier


Mike Assistant Prof 3 no (Model)
Mary Assistant Prof 7 yes
Bill Professor 2 yes
Jim Associate Prof 7 yes
IF rank = ‘professor’
Dave Assistant Prof 6 no
Anne Associate Prof 3 no
OR years > 6 4
THEN tenured = ‘yes’
PROCESS 2:MODEL USAGE FOR PREDICTION

Classifier

Testing
Data Unseen Data

(Jeff, Professor, 4)
NAME RANK YEARS TENURED
Tom Assistant Prof 2 no Tenured?
Merlisa Associate Prof 7 no
George Professor 5 yes 5

Joseph Assistant Prof 7 yes


Naive Bayes Classifier
Calculates the probability of a hypothesis (class label) given the
evidence (input features).

 Naive Bayes is a probabilistic algorithm commonly used for


classification tasks, especially in natural language processing and text
classification

Simple but effective algorithm


Introduction
It's called "naive" because it makes a strong assumption of feature
independence, which means it assumes that the features used to
describe the input are conditionally independent given the class label.

Consider a bag of fruits containing apples, bananas, and oranges


having features like color, shape, and size
Teach a computer to recognize these fruits based on their features.
Naïve Bayes Overview
1.Collect Data:
◦ gather data

◦ note down its features – like whether it's red or yellow, large or small, and so on.

2.Calculate Probabilities: Naive Bayes calculates the probability of a fruit


being, say, an apple, given its features. It does this by looking at how often
certain features (like being red and round) occur in your data set for apples.
Naïve Bayes Overview
3. Assumption of Independence (Naive Assumption): The "naive" part of
Naive Bayes comes from assuming that the features you're considering (like
color, shape, size) are independent of each other. It means that knowing the color
of the fruit doesn't give you any information about its shape, and vice versa.

4. Bayes' Theorem: Calculates the probability of an event (like a fruit

being an apple) based on the probabilities of certain related events (like the

fruit being red and round).


Naïve Bayes Overview
5. Classification: When you give the classifier the features of a new
fruit, it calculates the probabilities for each category (apple, banana, or
orange). The category with the highest probability is the classifier's
best guess for what the fruit is.
Naïve Bayes Classifier: An Example
P(Ci): P(buys_computer = “yes”) = 9/14 = 0.643
P(buys_computer = “no”) = 5/14= 0.357

Compute P(X|Ci) for each class

P(age = “<=30” | buys_computer = “yes”) = 2/9 = 0.222


P(age = “<= 30” | buys_computer = “no”) = 3/5 = 0.6

P(income = “medium” | buys_computer = “yes”) = 4/9 = 0.444


P(income = “medium” | buys_computer = “no”) = 2/5 = 0.4

25
Naïve Bayes Classifier: An Example
P(student = “yes” | buys_computer = “yes) = 6/9 = 0.667

P(student = “yes” | buys_computer = “no”) = 1/5 = 0.2

P(credit_rating = “fair” | buys_computer = “yes”) = 6/9 = 0.66

P(credit_rating = “fair” | buys_computer = “no”) = 2/5 = 0.4

26
Naïve Bayes Classifier: An Example
X = (age <= 30 , income = medium, student = yes, credit_rating =
fair)

P(X|Ci) : P(X|buys_computer = “yes”) = 0.222 x 0.444 x 0.667 x 0.667


= 0.044
P(X|buys_computer = “no”) = 0.6 x 0.4 x 0.2 x 0.4 = 0.019

P(X|Ci)*P(Ci) : P(X|buys_computer = “yes”) * P(buys_computer =


“yes”) = 0.028
P(X|buys_computer = “no”) * P(buys_computer =
“no”) = 0.007
27
Avoiding the Zero-Probability Problem

Naïve Bayesian prediction requires each conditional prob.


be non-zero. Otherwise, the predicted prob. will be zero

n
P( X | C i)   P( x k | C i)
k 1

Ex. Suppose a dataset with 1000 tuples, income=low (0),


income= medium (990), and income = high (10)

29
Avoiding the Zero-Probability Problem
Use Laplacian correction (or Laplacian estimator)

◦ Adding 1 to each case


Prob(income = low) = 1/1003
Prob(income = medium) = 991/1003
Prob(income = high) = 11/1003

◦ The “corrected” prob. estimates are close to their


“uncorrected” counterparts

30
Solve this:

You might also like