Rule-based Classification in Data Mining
Rule-based classification in data mining is a technique in which class
decisions are taken based on various “if...then… else” rules. Thus, we define it as a
classification type governed by a set of IF-THEN rules. We write an IF-THEN rule as:
“IF condition THEN conclusion.”
Rule Based Algorithm
IF-THEN Rule
To define the IF-THEN rule, we can split it into two parts:
Rule Antecedent: The IF part of the rule is called rule
antecedent or precondition This is the “if condition” part of the rule. This part
is present in the LHS(Left Hand Side). The antecedent can have one or more attributes
as conditions, with logic AND operator.
Rule Consequent: The THEN part of the rule is called rule
consequent .This is present in the rule's RHS(Right Hand Side). The rule
consequent consists of the class prediction.
Assessment of Rule
In rule-based classification in data mining, there are two factors based on which we
can access the rules. These are:
Coverage of Rule: The fraction of the records which satisfy the antecedent conditions
of a particular rule is called the coverage of that rule.
We can calculate this by dividing the number of records satisfying the rule(n1) by the
total number of records(n).
Coverage(R) = n1/n
Accuracy of a rule: The fraction of the records that satisfy the antecedent conditions
and meet the consequent values of a rule is called the accuracy of that rule.
We can calculate this by dividing the number of records satisfying the consequent
values(n2) by the number of records satisfying the rule(n1).
Accuracy(R) = n2/n1
Properties of Rule-Based Classifiers
There are two significant properties of rule-based classification in data mining. They
are:
Rules may not be mutually exclusive
Rules may not be exhaustive
Rules may not be mutually exclusive in nature
Many different rules are generated for the dataset, so it is possible and likely
that many of them satisfy the same data record. This condition makes the rules not
mutually exclusive.
Since the rules are not mutually exclusive, we cannot decide on classes that cover
different parts of data on different rules. But this was our main objective. So, to solve
this problem, we have two ways:
The first way is using an ordered set of rules. By ordering the rules, we set priority
orders. Thus, this ordered rule set is called a decision list. So the class with the highest
priority rule is taken as the final class.
The second solution can be assigning votes for each class depending on their weights.
So, in this, the set of rules remains unordered.
Rules may not be exhaustive in nature
It is not a guarantee that the rule will cover all the data entries. Any of the
rules may leave some data entries. This case, on its occurrence, will make the rules
non-exhaustive. So, we have to solve this problem too. So, to solve this problem, we
can make use of a default class. Using a default class, we can assign all the data
entries not covered by any rules to the default class. Thus using the default class will
solve the problem of non-exhaustivity.
An example of a rule-based algorithm is a decision tree. For example, a decision tree for
predicting whether a person will buy a product might start by splitting the data based on
age. If the person is over a certain age, the tree might next split based on income, and so
on. The final leaf nodes would be the predicted decision, such as "buy" or "don't buy".
Rule Extraction
To extract a rule from a decision tree −
One rule is created for each path from the root to the leaf node.
To form a rule antecedent, each splitting criterion is logically ANDed.
The leaf node holds the class prediction, forming the rule consequent.
Rule Pruning
The rule is pruned is due to the following reason −
The Assessment of quality is made on the original set of training data. The
rule may perform well on training data but less well on subsequent data.
That's why the rule pruning is required.
The rule is pruned by removing conjunct. The rule R is pruned, if pruned
version of R has greater quality than what was assessed on an
independent set of tuples.
FOIL is one of the simple and effective method for rule pruning. For a
given rule R,
FOIL_Prune = pos - neg / pos + neg
where pos and neg is the number of positive tuples covered by R,
respectively.
Note − This value will increase with the accuracy of R on the pruning set.
Hence, if the FOIL_Prune value is higher for the pruned version of R, then
we prune R.
