INDEX

EXP. NO. NAME OF THE EXPERIMENT REMARKS

1 Implement Data Preprocessing.

2 Implement Simple Linear Regression.

3 Simulate Multiple Linear Regression.

4 Implement Decision Tree.

5 Deploy Random Forest Classification.

6 Simulate Naïve Bayes Theorem.

7 Implement K-Nearest Neighbors(K-NN),

K means.

8 Deploy Support Vector Machine, Apriori

Algorithm.
9 Simulate Artificial Neural Network.

10 Implement the Genetic Algorithm code.

 EXPERIMENT-1
AIM: Implement Data Preprocessing.

Data preprocessing is a data mining technique which is used to transform the raw data in a useful
and efficient format.

Steps Involved in Data Preprocessing:

1. Data Cleaning:
The data can have many irrelevant and missing parts. To handle this part, data cleaning is
done. It involves handling of missing data, noisy data etc.
• (a). Missing Data:
This situation arises when some data is missing in the data. It can be handled in various
ways.
Some of them are:
1. Ignore the tuples:
This approach is suitable only when the dataset we have is quite large and multiple
values are missing within a tuple.
2. Fill the Missing values:
There are various ways to do this task. You can choose to fill the missing values
manually, by attribute mean or the most probable value.
• (b). Noisy Data:
Noisy data is a meaningless data that can’t be interpreted by machines.It can be generated
due to faulty data collection, data entry errors etc. It can be handled in following ways :

1. Binning Method:
This method works on sorted data in order to smooth it. The whole data is divided
into segments of equal size and then various methods are performed to complete the
task. Each segmented is handled separately. One can replace all data in a segment by
its mean or boundary values can be used to complete the task.
 2. Regression:
Here data can be made smooth by fitting it to a regression function.The regression used
may be linear (having one independent variable) or multiple (having multiple independent
variables).
3. Clustering:
This approach groups the similar data in a cluster. The outliers may be undetected or it will
fall outside the clusters.
2. Data Transformation:
This step is taken in order to transform the data in appropriate forms suitable for mining process.
This involves following ways:
1. Normalization:
It is done in order to scale the data values in a specified range (-1.0 to 1.0 or 0.0 to 1.0)
2. Attribute Selection:
In this strategy, new attributes are constructed from the given set of attributes to help the
mining process.
3. Discretization:
This is done to replace the raw values of numeric attribute by interval levels or conceptual
levels.
4. Concept Hierarchy Generation:
Here attributes are converted from level to higher level in hierarchy. For Example-The attribute
“city” can be converted to “country”.
3. Data Reduction:
Since data mining is a technique that is used to handle huge amount of data.
While working with huge volume of data, analysis became harder in such cases. In order to get rid
of this, we uses data reduction technique. It aims to increase the storage efficiency and reduce data
storage and analysis costs. The various steps to data reduction are:

1. Data Cube Aggregation:

Aggregation operation is applied to data for the construction of the data cube.
2. Attribute Subset Selection:
The highly relevant attributes should be used, rest all can be discarded. For performing
attribute selection, one can use level of significance and p- value of the attribute. The
attribute having p-value greater than significance level can be discarded.
3. Numerosity Reduction:
This enable to store the model of data instead of whole data, for example: Regression Models.
4. Dimensionality Reduction:
This reduce the size of data by encoding mechanisms.It can be lossy or lossless. If after
reconstruction from compressed data, original data can be retrieved, such reduction are called
lossless reduction else it is called lossy reduction. The two effective methods of
dimensionality reduction are: Wavelet transforms and PCA (Principal Component Analysis).

PROGRAM:
import pandas as pd import
numpy as np
data = pd.read_csv("data1.csv")
print("data\n") print(data.head())

# find missing data row indi = [] for

index,item in data.iterrows():
if(str(item["RM"]) == 'nan'):
indi.append(index) print("Index where
data is null\n") print(indi)
# using function print("Using function")
null_data = pd.isnull(data["RM"])

print(null_data) print(data[null_data])

# Train-Test Splitting
def split_train_test_data(data, test_ratio):
np.random.seed(42) shuffled =
np.random.permutation(len(data)) test_set_size = int(len(data) *
test_ratio) test_indices = shuffled[:test_set_size] train_indices =
shuffled[test_set_size:] return data.iloc[train_indices],
data.iloc[test_indices]
train_set, test_set = split_train_test_data(data, 0.4)
print(f"Rows in train set: {len(train_set)}\nRows in test set: {len(test_set)} \n")

# using function print("Using Function") from sklearn.model_selection import

train_test_split train_set_using_function, test_set_using_function = train_test_split(data, te
st_size=0.2, random_state=42) # , random_state=42 print(f"Rows in train set:
{len(train_set_using_function)}\nRows in test set:
{len(test_set_using_function)}\n")

OUTPUT:
 EXPERIMENT-2
AIM: Implement Simple Linear Regression.
Simple Linear Regression is a type of Regression algorithms that models the relationship
between a dependent variable and a single independent variable. The relationship shown by a
Simple Linear Regression model is linear or a sloped straight line, hence it is called Simple
Linear Regression.

For simple linear regression, the form of the model is-

Y = β0 + β1X

Here,
• Y is a dependent variable.
• X is an independent variable.
• β0 and β1 are the regression coefficients.
• β0 is the intercept or the bias that fixes the offset to a line.
• β1 is the slope or weight that specifies the factor by which X has an impact on Y.

There are following 3 cases possible-

Case-01: β1 < 0

• It indicates that variable X has negative impact on Y.

• If X increases, Y will decrease and vice-versa.

Case-02: β1 = 0

• It indicates that variable X has no impact on Y.

• If X changes, there will be no change in Y.
Case-03: β1 > 0

• It indicates that variable X has positive impact on Y.

• If X increases, Y will increase and vice-versa.

PROGRAM:
import pandas from pandas import DataFrame import
matplotlib.pyplot as plt from sklearn.linear_model import
LinearRegression from scipy.stats import pearsonr
data = pandas.read_csv('Linear_regression_basic.csv')
print("Dataset : \n",data)
print(data.describe())
x = DataFrame(data,columns=['x']) y
= DataFrame(data,columns=['y'])

# plt.figure(figsize=(10,10)) plt.title('LINEAR
REGRESSION') plt.xlabel('X axis') #label of X
axis plt.ylabel('Y axis') #label of Y axis
plt.ylim(0,7) #for Y axis limit
plt.xlim(0,8) #for X axis limit plt.grid()
#for grid
# plt.scatter(x,y,alpha=(0.7)) #Visibility of point
plt.scatter(x,y,color='green',s=50) #s for size plt.show()
print("Data is x : \n",data['x'])
print("Data in y : \n",data['y'])
# Correlation Coefficient corr,_ =
pearsonr(data['x'],data['y'])
print("Correlation Coefficient : ",corr)
regression = LinearRegression() regression.fit(x,y)
print("Regression Cofficient : ",regression.coef_)
# Intercept print("Regression Intercept :
",regression.intercept_)
plt.figure(figsize=(10,6)) plt.title('LINEAR REGRESSION')
plt.xlabel('x --->') plt.ylabel('y --->') plt.ylim(0,15)
plt.xlim(0,15) plt.scatter(x,y,alpha=(0.5))
plt.plot(x,regression.predict(x),color = "red",linewidth=2)
plt.show() print("Accuracy score : ",regression.score(x,y))
print(" New data
") data['y'][2] = 7

print(data)
X = DataFrame(data,columns=['x'])
Y = DataFrame(data,columns=['y']) plt.title('LINEAR
REGRESSION NEW') plt.xlabel('X axis') #label of X axis
plt.ylabel('Y axis') #label of Y axis plt.ylim(0,10)
#for Y axis limit plt.xlim(0,10) #for X axis limit
plt.grid() #for grid
# plt.scatter(x,y,alpha=(0.7)) #Visibility of point
plt.scatter(X,Y,color='green',s=50) #s for size plt.show()
regression = LinearRegression()
regression.fit(X,Y)
print("Regression cofficient : ",regression.coef_)
print("Regression Intercept : ",regression.intercept_)
plt.figure(figsize=(10,6))
plt.title('LINEAR REGRESSION
NEW')
plt.xlabel('x --->') plt.ylabel('y --->') plt.ylim(0,15)
plt.xlim(0,15) plt.scatter(X,Y)
plt.plot(X,regression.predict(X),color = "red",linewidth=2)
plt.show()
print("New score : ",regression.score(X,Y))

OUTPUT:
 EXPERIMENT-3
AIM: Simulate Multiple Linear Regression.

In multiple linear regression, the dependent variable depends on more than one independent
variables.

For multiple linear regression, the form of the model is-

Y = β0 + β1X1 + β2X2 + β3X3 + …… + βnXn

Here,
• Y is a dependent variable.
• X1, X2, …., Xn are independent variables.
• β0, β1,…, βn are the regression coefficients.
βj (1<=j<=n) is the slope or weight that specifies the factor by
which Xj has an impact on Y.

PROGRAM:

import pandas as pd import numpy as

np from sklearn import linear_model
data = pd.read_csv("mlr.csv")
print("\n Dataset\n")
print(data.head())
print("length of data : ",data.shape)

# dependent variable = price

# independent variables (features) = area,bedrooms,age
# price = m1 x area + m2 x bedrooms + m3 x age + m4
# m1,m2,m3 = coefficient
# m4 = intercept

# Train-Test Splitting
from sklearn.model_selection import train_test_split train_set, test_set =
train_test_split(data, test_size=0.2, random_state=42) print(f"Rows in train set:
{len(train_set)}\nRows in test set: {len(test_set)} \n") print("Train set : ",train_set)
print("Test set : ",test_set)
reg = linear_model.LinearRegression()
reg.fit(train_set[["area","bedrooms","age"]],train_set["price"])
print("Regression Cofficient : ",reg.coef_) #m1,m2,m3
print("Intercept : ",reg.intercept_) #m4
print("predicted value of test set : ")
print(reg.predict(test_set[["area","bedrooms","age"]]))

OUTPUT:
 EXPERIMENT-4
AIM: Implement Decision Trees.

o Decision Tree is a Supervised learning technique that can be used for both
classification and Regression problems, but mostly it is preferred for solving
Classification problems. It is a tree-structured classifier, where internal nodes
represent the features of a dataset, branches represent the decision rules and
each leaf node represents the outcome.
o In a Decision tree, there are two nodes, which are the Decision Node and Leaf Node.
Decision nodes are used to make any decision and have multiple branches, whereas
Leaf nodes are the output of those decisions and do not contain any further branches.
o The decisions or the test are performed on the basis of features of the given dataset. o
It is a graphical representation for getting all the possible solutions to a
problem/decision based on given conditions. o It is called a decision tree because,
similar to a tree, it starts with the root node, which expands on further branches and
constructs a tree-like structure.
o In order to build a tree, we use the CART algorithm, which stands for
Classification and Regression Tree algorithm.
o A decision tree simply asks a question, and based on the answer (Yes/No), it further
split the tree into subtrees.

ALOGRITHM OF DECISION TREES:

o Step-1: Begin the tree with the root node, says S, which contains the complete
dataset.
o Step-2: Find the best attribute in the dataset using Attribute Selection Measure
(ASM). o Step-3: Divide the S into subsets that contains possible values for the best
attributes.
o Step-4: Generate the decision tree node, which contains the best attribute.
o Step-5: Recursively make new decision trees using the subsets of the dataset created
in step -3. Continue this process until a stage is reached where you cannot further
classify the nodes and called the final node as a leaf node.
 Implementation:

import pandas as pd

df = pd.read_csv("titanic.csv") print(df.head(5)) print("Shape : ",df.shape) data =

pd.DataFrame(df,columns=["Pclass","Sex","Age","Fare","Survived"])
print(data.head()) input = data.drop('Survived',axis=1) input.Sex =
input.Sex.map({'male':1,'female':2}) target = data.Survived print("Input Data set : \n")
print(input) print("Target Data set : \n") print(target) print(input.info()) input.Age =
input.Age.fillna(input.Age.mean())
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(input,target,test_size=0.2

) print("length of Train Dataset",len(X_train))
print("length of test Dataset",len(X_test))
from sklearn import tree
model = tree.DecisionTreeClassifier()
model.fit(X_train,y_train) print("predicted values on
test set : \n") print(model.predict(X_test))
print("Score : ",model.score(X_test,y_test))
print("Confusion Matrix : ") y_predicted =
model.predict(X_test)
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_predicted)
print(cm)
import seaborn as sn
import matplotlib.pyplot as plt
plt.figure(figsize = (10,7))
sn.heatmap(cm, annot=True)
plt.xlabel('Predicted')
plt.ylabel('Truth')
 Dataset :

Titanic data set

Output:
 EXPERIMENT-5
AIM: Implement Random Forest Classification.

Random Forest is a popular machine learning algorithm that belongs to the supervised
learning technique. It can be used for both Classification and Regression problems in ML. It
is based on the concept of ensemble learning, which is a process of combining multiple
classifiers to solve a complex problem and to improve the performance of the model.

As the name suggests, "Random Forest is a classifier that contains a number of decision
trees on various subsets of the given dataset and takes the average to improve the
predictive accuracy of that dataset." Instead of relying on one decision tree, the random
forest takes the prediction from each tree and based on the majority votes of predictions, and
it predicts the final output.

The greater number of trees in the forest leads to higher accuracy and prevents the
problem of overfitting.

ALGORITHM OF RANDOM FOREST ALOGRITHM:

Random Forest works in two-phase first is to create the random forest by combining N decision
tree, and second is to make predictions for each tree created in the first phase.

The Working process can be explained in the below steps and diagram:

Step-1: Select random K data points from the training set.

Step-2: Build the decision trees associated with the selected data points (Subsets).

Step-3: Choose the number N for decision trees that you want to build.

Step-4: Repeat Step 1 & 2.

Step-5: For new data points, find the predictions of each decision tree, and assign the new data
points to the category that wins the majority votes.

Python Code:
# Random Forest Classification
# Importing the libraries import
numpy as np import
matplotlib.pyplot as plt import
pandas as pd
# Importing the dataset
dataset = pd.read_csv('Social_Network_Ads.csv') X =
dataset.iloc[:, [2, 3]].values y = dataset.iloc[:, -1].values

# Splitting the dataset into the Training set and Test set from
sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25, ra ndom_state = 0)

# Feature Scaling from sklearn.preprocessing import

StandardScaler sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

# Training the Random Forest Classification model on the Training set from
sklearn.ensemble import RandomForestClassifier classifier =
RandomForestClassifier(n_estimators = 10, criterion = 'entropy', random_state = 0)
classifier.fit(X_train, y_train)
# Predicting the Test set results y_pred =
classifier.predict(X_test)

# Making the Confusion Matrix from

sklearn.metrics import confusion_matrix cm =
confusion_matrix(y_test, y_pred) print(cm)

# Visualising the Training set results from

matplotlib.colors import ListedColormap
X_set, y_set = X_train, y_train
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() -
1, stop = X_set[:, 0].max() + 1, step = 0.01),
np.arange(start = X_set[:, 1].min() -
1, stop = X_set[:, 1].max() + 1, step = 0.01)) plt.contourf(X1, X2,
classifier.predict(np.array([X1.ravel(), X2.ravel()]).T).
reshape(X1.shape), alpha = 0.75, cmap = ListedColormap(('red',
'green'))) plt.xlim(X1.min(), X1.max()) plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1], c=
ListedColormap(('red', 'green'))(i), label = j) plt.title('Random Forest
Classification (Training set)') plt.xlabel('Age') plt.ylabel('Estimated Salary')
plt.legend() plt.show()

# Visualising the Test set results

from matplotlib.colors import ListedColormap
X_set, y_set = X_test, y_test
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() -
1, stop = X_set[:, 0].max() + 1, step = 0.01),
np.arange(start = X_set[:, 1].min() -
1, stop = X_set[:, 1].max() + 1, step = 0.01)) plt.contourf(X1, X2,
classifier.predict(np.array([X1.ravel(), X2.ravel()]).T).
reshape(X1.shape), alpha = 0.75, cmap = ListedColormap(('red',
'green'))) plt.xlim(X1.min(), X1.max()) plt.ylim(X2.min(), X2.max()) for i, j in
enumerate(np.unique(y_set)):
plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1], c=
ListedColormap(('red', 'green'))(i), label = j) plt.title('Random Forest
Classification (Test set)') plt.xlabel('Age') plt.ylabel('Estimated Salary')
plt.legend() plt.show()

Output:
[[63 5]
[ 4 28]]
 EXPERIMENT-6

AIM: Simulate Naïve Bayes algorithm.

o Naïve Bayes algorithm is a supervised learning algorithm, which is based on Bayes theorem
and used for solving classification problems.
o It is mainly used in text classification that includes a high-dimensional training dataset.
o Naïve Bayes Classifier is one of the simple and most effective Classification algorithms
which helps in building the fast machine learning models that can make quick predictions. o
It is a probabilistic classifier, which means it predicts on the basis of the probability of
an object.
o Some popular examples of Naïve Bayes Algorithm are spam filtration, Sentimental
analysis, and classifying articles.

The Naïve Bayes algorithm is comprised of two words Naïve and Bayes, Which can be described
as:

o Naïve: It is called Naïve because it assumes that the occurrence of a certain feature is
independent of the occurrence of other features. Such as if the fruit is identified on the bases
of color, shape, and taste, then red, spherical, and sweet fruit is recognized as an apple.
Hence each feature individually contributes to identify that it is an apple without depending
on each other.
o Bayes: It is called Bayes because it depends on the principle of Bayes' Theorem.

Bayes' Theorem:

o Bayes' theorem is also known as Bayes' Rule or Bayes' law, which is used to determine the
probability of a hypothesis with prior knowledge. It depends on the conditional probability.
o The formula for Bayes' theorem is given as:

Where,

P(A|B) is Posterior probability: Probability of hypothesis A on the observed event B.

P(B|A) is Likelihood probability: Probability of the evidence given that the probability of a
hypothesis is true.

P(A) is Prior Probability: Probability of hypothesis before observing the evidence.

P(B) is Marginal Probability: Probability of Evidence.

Working of Naïve Bayes' Classifier:

Working of Naïve Bayes' Classifier can be understood with the help of the below example:
Suppose we have a dataset of weather conditions and corresponding target variable "Play". So
using this dataset we need to decide that whether we should play or not on a particular day
according to the weather conditions. So to solve this problem, we need to follow the below steps:

1. Convert the given dataset into frequency tables.

2. Generate Likelihood table by finding the probabilities of given features.
3. Now, use Bayes theorem to calculate the posterior probability.

PROGRAM:

import numpy as np # Import

LabelEncoder from sklearn import
preprocessing #Import Gaussian Naive
Bayes model
from sklearn.naive_bayes import GaussianNB

# Assigning features and label variables

wheather=['Sunny','Sunny','Overcast','Rainy','Rainy','Rainy','Overcast','Sunny ','Sunny',
'Rainy','Sunny','Overcast','Overcast','Rainy']
temp=['Hot','Hot','Hot','Mild','Cool','Cool','Cool','Mild','Cool','Mild','Mild
','Mild','Hot','Mild'] humidity = ['high','high','high','high','normal','normal','normal','high','nor
mal','normal','normal','high','normal','high'] windy =
['false','true','false','false','false','true','true','false','false',
'false','true','true','false','true']
play=['No','No','Yes','Yes','Yes','No','Yes','No','Yes','Yes','Yes','Yes','Yes
','No']

#creating labelEncoder le --> Label Encoder le =

preprocessing.LabelEncoder()
# Converting string labels into numbers. wheather_encoded=le.fit_transform(wheather)

# Converting string labels into numbers

temp_encoded=np.array(le.fit_transform(temp))
# Converting string labels into numbers
humidity_encoded=np.array(le.fit_transform(humidity))
windy_encoded=np.array(le.fit_transform(windy))
label=le.fit_transform(play)

temp=np.array(temp) label=np.array(label)
for r in range(0,len(wheather)):
print ("%10s\t%d\t%10s\t%d\t%10s\t%d\t%10s\t%d\t%10s\t%d"%(wheather[r],whe
ather_encoded[r],temp[r],temp_encoded[r],humidity[r],humidity_encoded[r],windy
[r],windy_encoded[r],play[r],label[r]))

#Combinig weather and temp into single listof tuples

#features=zip(wheather_encoded,temp_encoded)
#features=[wheather_encoded,temp_encoded]
features=[] print("\nFeature Array") for r in
range(0,len(wheather)):
temp=[] temp.append(wheather_encoded[r])
temp.append(temp_encoded[r])
temp.append(humidity_encoded[r])
temp.append(windy_encoded[r])
features.append(temp)
 print ("%5d%5d%5d%5d"%(features[r][0],features[r][1],features[r][2],featur es[r][3]))

#Create a Gaussian Classifier model =

GaussianNB()

# Train the model using the training sets

model.fit(features,label)

#Predict Output predicted= model.predict([[2,1,0,0]]) # 0:Overcast, 2:Mild

if(predicted==0): print ("Predicted Value:", predicted,"\tPlay: NO")
if(predicted==1): print ("Predicted Value:", predicted,"\tPlay: YES")

OUTPUT:
 EXPERIMENT-7
AIM: Implement K-Nearest Neighbors (K-NN), K-means.

K-NN Algorithm:
o K-Nearest Neighbour is one of the simplest Machine Learning algorithms based on
Supervised Learning technique.
o K-NN algorithm assumes the similarity between the new case/data and available cases and
put the new case into the category that is most similar to the available categories.
o K-NN algorithm stores all the available data and classifies a new data point based on the
similarity. This means when new data appears then it can be easily classified into a well
suite category by using K- NN algorithm.
o K-NN algorithm can be used for Regression as well as for Classification but mostly it is used
for the Classification problems.
o K-NN is a non-parametric algorithm, which means it does not make any assumption on
underlying data.
o It is also called a lazy learner algorithm because it does not learn from the training set
immediately instead it stores the dataset and at the time of classification, it performs an
action on the dataset.
o KNN algorithm at the training phase just stores the dataset and when it gets new data, then it
classifies that data into a category that is much similar to the new data.

K-NN Algorithm:

o Step-1: Select the number K of the neighbors o Step-2: Calculate the Euclidean distance of
K number of neighbors o Step-3: Take the K nearest neighbors as per the calculated
Euclidean distance.
o Step-4: Among these k neighbors, count the number of the data points in each category.

o Step-5: Assign the new data points to that category for which the number of the neighbor is
maximum.

o Step-6: Our model is ready.

Program:
import pandas as pd from sklearn.datasets
import load_iris iris = load_iris()
print("IRIS\nFeature Names:\n",iris.feature_names)
print("\nTarget Names:\n",iris.target_names) df =
pd.DataFrame(iris.data,columns=iris.feature_names)
df['target'] = iris.target
print("\nDATASET:\n",df.head())
print("Shape: ",df.shape)

from sklearn.model_selection import train_test_split X =

df.drop(['target'], axis='columns') y = df.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
from sklearn.neighbors import KNeighborsClassifier model =
KNeighborsClassifier(n_neighbors=5) model.fit(X_train,y_train) print("Predicted
values on testing dataset: \n",model.predict(X_test)) print("MOdel Scores :
",model.score(X_test,y_test))

Output:

k-Means Algorithm:

K-Means Clustering is an Unsupervised Learning algorithm, which groups the unlabelled dataset
into different clusters. Here K defines the number of predefined clusters that need to be created in
the process, as if K=2, there will be two clusters, and for K=3, there will be three clusters, and so
on.
It allows us to cluster the data into different groups and a convenient way to discover the categories
of groups in the unlabeled dataset on its own without the need for any training.

It is a centroid-based algorithm, where each cluster is associated with a centroid. The main aim of
this algorithm is to minimize the sum of distances between the data point and their corresponding
clusters.

The algorithm takes the unlabeled dataset as input, divides the dataset into knumber of clusters,
and repeats the process until it does not find the best clusters. The value of k should be
predetermined in this algorithm. k-Means ALGORITHM:

Step-1: Select the number K to decide the number of clusters.

Step-2: Select random K points or centroids. (It can be other from the input dataset).

Step-3: Assign each data point to their closest centroid, which will form the predefined K
clusters.

Step-4: Calculate the variance and place a new centroid of each cluster.

Step-5: Repeat the third steps, which means reassign each datapoint to the new closest centroid
of each cluster.

Step-6: If any reassignment occurs, then go to step-4 else go to FINISH.

Step-7: The model is ready.

Program:
from sklearn.cluster import KMeans import pandas as
pd from sklearn.preprocessing import MinMaxScaler
from matplotlib import pyplot as plt
df = pd.read_csv("income.csv")
print("DATAFRAME:\n",df.head())
plt.scatter(df.Age,df['Income($)'])
plt.xlabel('Age') plt.ylabel('Income($)')
plt.show()
km = KMeans(n_clusters=3) y_predicted =
km.fit_predict(df[['Age','Income($)']]) print("Y
predicted:",y_predicted) df['cluster']=y_predicted print("New
dataframe:\n",df.head()) print("\nCluster Centers :
",km.cluster_centers_)
df1 = df[df.cluster==0] df2 = df[df.cluster==1] df3 =
df[df.cluster==2]
plt.scatter(df1.Age,df1['Income($)'],color='green')
plt.scatter(df2.Age,df2['Income($)'],color='red')
plt.scatter(df3.Age,df3['Income($)'],color='black')
plt.scatter(km.cluster_centers_[:,0],km.cluster_centers_[:,1],color='purple',m
arker='*',label='centroid') plt.xlabel('Age') plt.ylabel('Income ($)') plt.legend() plt.show()

# Preprocessing using min max scaler scaler = MinMaxScaler()

scaler.fit(df[['Income($)']]) df['Income($)'] =
scaler.transform(df[['Income($)']])
scaler.fit(df[['Age']]) df['Age'] =
scaler.transform(df[['Age']]) print("\nDataframe
:\n",df.head()) plt.scatter(df.Age,df['Income($)'])
plt.show()
km = KMeans(n_clusters=3) y_predicted =
km.fit_predict(df[['Age','Income($)']]) print("\nY predicted :
",y_predicted) df['cluster']=y_predicted
print("\nDataset\n",df.head())
print("\nCluster Centers : ",km.cluster_centers_)
df1 = df[df.cluster==0] df2 = df[df.cluster==1] df3 = df[df.cluster==2]
plt.scatter(df1.Age,df1['Income($)'],color='green')
plt.scatter(df2.Age,df2['Income($)'],color='red')
plt.scatter(df3.Age,df3['Income($)'],color='black')
plt.scatter(km.cluster_centers_[:,0],km.cluster_centers_[:,1],color='purple',m
arker='*',label='centroid') plt.legend() plt.show()
Outpu
 EXPERIMENT-8
AIM: Deploy Support Vector Machine, Apriori algorithm.

Support Vector Machine

Support Vector Machine or SVM is one of the most popular Supervised Learning algorithms, which
is used for Classification as well as Regression problems. However, primarily, it is used for
Classification problems in Machine Learning.

The goal of the SVM algorithm is to create the best line or decision boundary that can segregate n-
dimensional space into classes so that we can easily put the new data point in the correct category in
the future. This best decision boundary is called a hyperplane.

SVM chooses the extreme points/vectors that help in creating the hyperplane. These extreme cases
are called as support vectors, and hence algorithm is termed as Support Vector Machine.

PROGRAM:
import pandas as pd from sklearn.datasets
import load_digits digits = load_digits()
print("Digits Target : ",digits.target) df =
pd.DataFrame(digits.data,digits.target) df['target'] =
digits.target
print("\nDATASET DIGIT\n",df.head())

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(df.drop('target',axis='col umns'), df.target,
test_size=0.3) print("\nRBF model\n") from sklearn.svm import SVC rbf_model =
SVC(kernel='rbf')
rbf_model.fit(X_train, y_train) print("RBF model
Score",rbf_model.score(X_test,y_test)) print("\nUsing Linear kernel\n")
linear_model = SVC(kernel='linear') linear_model.fit(X_train,y_train)
print("linear model Score",linear_model.score(X_test,y_test))

OUTPUT:
APRIORI ALGORITHM:

The Apriori algorithm uses frequent itemsets to generate association rules, and it is designed to
work on the databases that contain transactions. With the help of these association rule, it
determines how strongly or how weakly two objects are connected. This algorithm uses a breadth-
first search and Hash Tree to calculate the itemset associations efficiently. It is the iterative
process for finding the frequent itemsets from the large dataset.

This algorithm was given by the R. Agrawal and Srikant in the year 1994. It is mainly used for
market basket analysis and helps to find those products that can be bought together. It can also be
used in the healthcare field to find drug reactions for patients.

PROGRAM:
import numpy as np import pandas as pd from mlxtend.frequent_patterns
import apriori, association_rules
data = pd.read_excel('Online_Retail.xlsx')
print("DataSet:",data.head()) print("Data Columns
: ",data.columns) print("Data Shape :
",data.shape)
# Stripping extra spaces in the description
data['Description'] = data['Description'].str.strip()

# Dropping the rows without any invoice number data.dropna(axis = 0,

subset =['InvoiceNo'], inplace = True) data['InvoiceNo'] =
data['InvoiceNo'].astype('str')
From the above output, it can be seen that paper cups and plates are bought together in
France.
 EXPERIMENT-9

AIM: Simulate Artificial Neural Network.

Neural networks (NN), also called artificial neural networks (ANN) are a subset of learning algorithms
within the machine learning field that are loosely based on the concept of biological neural networks.
Andrey Bulezyuk, who is a German-based machine learning specialist with more than five
years of experience, says that “neural networks are revolutionizing machine learning because they are
capable of efficiently modelling sophisticated abstractions across an extensive range of disciplines
and industries.”

Basically, an ANN comprises of the following components:

An input layer that receives data and pass it on
A hidden layer
An output layer
Weights between the layers

A deliberate activation function for every hidden layer. In this simple neural network Python
tutorial, we’ll employ the Sigmoid activation function.
There are several types of neural networks. In this project, we are going to create the feed-forward or
perception neural networks. This type of ANN relays data directly from the front to the back.

Training the feed-forward neurons often need back-propagation, which provides the network with
corresponding set of inputs and outputs. When the input data is transmitted into the neuron, it is
processed, and an output is generated.
Summarizing an Artificial Neural Network:
1. Take inputs
2. Add bias (if required)
3. Assign random weights to input features
4. Run the code for training.
5. Find the error in prediction.
6. Update the weight by gradient descent algorithm.
7. Repeat the training phase with updated weights.
8. Make predictions.

Python Code:
from joblib.numpy_pickle_utils import xrange from
numpy import *
class NeuralNet(object): def
__init__(self): # Generate random
numbers random.seed(1)

# Assign random weights to a 3 x 1 matrix,

self.synaptic_weights = 2 * random.random((3, 1)) - 1
# The Sigmoid function def
__sigmoid(self, x):
return 1 / (1 + exp(-x))

# The derivative of the Sigmoid function. # This is

the gradient of the Sigmoid curve. def
__sigmoid_derivative(self, x):
return x * (1 - x)

# Train the neural network and adjust the weights each time. def train(self,
inputs, outputs, training_iterations):
for iteration in xrange(training_iterations): # Pass the
training set through the network. output = self.learn(inputs)

# Calculate the error error =

outputs - output

# Adjust the weights by a factor factor = dot(inputs.T, error *

self.__sigmoid_derivative(output)) self.synaptic_weights += factor

# The neural network thinks.

def learn(self, inputs):
return self.__sigmoid(dot(inputs, self.synaptic_weights))

if __name__ == "__main__":
# Initialize neural_network = NeuralNet()

# The training set. inputs = array([[0, 1, 1], [1, 0, 0], [1, 0, 1]]) outputs = array([[1, 0,
1]]).T

# Train the neural network neural_network.train(inputs, outputs, 10000)

# Test the neural network with a test example. print(neural_network.learn(array([1, 0,

1])))
OUTPUT:
 EXPERIMENT-10
AIM: Implement the Genetic Algorithm Code.

Genetic Algorithm (GA) is a search-based optimization technique based on the principles of

Genetics and Natural Selection. It is frequently used to find optimal or near-optimal solutions to
difficult problems which otherwise would take a lifetime to solve. It is frequently used to solve
optimization problems, in research, and in machine learning.
WORKING OF GENETIC ALGORITHM:

1. Initial Population– Initialize the population randomly based on the data.

2. Fitness function– Find the fitness value of the each of the chromosomes(a chromosome is a set
of parameters which define a proposed solution to the problem that the genetic algorithm is trying
to solve)
3. Selection– Select the best fitted chromosomes as parents to pass the genes for the next generation
and create a new population
4. Cross-over– Create new set of chromosome by combining the parents and add them to new
population set
5. Mutation– Perform mutation which alters one or more gene values in a chromosome in the new
population set generated. Mutation helps in getting more diverse opportunity. Obtained
population will be used in the next generation
Repeat step 2-5 again for each generation.

Python Code:
import numpy as np import pandas as pd import random
import matplotlib.pyplot from sklearn.datasets import
load_breast_cancer from sklearn.model_selection import
train_test_split from sklearn.linear_model import
LogisticRegression from sklearn.metrics import
accuracy_score
#import the breast cancer dataset from sklearn.datasets import
load_breast_cancer cancer=load_breast_cancer() df =
pd.DataFrame(cancer['data'],columns=cancer['feature_names'])
label=cancer["target"]
#splitting the model into training and testing set X_train, X_test, y_train, y_test =
train_test_split(df, label, test_size=0.30,
random_state=101)
#training a logistics regression model logmodel = LogisticRegression()
logmodel.fit(X_train,y_train) predictions = logmodel.predict(X_test)
print("Accuracy = "+ str(accuracy_score(y_test,predictions))) #defining
various steps required for the genetic algorithm def
initilization_of_population(size,n_feat):
population = [] for i in
range(size):
chromosome = np.ones(n_feat,dtype=np.bool)
chromosome[:int(0.3*n_feat)]=False
np.random.shuffle(chromosome)
population.append(chromosome) return population
def fitness_score(population):
scores = [] for chromosome in
population:
logmodel.fit(X_train.iloc[:,chromosome],y_train) predictions =
logmodel.predict(X_test.iloc[:,chromosome])
scores.append(accuracy_score(y_test,predictions)) scores, population =
np.array(scores), np.array(population) inds = np.argsort(scores) return
list(scores[inds][::-1]), list(population[inds,:][::-1]) def
selection(pop_after_fit,n_parents):
population_nextgen = [] for i in range(n_parents):
population_nextgen.append(pop_after_fit[i]) return
population_nextgen
def crossover(pop_after_sel):
population_nextgen=pop_after_sel for i in
range(len(pop_after_sel)):
child=pop_after_sel[i]
child[3:7]=pop_after_sel[(i+1)%len(pop_after_sel)][3:7]
population_nextgen.append(child) return population_nextgen
def mutation(pop_after_cross,mutation_rate):
population_nextgen = [] for i in
range(0,len(pop_after_cross)):
chromosome = pop_after_cross[i] for j in
range(len(chromosome)): if random.random() <
mutation_rate: chromosome[j]= not
chromosome[j]
population_nextgen.append(chromosome)
#print(population_nextgen)

return population_nextgen

def generations(size,n_feat,n_parents,mutation_rate,n_gen,X_train,
 X_test, y_train, y_test):
best_chromo= [] best_score= []
population_nextgen=initilization_of_population(size,n_feat) for i in range(n_gen):
scores, pop_after_fit = fitness_score(population_nextgen) print(scores[:2])
pop_after_sel = selection(pop_after_fit,n_parents) pop_after_cross =
crossover(pop_after_sel) population_nextgen =
mutation(pop_after_cross,mutation_rate) best_chromo.append(pop_after_fit[0])
best_score.append(scores[0]) return best_chromo,best_score

OUTPUT: