Scribd
Upload
63 views25 pages

Customer Segmentation in Python Chapter3

The document discusses customer segmentation using k-means clustering in Python. It covers advantages of k-means clustering and its key assumptions of symmetric and similar variable distributions. It then discusses how to identify and manage skewed variables through transformations like logarithms and dealing with negative values. The document also covers centering and scaling variables to have equal means and variances before clustering. It emphasizes standardizing the data in the proper sequence of first unskewing, then centering, followed by scaling to meet k-means assumptions before applying the algorithm.

Uploaded by

Fgpeqw
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
Download as pdf or txt
63 views25 pages

Customer Segmentation in Python Chapter3

The document discusses customer segmentation using k-means clustering in Python. It covers advantages of k-means clustering and its key assumptions of symmetric and similar variable distributions. It then discusses how to identify and manage skewed variables through transformations like logarithms and dealing with negative values. The document also covers centering and scaling variables to have equal means and variances before clustering. It emphasizes standardizing the data in the proper sequence of first unskewing, then centering, followed by scaling to meet k-means assumptions before applying the algorithm.

Uploaded by

Fgpeqw
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
Download as pdf or txt
Download as pdf or txt
You are on page 1/ 25

DataCamp Customer Segmentation in Python

CUSTOMER SEGMENTATION IN PYTHON

Data pre-processing for k-


means clustering

Karolis Urbonas
Head of Data Science, Amazon
DataCamp Customer Segmentation in Python

Advantages of k-means clustering


One of the most popular unsupervised learning method
Simple and fast
Works well*

* with certain assumptions about the data


DataCamp Customer Segmentation in Python

Key k-means assumptions


Symmetric distribution of variables (not skewed)
Variables with same average values
Variables with same variance
DataCamp Customer Segmentation in Python

Skewed variables

Left-skewed

Right-skewed
DataCamp Customer Segmentation in Python

Skewed variables
Skew removed with logarithmic
transformation
DataCamp Customer Segmentation in Python

Variables on the same scale


datamart_rfm.describe()
K-means assumes equal mean
And equal variance
It's not the case with RFM data
DataCamp Customer Segmentation in Python

CUSTOMER SEGMENTATION IN PYTHON

Let's review the concepts


DataCamp Customer Segmentation in Python

CUSTOMER SEGMENTATION IN PYTHON

Managing skewed variables

Karolis Urbonas
Head of Data Science, Amazon
DataCamp Customer Segmentation in Python

Identifying skewness
Visual analysis of the distribution
If it has a tail - it's skewed
DataCamp Customer Segmentation in Python

Exploring distribution of Recency


import seaborn as sns
from matplotlib import pyplot as plt

sns.distplot(datamart['Recency'])
plt.show()
DataCamp Customer Segmentation in Python

Exploring distribution of Frequency


sns.distplot(datamart['Frequency'])
plt.show()
DataCamp Customer Segmentation in Python

Data transformations to manage skewness


Logarithmic transformation (positive values only)
import numpy as np
frequency_log= np.log(datamart['Frequency'])

sns.distplot(frequency_log)
plt.show()
DataCamp Customer Segmentation in Python

Dealing with negative values


Adding a constant before log transformation
Cube root transformation
DataCamp Customer Segmentation in Python

CUSTOMER SEGMENTATION IN PYTHON

Let's practice how to


identify and manage
skewed variables!
DataCamp Customer Segmentation in Python

CUSTOMER SEGMENTATION IN PYTHON

Centering and scaling


variables

Karolis Urbonas
Head of Data Science, Amazon
DataCamp Customer Segmentation in Python

Identifying an issue
datamart_rfm.describe()
Analyze key statistics of the dataset
Compare mean and standard
deviation
DataCamp Customer Segmentation in Python

Centering variables with different means


K-means works well on variables with the same mean
Centering variables is done by subtracting average value from each observation
datamart_centered = datamart_rfm - datamart_rfm.mean()
datamart_centered.describe().round(2)
DataCamp Customer Segmentation in Python

Scaling variables with different variance


K-means works better on variables with the same variance / standard deviation
Scaling variables is done by dividing them by standard deviation of each
datamart_scaled = datamart_rfm / datamart_rfm.std()
datamart_scaled.describe().round(2)
DataCamp Customer Segmentation in Python

Combining centering and scaling


Subtract mean and divide by standard deviation manually
Or use a scaler from scikit-learn library (returns numpy.ndarray object)
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(datamart_rfm)
datamart_normalized = scaler.transform(datamart_rfm)

print('mean: ', datamart_normalized.mean(axis=0).round(2))


print('std: ', datamart_normalized.std(axis=0).round(2))

mean: [-0. -0. 0.]


std: [1. 1. 1.]
DataCamp Customer Segmentation in Python

CUSTOMER SEGMENTATION IN PYTHON

Test different approaches


by yourself!
DataCamp Customer Segmentation in Python

CUSTOMER SEGMENTATION IN PYTHON

Sequence of structuring
pre-processing steps

Karolis Urbonas
Head of Data Science, Amazon
DataCamp Customer Segmentation in Python

Why the sequence matters?


Log transformation only works with positive data
Normalization forces data to have negative values and log will not work
DataCamp Customer Segmentation in Python

Sequence
1. Unskew the data - log transformation
2. Standardize to the same average values
3. Scale to the same standard deviation
4. Store as a separate array to be used for clustering
DataCamp Customer Segmentation in Python

Coding the sequence

Unskew the data with log transformation

import numpy as np
datamart_log = np.log(datamart_rfm)

Normalize the variables with StandardScaler

from sklearn.preprocessing import StandardScaler


scaler = StandardScaler()
scaler.fit(datamart_log)

Store it separately for clustering

datamart_normalized = scaler.transform(datamart_log)
DataCamp Customer Segmentation in Python

CUSTOMER SEGMENTATION IN PYTHON

Practice on RFM data!

You might also like