International Journal of Scientific & Engineering Research, Volume 4, Issue 8, August-2013 760

ISSN 2229-5518

Classification Using K Nearest Neighbor for Brain

Image Retrieval
P.A.Charde, S.D.Lokhande

Abstract— We propose the algorithm for the retrieval of the most visually similar images to a given query image from a database of
medical images by content. In this algorithm we take shape feature extraction by canny Edge detection and texture feature extraction by
using Gabor filter. Gabor filter is best feature extraction method for texture. And on the basis of these feature, medical images are classified
using KNN method. The retrieval performance of the proposed system is tested using large medical image database of about 500
computed tomography images of brain. The retrieval performance and retrieval complexity are measured and evaluated.

Index Terms—Canny Edge Detection, Gabor Filter, K Nearest neighbour classifier, Medical images, Precision, Recall, Accuracy.
——————————  ——————————

1 INTRODUCTION

I mage Retrieval has been active research area over the last
25 years, but first review articles on access methods in Im-
age database appeared in the early 80s. This section gives
The gradient intensity changes rapidly in the edge and
the maximal intensity change along a particular orientation
produces a peak or a zero-crossing. Hence, the first derivative
an Introduction to CBIR systems [1, 2, 3] and the features used and the second derivative of the gradient of every pixel in an
in these systems. The basic idea behind content-based image image are used to find edges in the image.
retrieval is that, when building an image database, or retriev- Canny method has proven to be superior over many of the

IJSER
ing an image from the database, we first extract features from available edge detection algorithms and thus was chosen
images, the features can be color, shape, texture etc. These fea- mostly for real time implementation and testing. Canny edge
tures are stored in the database for future use. When given a detection algorithm was introduced in 1986. It is considered as
query image, we similarly extract its features, and match these the modern "standard" in the sense that the validity of all oth-
vectors with those already in the database, if the distance be- er algorithms is often checked against it.
tween two feature vectors is small enough then we consider Texture is the key component of human visual perception.
the corresponding image in the database match the query. Everyone can recognize texture, but it is more difficult to de-
Retrieval results are ranked according to similarity index and fine. Texture is an important but difficult to describe feature in
group of similar target images are presented to users. an image. Many techniques have been developed for measur-
In most of the image retrieval systems, a query is specified ing texture similarity. Most technique compares second order
by an image to be matched. We refer to this as an overall statistics of query and stored images. These method calculate
search, as similarity is based on the overall properties of imag- measures of image texture such as the degree of contrast,
es. By contrast, there are also partial search querying systems coarseness, directionality and regularity [7, 8]. Alternative
that retrieve based on a particular region in an image. methods of texture analysis for image retrieval include the use
Content based medical image retrieval system has been of Gabor filters [5] have shown the image retrieval using Ga-
found useful in medical applications and the medical domain, bor features outperforms that using pyramid-structured wave-
as one of the principal application domains for content based let transform features, tree structured wavelet transform fea-
technologies. tures and multi-resolution simultaneous autoregressive model
Shape is the fundamental visual features in content based features.
image retrieval .Edge information can be used to feed-in vari- This paper describes the medical image retrieval system us-
ous applications seeking the shapes, size, or edge locations of ing shape feature extraction by canny edge detection and tex-
particular objects. In all edge detection algorithms, the main ture feature extraction by Gabor Filter. These features are used
objective is to locate the edge (intensity transitions) from the for training and classification using K nearest neighbor classi-
scene neither with prior information nor with human interpre- fier. Algorithm performance has been measured by the Preci-
tation. sion and Recall measures.
Some popular algorithms include Sobel, Roberts, Prewitt,
Laplace, LOG, and Canny Algorithm [9]. These edge detection
operators share almost the same concept which is to find the
2 PROPOSED METHOD
singularities and locate them accurately. The proposed method has training and classification phases.
In training phase, from a given set of training images the
_________________________________ shape and texture features are extracted and used to train the

P.A.Charde is currently pursuing masters degree program in Electronics

Engineering (Digital System) in University of Pune, India,
E-mail: prachiti.charde@gmail.com.
S.D.Lokhande is currently working as a Principal in Sinhgad College of En-
gineering in University of Pune, India.E-mail: sdlokhande.scoee@sinhgad.edu

IJSER © 2013
http://www.ijser.org
International Journal of Scientific & Engineering Research, Volume 4, Issue 8, August-2013 761
ISSN 2229-5518

Images for training Pre-processed image Feature Extraction

KNN
Classifier

Images for testing Pre-processed image Feature Extraction

Classification Label

Fig 1: A Block diagram of the proposed method using KNN

system using the K-nearest neighbor classifier. In classification Canny Edge Detector is referred to as “hysteresis”. It makes
phase a given test ct of brain image pre-processed and then use of both a high threshold and a low threshold.
texture features are extracted for classification. These features Step 5: Thinning: Using interpolation to find the pixels where
are queried to K-nearest neighbor classifier to label an un- the norms of gradient are local maximum.
known image. The block diagram of the proposed method is The following Fig 2 depicts the results after applying
given in Figure 1. Canny Edge Detection Algorithm.

IJSER
2.1 Pre-processing
Pre-processing includes conversion of Input image into gray Original Image Gaussian Filtered Image
scale image and noise removal. In this work median filter used
for de-noising.
In median filtering, the neighbouring pixels are ranked ac-
cording to brightness (intensity) and the median value ones
the new value becomes the new value for the central pixel. A
median filter offers three advantages
1. No reduction in contrast across steps since output values Magnitude of gradient Vectors Image after thresholding

available consists only those presents in the neighborhood.

2. Median filtering does not shift boundaries.
3. The median is, in a sense a more robust average that the
mean, as it is not affected by outliers.
2.2 Feature Extraction
The purpose of feature extraction is to reduce the original data Image Edges
set by measuring certain properties of features that distinguish
on input pattern from another.These extracted features used
for preparation of Training data. These features are saved into
feature library.

2.2.1 Shape Feature

Algorithm: For detecting edges using Canny Edge Detection Fig. 2: A Result after applying Canny Edge Detection
Algorithm
Step 1: Smoothing: Smooth the image with a two dimensional
Gaussian. In most cases the computation of a two dimensional 2.2.2 Texture Feature
Gaussian is costly, so it is approximated by two one dimen- Gabor filter [5] is one of the most popular signal processing
sional Gaussians. based approach for texture extraction. Basically, Gabor filters
Step 2: Finding Gradients: Take the gradient of the image this are a group of wavelets, with each wavelet capturing energy at
shows changes in intensity, which indicates the presence of a specific frequency and specific direction. It acts as a local
edges. This actually gives two results, the gradient in the x band pass filter with certain optimal joint localization proper-
direction and the gradient in the y direction. ties in both the spatial domain and the frequency domain. An
Step 3: Non-maximal suppression: Edges will occur at points image is filtered with a set of Gabor filters with different pre-
where the gradient is at a maximum. The magnitude and di- ferred orientations and spatial frequencies and the features,
rection of the gradient is computed at each pixel. which are obtained from a feature vector, is used further. Tex-
Step 4: Edge Threshold: The method of threshold used by the ture features are found by calculating the mean and variation

IJSER © 2013
http://www.ijser.org
 International Journal of Scientific & Engineering Research, Volume 4, Issue 8, August-2013 762
ISSN 2229-5518

of the Gabor filtered image. A feature vector f which represents the texture is created using
For a given image I(x, y) with size PxQ, its discrete Gabor μ mn and σ mn as the feature components. In our implementa-
wavelet transform is given by a convolution: tion we have used 4 scales and 6 orientations and the feature
vector is given by f = (μ00, σ00, μ01, σ01… μ35, σ35). An exam-
ple of a filter bank generated is shown in the figure 3.
( x, y )
Gmn= ∑ ∑ I ( x − s , y − t )ψ * ( s , t )
x y
mn (1)

where s and t are filter mask size variables and * mn Ψis the
complex conjugate of Ψmn which is a class of self-similar
functions generated from dilation and rotation of the follow-
ing mother wavelet:
1  1  x2 y 2 
ψ (=
x, y ) exp  −  2 + 2   .exp ( j 2π Wx ) (2)
2πσ xσ y  2  σ σ  
Where W is the modulation frequency. The self similar Gabor
wavelets are obtained through the generating function.

x a − m ( x cos θ + y sin θ ) (3)

Fig.3: Filter Response of A Gabor Filter Bank with 4 Scale and
y =a (− x sin θ + y cos θ )
−m
(4) 6 Orientation

where a>1 andθ = nΠ/N. The variables in the above equation

IJSER
2.3 Classification
are defined as follows:
1 K-Nearest Neighbour classification technique [4] is the sim-
a = (U h / U l ) M −1 (5) plest technique conceptually and computationally that pro-
vides good classification accuracy. The K-NN algorithm is
Wmn = a U l m
(6) based on a distance function and a voting function in k-
Nearest Neighbours, the metric employed is the Euclidean
σ xmn =
( a + 1) 2 ln 2 (7)
distance. The K-NN has higher accuracy and stability for MRI
2π a m ( a − 1) U l data than other common statistical classifiers, but has a slow
running time. Yet, the issues of poor run time performance is
1 not such a problem these days with the computational power
σ ymn = 2
(8) that is available [6]
U h2  1 
 π  The k-nearest neighbour classifier is a conventional non-
2π tan   −  parametric supervised classifier that is said to yield good per-
 2 N  2 ln 2  2πσ xmn 
formance for optimal values of k. Like most guided learning
In our implementation, we used the following constants: algorithms, K-NN algorithm consists of a training phase and a
Ul = 0.05, Uh = 0.4, s and t range from 0 to 60. After applying testing phase. In the training phase, data points are given in a
Gabor filters on the image with different orientation and at n-dimensional space. These training data points have labels
different scale, we obtain an array of magnitudes: associated with them that designate their class.
K-NN algorithm comprises of following stages:
E ( m.n ) = ∑∑ Gmn ( x, y ) (9) 1. Determine a suitable distance metric.
x y
2. In the training phase: Stores all the training data set P in
=m 0,1....., M −=
1; n 0,1....., N − 1 pairs (according to the selected features) P = (y i ; c i ), i = 1.
Where, y i is a training pattern in the training data set, c i is its
These magnitudes represent the energy content at different scale and corresponding class and n is the amount of training patterns.
orientation of the image. Texture feature are found by calculating the 3. During the test phase: Computes the Distances between
mean μ mn and standard deviation σ mn of the energy magnitude. the new feature vector and all the stored features (training
data).
E (m, n) 4. The k nearest neighbours are chosen and asked to vote
µ mn = (10) for the class of the new example. The correct classification giv-
P×Q en in the test phase is used to assess the correctness of the al-
gorithm. If this is not satisfactory, the k value can be tuned
until a reasonable level of correctness is achieved.
∑ ∑ ( G ( x, y ) − µ )
2
mn mn

σ mn =
x y
(11) 3 EXPERIMENTAL RESULT
P×Q This section depicts the retrieval effectiveness of the proposed

IJSER © 2013
http://www.ijser.org
International Journal of Scientific & Engineering Research, Volume 4, Issue 8, August-2013 763
ISSN 2229-5518

Fig. 4: Representative image of each class

system. The medical image retrieval system was implemented

using MATLAB version 7.13.0.564 with Pentium–IV processor
on Windows platform. Retrieval performance in terms of Pre-
cision and Recall of proposed system was tested by conduct-
ing experiments on database consists of 500 medical images of
CT scan of brain. These images are finely defined into 10 cate-
gories as shown in figure 4.Most images are grey level images
converted from the DICOM format into BMP. Ten images

IJSER
from the database collection representing various anatomical
regions were chosen as the query images for evaluation of the
system. Precision (P) is used to represent the retrieval accuracy
which is the percentage of similar images retrieved with re-
spect to the number of retrieved images and Recall(R) is the
percentage of relevant images among all possible relevant im-
ages. Precision and Recall is defined as:

number of relevant images retrieved

P= (12)
total number of images retrieved
number of relevant images retrieved Fig. 5(a): GUI of proposed system using Combined Feature
R= (13)
total number of relevant image in database

By the above equation we can conclude that precision and

Recall measures the retrieval accuracy. It is calculated as the
average of 10 queries. Table 1 shows Retrieval Performance of
Canny Edge Detection, Gabor Filter and Combining both
Method obtained by varying the values of n1. Fig 5(a), 5(b),
6(a) and6 (b) gives the screen shot of the retrieval of a sample
query image using Combined Method and Gabor Filter. Fig.7
shows Average Precision and Recall graph of Canny Edge, GF
and combining both method.
Classification rate is defined as the percentage of images
classified to i. Table 1 shows theretrieval performance using
average precision and recall values for 10 randomly selected
queries. Table 2 shows that a comparision of classification
using KNN for different value of k. Table 3 shows that images
of each class in testing set are almost classified to its true class
or its four most adjacent classes. The average of classification
rate of the KNN classifier is 52.30%.
We apply this classifier to automatically classify images in
our brain image database.. In the process of retrieval, the im-
ages in database, whose class label are in the scope of [i-3, i+3],
are selected to compare with the query image which is sup- Fig.5 (b): Retrieved images for the query image using combined feature
in fig. 5(a)
posed to have been classified into i-th class. Applying this

IJSER © 2013
http://www.ijser.org
International Journal of Scientific & Engineering Research, Volume 4, Issue 8, August-2013 764
ISSN 2229-5518

classifier in our system can prevent retrieving images which

are not similar to the query image on anatomical structure and Table1. Retrieval Performance of Canny Edge Detection, Gabor Filter and
content from the database. Combining both Method

Total GF CED Combined Method

number
of re- Average Average Average Average Average Average
trieved Precision Recall Precision
Recall Precision Recall
images

5 100 10.74 96 10.34 100 10.75

10 97 20.88 76 16.64 97 20.88

15 93.33 30.18 57.33 18.68 93.33 30.19

20 88.5 30.28 45.5 19.70 89 38.49

25 80.8 43.97 40 21.52 81.6 44.37

30 72 47.43 36.33 23.34 72 47.44

35 63.14 48.23 32.85 24.57 63.14 48.44

40 56.25 49.83 30.5 25.97 56.75 49.65

45 50.67 49.86 29.33 27.97 51.56 50.70

50 46.4 50.69 28 29.57 46.4 50.70

Fig. 6(a): GUI of proposed system using Gabor Feature

IJSER
Average Precision

Average Recall

Fig.6 (b): Retrieved images for the query image using Gabor
Feature in fig. 5(a) Fig.7: Average Precision and Recall graph of Canny Edge, GF and com-
bining both method

Table 2: A comparision of Classification Using KNN for different

values of k
KNN
k=11 k=13 k=15
Average of Classifica-
96.363 95.38 93.33
tion Rate (%)

IJSER © 2013
http://www.ijser.org
International Journal of Scientific & Engineering Research, Volume 4, Issue 8, August-2013 765
ISSN 2229-5518

Table 3: Detailed Result of Classification Using KNN

Predicted

Classification
Accuracy (%)
Class

Class 10
Class 1

Class 2

Class 3

Class 4

Class 5

Class 6

Class 7

Class 8

Class 9

Total
True
Class 1 12 12 13 5 8 50 24

Class 2 23 11 14 1 49 46.94
Class 3 35 3 8 4 50 70

Class 4 16 25 6 3 50 50
Class 5 12 31 7 50 62

Class 6 26 15 7 48 31.25
Class 7 7 20 23 50 40
Class 8 4 11 4 30 1 50 60
Class 9 3 25 2 19 49 38.78

IJSER
Class 10 30 30 100
Average (%) 52.30

[6] Li Ma, Crawford, M.M, Jinwen Tian, " Local Manifold Learning-Based (k-
CONCLUSION
NN) for Hyper spectral Image Classi_cation", IEEE Transaction,
In this paper, we proposed a method of medical image re- 48(11),2010,pp.4099-4109
trieval and classification based on Shape feature by Gabor fea- [7] W. Niblack, R. Barber, W. Equitz, M. Flickner, E. Glasman, D. Petkovic, P.
ture and use this feature to do the image retrieval and classifi- Yanker, C. Faloutsos and G. Taubin. The QBIC Project: Querying Image By
cation. Results of experimentation show that the texture fea- Content Using Color, Texture and Shape. In Proc. SPIE Storage and Retrieval
tures mentioned in this paper can fully describe the content of for Image and Video Databases, vol.1908, pp.173-187, 1993.
image, and improve the recall and precision rate and classifi- [8] F. Liu and R. W. Picard. Periodicity Directionality and Randomness: Wold
cation accuracy on medical image retrieval. In the system im- Features for Image Modelling and Retrieval. IEEE Trans. PAMI-18(7):722-733,
plementation process, the features of images are stored in the 1996.
image database uniformly. When users submit queries, the [9] H.M Zelelew, A.T Papagiannakis,"A volumetric thresholding algorithm for
system retrieves all the images in database, and then returns processing asphalt concrete x-ray CT images". International journal of pave-
the result ment engineering, Sept. 2007.

REFERENCES
[1] C. Faloutsos, R. Barber, M.Flickner, J,Hafner, W. Niblack, D.Petkovic, W.
Equitz. Efficient and effective querying by image content. Journal of Intell. Inf.
Syst. 3 (3-4), 231-262, 1994.
[2] A. Pentland, R.W.Picard, S.Scaroff. Photobook: Content-based manipulation
for image databases. International Journal of Computer Vision 18 (3), 233-254,
1996.
[3] F. Long, H.J Zhang, D.D Feng. Fundamentals of content-based image retriev-
al. Multimedia Information Retrieval and Management, Springer, Berlin,2003.
[4] Min-Ling, Zhi-Hua Zhou,” A K-nearest neighbour based algorithm for mul-
tilabel classification.” IEEE International Conference of Granular Compu-
ting,July 2005; pp: 718-72
[5] B. S. Manjunath and W. -Y. Ma. Texture Features for Browsing and Retrieval
of Large Image Data. IEEE Trans. PAMI-18 (8):837-842, August, 1996.

IJSER © 2013
http://www.ijser.org