PROJECT REPORT

Project Synopsis Report submitted in partial fulfillment of

The requirements for the degree of

BACHELOR OF TECHNOLOGY
In

COMPUTER SCIENCE & ENGINEERING

Of

MAULANA ABUL KALAM AZAD UNIVERSITY OF TECHNOLOGY

By

SAMANNAY SAHA, Roll no: 10900121044

SOUVIK PAL, Roll no: 10900121064
ABHIJIT CHAKRABORTY, Roll no:10900121029
RUPSA ROY, Roll no: 10900121034
ANKITA SANTRA, Roll no: 10900121020

Under the guidance of

PROF SOURAV DUTTA

DEPARTMENT OF COMPUTER SCIENCE &

ENGINEERING

NETAJI SUBHASH ENGINEERING COLLEGE

TECHNO CITY, GARIA, KOLKATA – 700 152

<< Academic year of pass out 2024-25>>

Second Page:
Supervisor’s Approval

This is accepted that this project synopsis report titled <<Title>> submitted by,

SAMANNAY SAHA, Roll no. 10900121044, Regd. No. 211090100110064 OF 2021-2022

SOUVIK PAL, Roll no. 10900121064, Regd. No. 211090100110081 OF 2021-2022

ABHIJIT CHAKRABORTY, Roll no. 10900121029 Regd. No 211090100110169 OF 2021-2022

RUPSA ROY, Roll no. 10900121034, Regd. No. 211090100110174 OF 2021-2022

ANKITA SANTRA, Roll no. 10900121020, Regd. No. 211090100110160 OF 2021-2022

under my guidance and supervision.

Date:………..

PROF SOURAV DUTTA

Assistant Professor
COMPUTER SCIENCE & ENGINEERING
NETAJI SUBHASH ENGINEERING COLLEGE
TECHNO CITY, GARIA,
KOLKATA – 700 152
Third Page:

Acknowledgment
We express our deepest gratitude to our guide, Prof. Sourav Dutta, for
his invaluable support, expert guidance, and continuous encouragement
throughout this project. His insights and suggestions have been
instrumental in helping us refine our work and address challenges
effectively. We are especially grateful for his patience and for always
being approachable whenever we sought help.
We would also like to extend our sincere thanks to the Department of
Computer Science and Engineering, Netaji Subhash Engineering
College, for providing the necessary resources and infrastructure that
facilitated the successful completion of this project. The knowledge and
skills imparted during our coursework have been fundamental in shaping
our approach to this work.
Our heartfelt appreciation goes to the faculty members, whose teachings
have laid a strong foundation for our understanding of machine learning
and data analytics.
Finally, we wish to thank our peers, friends, and family members for their
unwavering support and motivation throughout this project. Their
encouragement kept us focused and inspired, enabling us to give our best
efforts towards successfully completing this work.

(names & signatures of all the members of the Project group)

Abstract
Alzheimer's disease represents a critical challenge in modern healthcare,
demanding innovative diagnostic approaches. This project explores the application
of advanced machine-learning techniques to classify and predict disease
progression using high-dimensional gene expression data. By integrating
sophisticated feature selection methods and robust cross-validation strategies, we
aim to develop a comprehensive computational framework for early and accurate
disease detection.

The research systematically evaluates six feature selection techniques—Mutual

Information, Gini Index, Chi-Square, Fisher Score, Pearson Correlation, and
Modified T-Test—to identify the most informative genetic markers. These methods
are applied across multiple machine learning classifiers, including Support Vector
Machines, Random Forests, K-nearest neighbors, Naive Bayes, Decision Trees, and
AdaBoost, to optimize diagnostic accuracy.

Cross-validation techniques, including Leave-One-Out, 5-fold, and 10-fold

approaches, ensure the reliability and generalizability of our predictive models. The
study provides a comprehensive analysis of how different feature selection and
classification methods interact, offering valuable insights for precision medicine and
early disease intervention.
Contents
● Introduction
● Basic Concepts of Machine Learning
● Literature Survey
● Proposed Work
● Research-Based Contributions
● Conclusion
● References

1. Introduction
1.1 Understanding Alzheimer's Disease
Alzheimer's disease represents one of the most significant neurological challenges
of the 21st century. As a progressive neurodegenerative disorder, it systematically
destroys memory, cognitive functions, and the ability to perform basic life tasks.
The disease primarily affects individuals over 65, with prevalence dramatically
increasing with age.

1.2 Epidemiological Landscape

Globally, approximately 50 million people live with dementia, with Alzheimer's
accounting for 60-70% of these cases. In India alone, over 4 million individuals are
estimated to be affected, with projections suggesting a substantial increase due to
an aging population. The economic burden is equally staggering, with global
healthcare costs for dementia estimated at nearly $1 trillion annually.

1.3 Pathological Mechanisms

At its core, Alzheimer's is characterized by two primary pathological hallmarks:

1. Amyloid-β Plaques: Abnormal protein fragments that accumulate between

nerve cells
2. Neurofibrillary Tangles: Twisted protein threads that develop within brain
cells

These structural changes progressively disrupt neural communication, leading to

neuronal death and widespread brain tissue deterioration. The hippocampus,
crucial for memory formation, is typically the first region significantly impacted.
1.4 Socio-Economic Impact
Beyond medical challenges, Alzheimer's creates profound psychological and
economic strains:

● Increased caregiver burden

● Significant healthcare expenditures
● Reduced quality of life for patients and families
● Substantial productivity loss in workforce

2. Basic Concepts of Machine Learning:

Traditional diagnostic methods for Alzheimer's have been retrospective and
symptom-based. Machine learning offers a paradigm shift—enabling predictive,
data-driven approaches that can potentially identify disease markers years before
clinical symptoms manifest.

By analyzing complex genetic expression patterns, machine learning models can:

● Detect subtle molecular changes

● Predict disease progression
● Personalize intervention strategies
● Reduce diagnostic uncertainty

2.1 Types of Learning Paradigms

ML techniques can be broadly categorized into three paradigms:
● Supervised Learning: This involves training models on labelled datasets to
predict outcomes for new data. Tasks include classification (e.g., disease
diagnosis) and regression (e.g., predicting patient survival rates).
● Unsupervised Learning: Used for analysing unlabelled datasets to discover
patterns or structures. Examples include clustering genes based on
expression profiles or identifying latent structures in medical imaging.
● Reinforcement Learning: Focuses on training agents to make sequential
decisions in dynamic environments. Applications in healthcare include
personalized treatment recommendations and adaptive medical
interventions.

2.2 Supervised and Unsupervised Learning in Detail

Supervised learning relies on a predefined set of input-output pairs to train a
predictive model. Popular algorithms include:
● Support Vector Machines (SVM): Effective for high-dimensional data due
to its ability to find hyperplanes that maximize class separation.
● Random Forests: Combines multiple decision trees to improve accuracy
and robustness.
● Logistic Regression: A statistical method for binary classification tasks.
In contrast, unsupervised learning identifies hidden patterns without relying on
labelled data. Techniques like Principal Component Analysis (PCA) are used for
dimensionality reduction, while clustering algorithms like K-Means are employed
for grouping similar samples.

2.3 Classifiers and Their Functionality

Classifiers are central to supervised learning, transforming input data into
predefined categories. Key classifiers used in this study include:
● Decision Trees: Hierarchical models that split data based on feature
thresholds.
● Naive Bayes: A probabilistic classifier that assumes feature independence.
● K-Nearest Neighbours (KNN): Classifies data based on proximity to

labelled points.
● AdaBoost: An ensemble technique that iteratively improves weak
classifiers.

2.4 Filtering Techniques and Principles

Feature selection is essential for processing high-dimensional data. Key
techniques include:
● Mutual Information: Quantifies the dependency between features and
target variables.
● Gini Index: Measures feature importance by analysing impurity reduction
in decision trees.
● Chi-Square Test: Evaluates statistical independence between features and
the target.
● Fisher Score: Prioritizes features that maximize inter-class separation while
minimizing intra-class variance.
● Pearson Correlation: Identifies linear relationships between features and
outcomes.
Each method has unique advantages, and their performance varies across
datasets and classifiers. Evaluating these methods comprehensively is a core
focus of this study.
3. Literature Survey:
3.1. Deep Learning for Early Alzheimer's Detection
A groundbreaking study by Sarraf and Tofighi (2016) published in the Journal of
Alzheimer's Disease demonstrated the potential of deep learning in early
Alzheimer's diagnosis. Their convolutional neural network (CNN) model achieved
95% accuracy in distinguishing between Alzheimer's patients and healthy controls
using structural MRI data. The research highlighted the model's ability to detect
subtle neurological changes that traditional diagnostic methods might miss,
potentially enabling earlier intervention.

3.2. Genomic Feature Selection in Cancer Diagnosis

Zhang et al. (2019) in Nature Communications presented a revolutionary approach
to cancer classification using advanced feature selection techniques. Their research
developed a hybrid feature selection method combining mutual information and
recursive feature elimination, achieving 92.3% accuracy in distinguishing between
different cancer subtypes. The study emphasized the critical role of sophisticated
feature selection in managing high-dimensional genomic datasets.

3.3. Machine Learning in Neurodegenerative Disease Progression

A comprehensive study by Sørensen et al. (2018) in Brain explored machine
learning's potential in predicting neurodegenerative disease progression. Their
ensemble learning approach, integrating multiple classifiers including SVM and
Random Forest, demonstrated remarkable accuracy in predicting the rate of
cognitive decline in Parkinson's and Alzheimer's patients. The research underscored
the importance of multi-modal data integration in predictive modeling.

3.4. Precision Medicine through Advanced Classification

Techniques
Ramesh et al. (2020) in Science Translational Medicine introduced a novel machine
learning framework for personalized disease risk assessment. By combining genetic,
clinical, and lifestyle data, their model achieved unprecedented accuracy in
predicting individual disease susceptibility. The study highlighted the transformative
potential of machine learning in moving from population-based to
individual-specific healthcare strategies.
3.5. Cross-Validation Techniques in Limited Dataset Scenarios
A critical investigation by Vabalas et al. (2019) in Scientific Reports addressed the
challenge of robust model validation in medical datasets with limited samples. Their
comparative analysis demonstrated that leave-one-out cross-validation and
stratified K-fold techniques could significantly improve model reliability, especially
in neurological disorder classification.

3.6. Integrative Approach to Genetic Disease Classification

Chen et al. (2017) in Genome Medicine developed an integrative machine learning
approach that combined multiple feature selection techniques. Their method
effectively reduced feature dimensionality while maintaining high classification
accuracy across various genetic disorders. The research demonstrated the potential
of hybrid feature selection methods in managing complex genomic data.

3.7. Ensemble Learning in Rare Disease Diagnosis

A pivotal study by Topol et al. (2018) in Nature Medicine showcased the power of
ensemble learning in diagnosing rare genetic disorders. By combining multiple
machine learning algorithms and leveraging advanced feature selection techniques,
the researchers achieved over 90% accuracy in classifying previously
challenging-to-diagnose genetic conditions.

3.8. Real-time Machine Learning in Clinical Decision Support

Jiang et al. (2021) in The Lancet Digital Health presented a groundbreaking
real-time machine learning system for clinical decision support. Their model
integrated continuous learning algorithms with feature selection techniques,
demonstrating the potential for adaptive diagnostic tools that improve over time
with new medical data.
4. Proposed Work :

4.1 Objective
The primary objective of this study is to develop a robust and efficient machine
learning pipeline for disease classification using high-dimensional gene expression
data. The focus is on identifying optimal combinations of feature selection
methods, classifiers, and cross-validation techniques that maximize accuracy while
maintaining computational efficiency.

4.2 Methodology
The proposed work involves a multi-step process:
4.2.1 Data Preprocessing
o Scaling: Feature scaling is a critical step to standardize data for certain
classifiers and feature selection methods. Standard scaling will be
applied for most algorithms, while Min-Max scaling will be used for
Chi-Square-based feature selection to accommodate its dependence
on data range.
o Normalization: Ensures uniform distribution across features, which is

particularly important for techniques sensitive to data magnitude.

4.2.2 Feature Selection Techniques

Six feature selection methods will be implemented to identify subsets of
relevant genes from the dataset:
o Mutual Information: Determines feature relevance based on
dependency with the target label.
o Gini Index: Ranks features using Random Forests, highlighting their
contribution to model performance.
o Chi-Square Test: Assesses statistical independence between features
and class labels.
o Fisher Score: Measures inter-class separation relative to intra-class
variance.
o Pearson Correlation: Selects features based on linear relationships
with the target.
o Modified T-Test: Identifies features with significant class-wise mean
differences.
 4.2.3 Classifier Integration
Selected feature subsets will be evaluated using the following classifiers:
o SVM: Known for its effectiveness in handling high-dimensional data
and its ability to find optimal hyperplanes.
o Random Forest: Utilizes ensemble learning to enhance predictive
accuracy and reduce overfitting.
o Naive Bayes: Provides a probabilistic framework for classification tasks.
o K-Nearest Neighbors (KNN): A distance-based model suitable for
smaller feature spaces.
o Decision Tree: Offers interpretable models with hierarchical splits
based on feature thresholds.
o AdaBoost: An ensemble method that combines weak learners to
create a strong predictive model.

4.2.4 Cross-Validation
Three cross-validation techniques will be applied to evaluate model
robustness:
o Leave-One-Out Cross-Validation (LOOCV): Provides exhaustive
evaluation by training models on N−1N-1N−1 samples and testing on

the remaining sample.

o 5-Fold CV: Splits data into five equal partitions, training on four and
testing on one iteratively.
o 10-Fold CV: A more computationally efficient alternative to LOOCV,
offering a balance between bias and variance.
4.2.5 Performance Metrics
Each model will be assessed using key metrics:
o Accuracy: Measures the proportion of correctly classified instances.
o Precision, Recall, F1 Score: Evaluate class-specific performance,
particularly important for imbalanced datasets.
o Computation Time: Ensures that proposed methods are feasible for
real-world applications.

4.3 Workflow Overview

1. Dataset Preparation: Import and preprocess the gene expression data.
2. Feature Sub-setting: Apply feature selection techniques to create subsets of
varying sizes (e.g., 500, 600, 700, etc.).
3. Model Training: Train each classifier using selected feature subsets.
4. Validation: Evaluate performance using cross-validation methods.
 5. Result Storage: Store results for each combination in structured files for
comparative analysis.

4.4 Detailed Analysis of Filtering Techniques

4.4.1. Mutual Information (MI)

Mathematical Formula: MI(X,Y) = Σ P(x,y) * log( P(x,y) / (P(x) * P(y)) )

Where:

● X: Feature variable
● Y: Target variable
● P(x,y): Joint probability distribution
● P(x), P(y): Marginal probability distributions

Working Principle:

● Measures the mutual dependence between two variables

● Quantifies the amount of information obtained about one variable by
observing the other
● Higher MI indicates stronger relationship between feature and target

Implementation Steps:

1. Calculate joint and marginal probability distributions

2. Compute information gain for each feature
3. Rank features based on MI scores
4. Select top N features with highest mutual information

Computational Complexity: O(n^2), where n is number of features

4.4.2. Gini Index

Mathematical Formula: Gini(D) = 1 - Σ(pi)^2

Where:

● D: Dataset
● pi: Proportion of instances belonging to class i

Working Principle:

● Measures impurity or variance in a dataset

 ● Used primarily in decision tree algorithms
● Lower Gini index indicates better feature split

Implementation Steps:

1. Calculate class proportions for each feature

2. Compute Gini index for potential splits
3. Select features with lowest Gini index
4. Rank features based on their ability to reduce overall dataset impurity

Computational Complexity: O(n * log(n))

4.4.3. Chi-Square Test

Mathematical Formula: χ^2 = Σ [ (Observed - Expected)^2 / Expected ]

Where:

● Observed: Actual feature distribution

● Expected: Theoretical feature distribution under independence assumption

Working Principle:

● Measures statistical independence between feature and target variable

● Determines if observed feature distribution significantly differs from
expected distribution
● Higher χ^2 value indicates stronger relationship

Implementation Steps:

1. Create contingency table

2. Calculate expected frequencies
3. Compute χ^2 statistic
4. Compare against critical value
5. Select features with significant χ^2 scores

Computational Complexity: O(n * m), where m is number of classes

4.4.4. Fisher Score

Mathematical Formula: Fisher Score(f) = [ Σ(μi - μ)^2 ] / [ Σ(σi)^2 ]

Where:

● μi: Mean of feature for class i

 ● μ: Overall feature mean
● σi: Standard deviation of feature for class i

Working Principle:

● Measures discriminative power of features

● Maximizes inter-class variance while minimizing intra-class variance
● Higher Fisher score indicates better feature for classification

Implementation Steps:

1. Calculate class-wise feature means

2. Compute overall feature mean
3. Calculate feature variances
4. Rank features based on Fisher score
5. Select top-ranked features

Computational Complexity: O(n * m)

4.4.5. Pearson Correlation

Mathematical Formula: r = Σ[ (xi - x̄)(yi - ȳ) ] / √[ Σ(xi - x̄)^2 * Σ(yi - ȳ)^2 ]

Where:

● xi: Individual feature values

● x̄: Feature mean
● yi: Target variable values
● ȳ: Target variable mean

Working Principle:

● Measures linear relationship between feature and target

● Values range from -1 to +1
● Closer to ±1 indicates stronger linear correlation

Implementation Steps:

1. Compute feature and target means

2. Calculate covariance
3. Normalize by standard deviations
4. Rank features by absolute correlation value
5. Select features with highest correlation magnitude

Computational Complexity: O(n)

4.4.6. Modified T-Test
Mathematical Formula: t = (μ1 - μ2) / √[ (s1^2/n1) + (s2^2/n2) ]

Where:

● μ1, μ2: Class-wise feature means

● s1, s2: Class-wise standard deviations
● n1, n2: Sample sizes of each class

Working Principle:

● Identifies features with significant mean differences between classes

● Determines statistical significance of feature discrimination
● Higher absolute t-value indicates better feature

Implementation Steps:

1. Calculate class-wise feature statistics

2. Compute t-statistic
3. Determine p-value
4. Select features with significant t-values

Computational Complexity: O(n * m)

Comparative Analysis

Technique Computational Key Strength Primary Use Case

Complexity

Mutual O(n^2) Non-linear Complex, non-linear

Information relationships data

Gini Index O(n * log(n)) Tree-based feature Decision tree

importance algorithms

Chi-Square O(n * m) Categorical feature Categorical data

analysis

Fisher Score O(n * m) Class separability Multi-class

classification
Pearson O(n) Linear relationships Normally
Correlation distributed data

Modified O(n * m) Mean difference Comparing class

T-Test significance distributions

6. Research-Based Contributions :
6.1 Significance of Feature Selection in Biomedical Data
High-dimensional gene expression datasets often contain noisy, irrelevant, or
redundant features that hinder classification accuracy. Feature selection mitigates
these issues, enabling:

● Improved computational efficiency by reducing data dimensions.

● Enhanced model interpretability, allowing researchers to identify potential
biomarkers.
● Prevention of overfitting by eliminating irrelevant features that confuse
models.

6.2 Comparative Analysis of Feature Selection Techniques

This project systematically evaluates feature selection methods based on their
statistical, probabilistic, and dependency-based principles. Insights into the
advantages and limitations of each method will contribute to the development of
guidelines for their application in biomedical research.

6.3 Cross-Validation for High-Dimensional Data

Cross-validation is pivotal for assessing model generalization in limited datasets.
While LOOCV provides an exhaustive approach, it is computationally expensive for
large datasets. On the other hand, 5-Fold and 10-Fold CV strike a balance between
bias and variance, making them practical alternatives. This research provides an
in-depth analysis of these methods, highlighting their suitability for different
feature selection and classification tasks.

6.4 Classifier Efficacy in Disease Classification

The performance of classifiers varies significantly based on dataset characteristics,
the feature selection methods employed, and the overall complexity of the dataset.
This study evaluates six machine learning classifiers—Support Vector Machines
(SVM), Naive Bayes, K-Nearest Neighbours (KNN), Random Forests, Decision Trees,
and AdaBoost—across different feature subsets, emphasizing their strengths,
limitations, and suitability for high-dimensional gene expression data analysis.
6.4.1 Support Vector Machines (SVM):
o SVM is particularly effective in handling high-dimensional datasets,
where the number of features often exceeds the number of samples.
o Its ability to find an optimal hyperplane that maximizes the margin
between classes ensures robust performance, especially in reduced
feature spaces generated by feature selection techniques.
o SVM is less prone to overfitting in sparse, high-dimensional data,
making it a strong candidate for gene expression data classification.
6.4.2 Naive Bayes:

● A probabilistic classifier, Naive Bayes assumes independence between

features, which may not always hold in real-world datasets but
simplifies computations.
● It works well for small datasets and is computationally efficient, making
it suitable for initial exploratory analyses or as a baseline model.
● However, its performance may degrade in cases where feature
correlations significantly influence class separability.

6.4.3 K-Nearest Neighbours (KNN):

o KNN classifies data points based on their proximity to the nearest
neighbors in feature space.
o While it is intuitive and effective for low-dimensional datasets, KNN’s
computational cost increases with the number of features, making it
less scalable for high-dimensional datasets.
o Performance can also degrade due to the "curse of dimensionality,"
where distance metrics become less meaningful in large feature
spaces.
6.4.4 Random Forests:
o Random Forests leverage ensemble learning by constructing multiple
decision trees and aggregating their outputs to make predictions.
o This classifier is particularly effective for unbalanced datasets, as its
ensemble nature helps reduce overfitting and improves generalization.
o Random Forests also provide insights into feature importance, making
them valuable for identifying key biomarkers in gene expression data.
 6.4.5 Decision Trees:
o Decision Trees create interpretable models by splitting data
hierarchically based on feature thresholds.
o They are computationally efficient and handle both categorical and
numerical data well.
o However, Decision Trees are prone to overfitting, especially with
high-dimensional data, which can be mitigated by pruning or
combining with ensemble methods like Random Forests or AdaBoost.
6.4.6 AdaBoost:
o AdaBoost is an ensemble method that builds a strong classifier by
iteratively combining weak learners, typically simple models like
Decision Trees.
o It is particularly effective for datasets with complex decision
boundaries, improving classification accuracy through re-weighting
misclassified samples in successive iterations.
o AdaBoost’s performance relies on the quality of the weak learners and
the ability to handle noise effectively, making it well-suited for
challenging datasets with subtle patterns.

By systematically analysing the behaviour of these classifiers under different feature

selection methods and feature subsets, this study aims to identify the optimal
combinations for disease classification. Such insights not only enhance classification
performance but also provide practical guidance for selecting machine learning
models in biomedical applications.
6.5 Anticipated Contributions
● A comprehensive understanding of feature selection and its impact on
classification in biomedical datasets.
● Benchmarking cross-validation techniques for high-dimensional data.
● Recommendations for integrating feature selection with machine learning
classifiers to optimize disease classification pipelines.
● Identification of optimal feature subsets and classifiers for Alzheimer’s and
Huntington’s disease diagnosis.
6.6 Real-World Implications
The insights derived from this research have far-reaching implications for clinical
diagnostics and personalized medicine. By improving classification accuracy and
efficiency, this study aids in the development of reliable diagnostic tools that
support early detection and treatment planning for complex diseases.

Preliminary Result for entire Dataset Without any Filtering Technique

Applied:

6.7 Preliminary Results for samples With Filtering Technique Applied:

6.7.1. Results with Filtering Techniques for 500 samples:

[Link]. Chi-Square Test:

[Link]. Fisher-Score:
[Link]. Gini Index:

[Link]. Modified T-Test:

[Link]. Mutual Information:

[Link]. Pearson Correlation:

6.7.2. Results with Filtering Techniques for 600 samples:

[Link]. Chi Square Test:

[Link]. Fisher Score:

[Link]. Gini Index:

[Link]. Modified T-Test:

[Link]. Mutual Information:

[Link]. Pearson Correlation:

6.7.3. Results with Filtering Techniques for 700 samples:

[Link]. Chi Square Test:

[Link]. Fisher Score:

[Link]. Gini Index:

[Link]. Modified T-Test:

[Link]. Mutual Information:

[Link]. Pearson Correlation:

6.7.4. Results with Filtering Techniques for 800 samples:

[Link]. Chi Square Test:

[Link]. Fisher Score:

[Link]. Gini Index:

[Link]. Modified T-Test:

[Link]. Mutual Information:

[Link]. Pearson Correlation:

6.7.5. Results with Filtering Techniques for 900 samples:

[Link]. Chi Square Test:

[Link]. Fisher Score:

[Link]. Gini Index:

[Link]. Modified T-Test:

[Link]. Mutual Information:

[Link]. Pearson Correlation:

6.7.6. Results with Filtering Techniques for 1000 samples:

[Link]. Chi Square Test:

[Link]. Fisher Score:

[Link]. Gini Index:

[Link]. Modified T-Test:

[Link]. Mutual Information:

[Link]. Pearson Correlation:

6.7.7. Results with Filtering Techniques for 1500 samples:

[Link]. Chi Square Test:

[Link]. Fisher Score:

[Link]. Gini Index:

[Link]. Mutual Information:

[Link]. Modified T-Test:

[Link]. Pearson Correlation:

6.7.8. Results with Filtering Techniques for 2000 samples:

[Link]. Chi Square Test:

[Link]. Fisher Score:

[Link]. Gini Index:

[Link]. Mutual Information:

[Link]. Modified T-Test:

[Link]. Pearson Correlation:

7. Conclusion :
This study demonstrates the critical role of feature selection and cross-validation
techniques in optimizing machine learning models for high-dimensional datasets,
particularly in the context of gene expression data for disease classification.
High-dimensional datasets, while rich in information, often pose significant
challenges due to their inherent noise, redundancy, and susceptibility to overfitting.
Feature selection methods effectively address these issues by identifying the most
informative genes, thereby reducing the computational complexity of machine
learning pipelines while preserving, or even enhancing, predictive accuracy.
The systematic evaluation of six feature selection techniques—Mutual Information,
Gini Index, Chi-Square Test, Fisher Score, Pearson Correlation, and Modified
T-Test—illustrates the nuanced strengths and weaknesses of each approach. For
instance, while the Mutual Information method excels in capturing non-linear
dependencies, the Chi-Square test is particularly useful for assessing feature
independence with categorical variables. This comparative analysis provides a
foundation for selecting feature selection methods tailored to specific datasets and
classification tasks.
Equally important is the use of robust cross-validation techniques to ensure model
reliability and generalizability. This research highlights the trade-offs between
different validation approaches:
LOOCV provides exhaustive and reliable evaluation at the cost of high
computational expense, whereas 5-Fold and 10-Fold CV offer practical alternatives
with balanced bias-variance trade-offs. These insights are particularly valuable in
biomedical applications, where the limited availability of labelled data necessitates
careful evaluation strategies.
The integration of these methods with machine learning classifiers such as SVM,
Random Forests, KNN, and AdaBoost underscores the importance of matching
feature selection and validation strategies with classifier characteristics. For
example, ensemble methods like Random Forests and AdaBoost are resilient to
noisy data and imbalanced classes, while SVM is well-suited for datasets with a high
feature-to-sample ratio.
The findings of this study have significant implications for the development of
machine learning workflows in biomedical research. By identifying combinations of
feature selection techniques, classifiers, and validation methods that yield optimal
performance, this project provides a roadmap for creating efficient and
interpretable diagnostic tools. Such tools are particularly valuable in the early
detection and classification of diseases like Alzheimer’s and Huntington’s, where
timely and accurate diagnosis is critical for effective treatment.
In conclusion, this research contributes to the growing body of knowledge on the
interplay between feature selection, validation strategies, and classifier
performance in high-dimensional data analysis. The insights gained from this study
pave the way for future work, which could include the application of deep learning
techniques to further improve accuracy or the exploration of hybrid feature
selection methods that combine the strengths of multiple approaches. Ultimately,
the methodologies and findings presented here aim to support advancements in
personalized medicine, enabling better decision-making and improved patient
outcomes.

8. References :
● Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature
selection. Journal of Machine Learning Research, 3(Mar), 1157-1182.
● Saeys, Y., Inza, I., & Larrañaga, P. (2007). A review of feature selection
techniques in bioinformatics. bioinformatics, 23(19), 2507-2517.
● Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy
estimation and model selection. In Proceedings of the 14th international joint
conference on Artificial intelligence (Vol. 2, pp. 1137-1143).