11 V May 2023

https://doi.org/10.22214/ijraset.2023.51433
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue V May 2023- Available at www.ijraset.com

Survey on Facial Emotion Recognition using Deep

Learning
Rachana G Raikar1, Sushma R.B 2, Rakshitha C3, Shreegowri S4, V Indushri5
1, 3, 4, 5
UG Students, Dept. of CS&E, JNNCE, Shimoga, India
2
Assistant Prof., Dept. of CS&E, JNNCE, Shimoga, India

Abstract: The facial emotion recognition that extracts the expression of human which conveys a lot of information visually. This
plays a crucial role in the human-machine interaction with which other applications are implemented in several fields of
technology. Facial emotion recognition is performed in four stages consisting of pre-processing, face detection, feature
extraction and expression classification. This concept includes various deep learning methods (convolutional neural network) to
identify the seven key emotions namely: happy, sad, angry, fear, surprise, disgust and neutral. Recognising the facial expression
by computer with high recognition rate is still a challenging job in this current industry. Some of the applications using these
recognition systems are beyond the implementation stage that may include human behaviour understanding, detection of mental
disorders, and synthetic human expressions. This research uses FER2013 dataset which helps to clearly classify the seven
expressions that is proven to be effective for human emotion recognition.
Keywords: facial expression recognition, feature points, image processing, expressions, emotion analysis, Support Vector
Machine

I. INTRODUCTION
In the modern technology our desires went huge and it has no bounds. The research mainly going on through the digital image and
image processing fields. The way of progress and implementation is drastically increasing exponentially. The processing of image is
a vast area of research now-a-days along with its applications.
Image processing helps to output the expression of image along with the specification of emotion that the image considers as input.
Our emotion is revealed by the expressions in our face. Facial Expressions plays an important role in analysing the application or
situation. Facial expression is a non-verbal scientific gesture which gets expressed in our face as per our emotions. Automatic
recognition of facial expression plays an important role in artificial intelligence and robotics and thus it is a need of the generation.
Some application related to this includes security, law enforcement and many more for tracking purpose.
This project mainly works to develop the facial emotion recognition automatically using the considered emotions from the dataset
that is implemented into the input and classified it into seven different expression classes such as:
1) Neutral
2) Angry
3) Disgust
4) Fear
5) Happy
6) Sadness
7) Surprise

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 291
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue V May 2023- Available at www.ijraset.com

Support Vector Machine (SVM) is a supervised classification algorithm where a line is drawn between two different categories to
differentiate between them. This model is a powerful tool to identify predictive model or classifiers. Existing system shows
accuracy more than 90% when SVM (Support Vector Machine) classifier is used.
The technologies used for human computer interaction are increasing day by day. Facial emotion recognition is one among the
trending technologies that are implemented here. Facial expressions are fastest means of communication while conveying any type
of information. Recently, devices interact with human along with the performance of coding. But, if machine can extract real time
data of the human it is interacting with, it would help to improvise human machine interaction. FER, if implemented, may lead to a
natural Human Machine Interface (HMI).

Also, Human emotions helps in the implementation of:

 Training AI agents
 To make better decisions for employees.
 To improve focus and performance.

II. LITERATURE REVIEW

Harisu Abdullahi Shehu at el. [1] Face expression recognition has become on of the important area of research in research years.
Some of the miscellaneous regularization methods have been utilized in order help combat overfitting, to reduce training time and to
generalize their model. Haar-cascade classifier is feature-based object detection algorithm to detect objects from images. This Haar-
cascade classifier is used to crop the faces and focuses on region of interest, it hypothesizes that it would attain a fast convergence
without using the whole image to analyse facial expressions. HC is used to identify the faces in the input image or the images are
analysed through the dataset. The detected faces are cropped and resized into particular pixel i.e., 128x128 pixels. The cropped
images or faces are then fed into the model for training and emotion analysis of input image. The labels are given to the image and
analyse its effect on databases. Facial emotion recognition is a trending area of research that has gained attention in the past few
decades due to increase in the usage of cameras.
Noel Jaymon at el.[2] This paper has proposed methodology which can be split into two stages i.e., Model Training and Real time
detection. The dataset is imported into pandas data-frame using pandas library through which all pre-processing open was conducted.
The image representation is done through the pixel value. The images and also live video inputs are adapted for input which is saved,
and the input is converted into grayscale images, that helps to fasten the process.
K.S Chandraprabha at el.[3] Real time Employee Emotion Detection System (RtEED) which uses machine learning to detect the
emotions of the employees, it is a real-time detection using image processing with human-friendly machine interaction. RtEED
system helps the employer to depict the emotions and produce information on the personalities and thoughts of the employees.
RtEED system is designed to identify the basic six emotions that are extracted from the dataset such as: happy, sad, surprise, anger,
fear and disgust. These helps to output the accurate results.
Wafa Mellouka at el.[4] Facial expression is mainly implemented in several areas such as safety health and human machine
interaction. Researchers in this area are interested to interpret different techniques, to code facial expressions and extract these
features for having better predictions by computer as an end result. Different architectures are exploited in this field in order to
examine the better model which is able to execute the accurate result. The researchers concentrated on automatic facial emotion
recognition i.e., FER via deep learning for better performance of the considered architecture. The purpose of this is to serve and
guide researchers by reviewing recent works and providing insights to make improvements.
Table-1: Comparison of datasets
Training Testing Accuracy
FER2013 CK+ 76.05
FER2013 JAFFE 50.70
Nithya Roopa S at el. [5] Facial emotion recognition is a part of expression detection which is gaining lots of importance in this
current world. The need for it and usage of facial expression have been tremendously increasing day by day. There may be many
methods to identify expressions using machine learning and artificial intelligence techniques, these help in the classification of
image which helps in recognition and classification of expressions according to the images that are given as input to the system.
Various datasets are verified and investigated for training expression recognition models are explained in this paper. Inception Net is
used for emotion recognition with Kaggle and Karolinska Directed Emotional Faces datasets.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 292
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue V May 2023- Available at www.ijraset.com

Table-2: Summary of survey

TITLE METHODOLOGY ADVANTAGES LIMITATIONS

Facial Recognition Deep learning Neural High accuracy More memory

Using Deep Learning Network
[1]

Real Time Emotion OpenCV Improving learning performance It is a challenge to make

Detection Using Deep emotion available in
Learning [2] Manage stress. different languages.

Adopt healthier and more Different types and

productive working styles. versions of the software
such as dataset input is only
textual data, image, pattern

Real time-Employee Machine learning It provides a simple architecture Dataset input is only textual
Emotion Detection for developers. data, image, pattern, video
System (RtEED) Using and audio inputs are
Machine Learning [3] Improving learning performance invalid.

It is a challenge to make
emotion available in
different languages.

Facial Recognition Deep Convolutional To increase the diversity of the A difficult and sensitive
Using Deep Learning Neural Network images and eliminate the task in order to have a
[4] overfitting problem. better classification.

Recognition rate varies from one Learning only the six-basic

database to another with the same emotion plus neutral.
DL model.

Emotion Recognition CNN-LSTM Recognition of emotions, using Convolutional neural

From Facial Expression different classes of classifiers networks is not infused for
Using Deep emotions from images
Learning[5] Helps in human-computer
interaction

III. CONCLUSION
Classification of human expressions along with static facial images using deep learning techniques which is considered as one of the
difficult tasks to identify emotions on diversified face expression from different angles. This system works with live capturing of
images and captured series of images with which the classification of expressions or emotions are neatly observed. SVM is a
supervised classification algorithm where a line is drawn between two different categories to differentiate between them. SVM
classifier used in the system gives the expected output properly. So, this paper concentrates on future learning, face and facial
component detection and feature extraction with expression classification by using deep learning algorithm.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 293
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue V May 2023- Available at www.ijraset.com

REFERENCES
[1] M. Hossain and G. Muhammad, "Emotion Recognition Using Deep Learning Approach from Audio-Visual Emotional Big Data," Information Fusion, vol. 49,
pp. 69- 78, September 2019.
[2] Nithya Roopa. S, "Emotion Recognition From Facial Expression Using Deep Learning", Vol.8, pp.91-95, August 2019.
[3] Wafa Mellouka, Wahida Handouzia, "Facial emotion recognition using deep learning", pp.690-697, August 2020.
[4] Md. Forhad Ali, Mehenag Khatun, Nakib Aman Turzo, "Facial Emotion Detection Using Neural Network", International Journal of Scientific & Engineering
Research Volume 11ISSN 2229-5518, 2020.
[5] Samadhi P. K. Wickrama Arachchilage and Ebroul Izquierdo, "Deep-learned faces: a survey", Arachchilage and Izquierdo EURASIP Journal on Image and
Video Processing, PP.2-33, 2020
[6] Ninad Mehendale, ―Face Emotion Recognition Using Convolutional Neural Networkǁ, Feb 2020
[7] Anna Pecchinenda, Vivian Ciaramitaro, Jun Moriya, Daniel Fitousi, "Stereotypical Processing of Emotional Faces:Perceptual and Decisional Components",
Article in Frontiers in Psychology,Volume 12, Article 733432, pp. 1-25., October 2021
[8] Harisu Abdullahi Shehu, Md. Haidar Sharif and Sahin Uyaver, "Facial Expression Recognition Using Deep Learning", AIP Conference Proceedings 2334,
070003 (2021), 02 March 2021
[9] Md.Rabiul Islam, Md.Ali Moni, Md.Rashed-AI-Mahfuz,―Emotion Recognition From EEG Signal Focusing on Deep Learning and Shallow Learning
Techniquesǁ, June 2021.
[10] Noel Jaymon, Sushma Nagdeote, Ayush Yadav, Ryan Rodrigues, ―Real Time Emotion Detection Using Deep Learningǁ, Feb 2021.
[11] K.S Chandraprabha, A.N Shwetha, M.Kavitha, R.Sumathi, ―Real time-Employee Emotion Detection System (RtEED) Using Machine Learningǁ ,March 2021.
[12] Rashid Jahangir, Ying Wah Teh, Faiqa Hanif, Ghulam Mujtaba,―Deep Learning approaches for Speech Emotion Recognition: State of the art and research
challengesǁ July 2021.
[13] Ke Zhang, Yuanqing Li, Jingyu Wang, Erik Cambria, “Real-Time Video Emotion Recognition Based on Reinforcement Learning and Domain Knowledgeǁ
April 2021.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 294