11 V May 2023

https://doi.org/10.22214/ijraset.2023.52016
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue V May 2023- Available at www.ijraset.com

Human Emotion Detection Using Deep Learning

Mr. Pandarinathan1, S.Velan2, S. Deepak3
1, 2, 3
Department of Computer Science and Engineering, Sri Muthukumaran Institute of Technology, Chennai-69

Abstract: Human emotions are the mental state of feelings and are spontaneous. There is no clear connection between emotions
and facial expressions and there is significant variability making facial recognition a challenging research area. Emotion
recognition becomes a popular topic of deep learning and provides more application fields in our daily life. The primary idea of
our project is to process the input images of human facial emotion to train the pre-trained models on datasets. In recent
years, Machine Learning (ML) and Neural Networks (NNs) have been used for emotion recognition. In this project, a
Convolutional Neural Network (CNN) is used to extract features from images to detect emotions. The numerical result of the
algorithm will show a probabilistic result of each labeled class.

I. INTRODUCTION
Facial emotions are important factors in human communication that help to understand the intentions of others. In general, people
infer the emotional state of other people, such as joy, sadness and anger, using facial expressions. Facial expressions are one of the
main information channels in interpersonal communication. Therefore, it is natural that facial emotion research has gained a lot of
attention over the past decade with applications in perceptual and cognitive sciences . Interest in automatic Facial Emotion
Recognition (FER) has also been increasing recently with the rapid development of Artificial Intelligent (AI) techniques. They are
now used in many applications and their exposure to humans is increasing. To improve Human Computer Interaction (HCI) and
make it more natural, machines must be provided with the capability to understand the surrounding environment, especially the
intentions of humans. Machines can capture their environment state through cameras and sensors. In recent years, Deep Learning
(DL) algorithms have proven to be very successful in capturing environment states. Emotion detection isnecessary for machines to
better serve their purpose since they deliver information about the inner state of humans. A machine can use a sequence of facial
images with DL techniques to determine human emotions.

II. LITERATURE SURVEY

1) Facial Emotion Detection Using Deep Learning
Arpankumar Patel, Ansel Joseph, ShubhamSurvase, Rohini Nair
Abstract: Human Emotion detection from image is one of the most powerful and challenging research task in social communication.
Deep learning (DL) based emotion detection gives performance better than traditional methods with image processing. This paper
presents the design of an artificial intelligence (AI) system capable of emotion detection through facial expressions. It discusses
about the procedure of emotion detection, which includes basically three main steps: face detection, features extraction, and emotion
classification. This paper proposed a convolutional neural networks (CNN) based deep learning architecture for emotion detection
from images. The performance of the proposed method is evaluated using two datasets Facial emotion recognition challenge
(FERC-2013) and Japaness female facial emotion (JAFFE). The accuracies achieved with proposed model are 70.14 and 98.65
percentage for FERC2013 and JAFFE datasets respectively.

2) A Lightweight Convolutional Neural Network For Real-Time Facial Expression Detection

Abstract: In this paper our group proposes and designs a lightweight convolutional neural network (CNN) for detecting facial
emotions in real-time and in bulk to achieve a better classification effect. We verify whether our model is effective by creating a
real-time vision system. This system employs multi-task cascaded convolutional networks (MTCNN) to complete face detection and
transmit the obtained face coordinates to the facial emotions classification model we designed firstly. Then it accomplishes the task
of emotion classification.
Multi-task cascaded convolutional networks have a cascade detection feature, one of which can be used alone, thereby reducing the
occupation of memory resources. Our expression classification model employs Global Average Pooling to replace the fully
connected layer in the traditional deep convolution neural network model. Each channel of the feature map is associated with the
corresponding category, eliminating the black box characteristics of the fully connected layer to a certain extent.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 2225
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue V May 2023- Available at www.ijraset.com

At the same time, our model marries the residual modules and depth-wise separable convolutions, reducing large quantities of
parameters and making the model more portable. Finally, our model is tested on the FER-2013 dataset. It only takes 3.1% of the
16GB memory, that is, only 0.496GB memory is needed to complete the task of classifying facial expressions. Not only can our
model be stored in an 872.9 kilobytes file, but also its accuracy has reached 67% on the FER-2013 dataset. And it has good
detection and recognition effects on those figures which are out of the dataset

III. OBJECTIVES
In this project, we use a deep learning technique called Convolutional Neural Networks(CNNs) to establish a classification model
that combines feature extraction with classification to detect the facial emotions. To develop a facial expression recognition system.
To experiment machine learning algorithm in computer vision fields. To detect emotion thus facilitating Intelligent Human-
Computer Interaction.

IV. METHODOLOGY
1) Problem Statement
2) Proposed Systems
With the goal to improve the process of facial sentiment analysis systems, a classification mechanism is proposed using a CNN
architecture. The goals of this project are to establish a model that can classify 7 basic emotions: happy, sad, surprise, angry,
disgust, neutral, and fear..
3) Advantage of ProposedSystem
a) High picture quality improves the effectiveness of facial recognition
b) Even low resolution photographs are usable with the suggested method
c) Higher accuracy while being more computationally efficient.

V. MODULES
A. Face Capturing Module
Image Acquisition: Images used for facial expression recognition are static images or image sequences. Images of face can be
captured using camera. Face detection: Face Detection is useful in detection of facial image. Face Detection is carried out in
training dataset using Haar classifier called Voila-Jones face detector and implemented through Opencv. Haar like features encodes
the difference in average intensity in different parts of the image and consists of black and white connected rectangles in which the
value of thefeature is the difference of sum of pixel values in black and white regions.

B. Pre-Processing Module
Image pre-processing includes the removal of noise and normalization against the variation of pixel position or brightness.
1) Color Normalization
2) Histogram Normalization
The amount of training data directly affects the final recognition effect. The larger the amount of data, the better the effect.

C. Training Module
Deep learning training is when a deep neural network (DNN) “learns” how to analyze a predetermined set of data and make
predictions about what it means. It involves a lot of trial and error until the network is able to accurately draw conclusions based on
the desired outcome.

D. Face Recognition Module

A Convolutional neural network (CNN)is a type of artificial neural network that has one or more convolution layers and are used
mainly for image processing, classification, segmentation and also for other auto correlated data. Deep learning is a machine
learning based artificial neural network that recognize objects in image by progressively extracting features from data through
higherlayers.As shown in figure in order to recognize face in an image we have to train the CNN with human faces.
The benefit of using CNNs is their ability to develop an internal representation of a two- dimensional image. This allows the model
to learn position and scale of faces in an image. After train the CNN it can able to recognize face in an image One can effectively
use ConvolutionalNeural Network for Image data. CNN that extractsfeatures in an image.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 2226
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue V May 2023- Available at www.ijraset.com

E. Expression Detection Module

General expression recognition methods include image reprocessing, facial feature extraction, and expression recognition. The
reprocessing stage of image expression recognition performs face detection to obtain facial region images. The recognition of
expressions in low-pixel facial images also requires image enhancement or image super resolution during reprocessing. Image
enhancement is to enhance the existing information of the image information by adding pixel. We use CNN for image recognition

VI. FLOWDIAGRAM

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 2227
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue V May 2023- Available at www.ijraset.com

VII. BLOCKDIAGRAM

VIII. ALGORITHM USED

Convolutional Neural Networks (CNN, or ConvNet) are a type of multi-layer neural networkthat is meant to discern visual patterns
from pixel images. In CNN, ‘convolution’ is referred to as the mathematical function. It’s a type of linear operation in which you
can multiply two functionsto create a third function that expresses how one function’s shape can be changed by the other. In simple
terms, two images that are represented in the form of two matrices, are multiplied to provide an output that is used to extract
information from the image. CNN is similar to other neural networks, but because they use a sequence of convolutional layers, they
add a layer of complexity to the equation. CNN cannot function without convolutional layers. In a variety of computer vision tasks,
CNN artificial neural networks have risen to the top. It has picked people’s interest in a variety of fields. A convolutional neural
network is made up of numerous layers, such as convolution layers, pooling layers, and fully connected layers, and it uses a back
propagation algorithm to learn spatial hierarchies of data automatically and adaptively. You will learn more about these terms in the
following section

IX. CONCLUSION
This project proposes an approach for recognizing the category of facial expressions. Face Detection and Extraction of expressions
from facial imagesis useful in many applications, such as robotics vision, video surveillance, digital cameras, security and human-
computer interaction. This project’s objective was to develop a facial expression recognition system implementing the computer
visions and enhancing the advanced feature extraction and classification in face expression recognition. In this project, seven
different facial expressions of different persons’ images from different datasets have been analyzed. This project involves facial
expression preprocessing of captured facial images followed by feature extraction using feature extraction using Local Binary
Patterns and classification of facial expressions based on training of datasets of facial images based on CNN.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 2228
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue V May 2023- Available at www.ijraset.com

REFERENCES
[1] Chu, William Wei-Jen Tsai, Hui-Chuan, YuhMin Chen and Min-Ju Liao. Facial expression recognition with transition detection for students with high-
functioning autism in adaptive e- learning.” Soft Computing: 1-27, 2017.
[2] Tzirakis, George Trigeorgis, Mihalis A. Nicolaou, Panagiotis, Bjrn W. Schuller, and Stefanos Zafeiriou. ”End-to-end multi-modal emotion recousing neural
networks.” IEEEJournal of Topics in Signal Processing 11, no. 8: 13011309, 2017
[3] Gampala, V., Kumar, M.S., Sushama, C. and Raj, E.F.I., 2020. Deep learning based image processing approaches for image deblurring. Materials Today:
Proceedings.
[4] Aneja, Gary Faigin, Deepali, Alex Colburn, Barbara Mones, and Linda Shapiro. ”Modeling stylized character expressions via deep learning.”In Asian
Conference on Computer Vision, pp.136-153. Springer, 2016
[5] Natarajan, V.A., Kumar, M.S., Patan, R., Kallam, S. and Mohamed, M.Y.N., 2020, September. Segmentation of Nuclei in Histopathology images using Fully
Convolutional Deep Neural Architecture. In 2020 International Conference on Computing and InformationTechnology (ICCIT1441)
[6] J. Nicholson, K. Takahashi, and R. Nakatsu. Emotion recognition in speech using neural networks. Neural computing applications, 9(4): 290-296, 2000.
[7] B. Fasel and J. Luettin. Automatic facial expression analysis: a survey. Pattern recognition, 36(1):259-275, 2003.
[8] A. Krizhevsky and G. Hinton. Learningmultiple layers of features from tiny images, 2009.
[9] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition,
2009. CVPR 2009. IEEEConference on, pages 248-255. IEEE, 2009.
[10] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural infor- mation
processing systems, pages 1097-1105, 2012.
[11] Y. Lv, Z. Feng, and C. Xu. Facial expression recognition via deep learning. In Smart Computing (SMARTCOMP), 2014 International Conference on, pages
303-308. IEEE, 2014.
[12] TFlearn. Tflearn: Deep learning libraryfeaturing a higher-level api for tensorflow. URL http://tflearn.org/.
[13] Open Source Computer Vision Face detectionusing haar cascades. URL:http://docs.opencv.org/master/d7/d8b/tutorialpyfacedetection.html

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 2229