Emotion Recognition from Facial Images Using Combination of Deep and Shallow Features with Machine Learning Algorithms

Emotion recognition from face images including several facial expressions plays a crucial role in various applications, including human-computer interaction, affective computing, and mental health assessment. In this project, we will introduce an approach with the combination of deep features and shallow features to detect emotion from facial Images. Specifically, we will adopt the Convolutional Neural Network (CNN) as the deep feature and texture, color, and shape features for shallow feature space. Different wavelet transforms, contourlet transforms, hue saturation value (HSV), local binary pattern (LBP), scale invariant feature transform (SIFT), histogram of oriented gradient (HOG), and gradient local autocorrelation will be considered as the shallow features. The focus of our work revolves around leveraging deep learning techniques to accurately identify and categorize emotions based on facial expressions. This project will deal with deep learning as well as machine learning algorithms for identifying the category of emotion from facial expressions. Softmax classification, multilayer perceptron, decision tree, random forest, k-nearest neighbor, Naïve-Bayes, Gradient Boosting, eXtreme Gradient Boosting, and support vector machine will be utilized as the classification stage.  For the experimental evaluation, we will employ the publicly available data sets (e.g., FER2013 dataset, MS-Kinect, CK+). Detection will be made based on some statistical measures where recall, precision, F1 score, and accuracy will be measured from the confusion matrix. We will analyze the experimental results using the receiver operating characteristic (ROC) curve and area under the ROC curve (AUC). 

• Research Project