Emotion Detection from Facial Images Using Deep Learning Algorithms

Facial Emotion Recognition (FER) is a crucial task in the field of affective computing, with applications extending from human-computer interaction to mental health monitoring. This study proposes a hybrid FER framework that effectively combines both deep and shallow feature extraction techniques to enhance emotion classification accuracy. Deep features were extracted using several pretrained Convolutional Neural Networks (CNNs) architectures, including VGG16, VGG19, ResNet50, MobileNetV2, InceptionV3, EfficientNetB2, and a custom-designed CNN. In parallel, shallow features were obtained using handcrafted methods such as Local Binary Patterns (LBP), Histogram of Oriented Gradients (HOG), HSV color histograms, Wavelet and Contourlet transforms, Scale-Invariant Feature Transform (SIFT), and Gradient Local Autocorrelation (GLAC). These features were concatenated to form a comprehensive hybrid representation of facial expressions. Multiple machine learning classifiers, including K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Decision Tree, Random Forest, Gradient Boosting, and Extreme Gradient Boosting (XGB), Naïve Bayes, were evaluated using the hybrid feature set. Experimental results demonstrated that ensemble-based classifiers, particularly XGB, consistently achieved the highest accuracy, F1-score, and AUC values. The findings validate the effectiveness of hybrid features in capturing both low-level and high-level characteristics of facial expressions, leading to robust and accurate emotion recognition. This work affords a solid basis for future research in real- time FER systems and multimodal emotion analysis.

• Supervision