|
Abstract.
The classification of emotional states (facial emotion recognition) is crucial for human-computer interaction, but its accuracy is often hampered by data variability and class imbalance. This paper proposes a method for creating a lightweight neural network to classify eight emotions, achieving high accuracy and a balanced F1-score. The method utilizes a combined dataset comprising over 127,937 samples from various known image collections, featuring an improved class distribution, and employs an optimized neural network architecture. Training involved data augmentation, early stopping, and hyperparameter optimization. The model achieved an accuracy of 80.6% and a macro-averaged F1-score of 0.806 on the test set, with per-class F1-scores ranging from 0.683 (sadness) to 0.907 (contempt). The developed lightweight model, with its low computational requirements, is suitable for integration into human-computer interaction, healthcare, and psychology systems, as well as for resource-constrained devices, ensuring reliable emotion recognition.
Keywords:
emotion recognition, emotion classification, lightweight neural network, EfficientNet-B3, deep learning, transfer learning, computer vision, affective computing.
DOI 10.14357/20718632260106
EDN OALXVH
PP. 66-79.
References
1. Lv Y, Feng Z. Facial Expression Recognition via Deep Learning. In: International Conference on Multimedia and Expo (ICME). 2014. 2. Wen Z, Lin W, Wang T, Xu G. Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition. MDPI Electronics. 2021;8(2):199. 3. Hu J, Shen L, Sun G. Squeeze-and-Excitation Networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2018. p. 7132–7141. 4. Wang K, Peng X, Yang J, Meng D, Qiao Y. Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition. arXiv. 2019. Available from: https://arxiv.org/abs/1905.04075 [Accessed 05 February 2026]. 5. Zhang Z, Gu J. Facial Affect Recognition in the Wild Using Multi-Task Learning Convolutional Network. arXiv. 2020. Available from: https://arxiv.org/abs/2002.00606 [Accessed 05 February 2026]. 6. Wang K, Peng X, Yang J, Meng D, Qiao Y. Suppressing Uncertainties for Large-Scale Facial Expression Recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020. 7. Farzaneh AH, Qi X. Facial Expression Recognition in the Wild via Deep Attentive Center Loss. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2021. 8. Antoniadis P, Kollias D, Zafeiriou S. Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognition. In: IEEE International Conference on Automatic Face and Gesture Recognition (FG). 2021. 9. Xue M, Wang Y, Li Y, Wang S, Li Z, Wang Y. TransFER: Learning Relation-Aware Facial Expression Representations with Transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2021. 10. Wang Y, Li S, Wang Y, Li Z, Wang Y. Adaptive Patch Selection to Improve Vision Transformers through Patch Attention. Appl Intell. 2022. 11. Li Y, Wang Y, Li Z, Wang Y. An Attentional Residual Feature Fusion Mechanism for Sheep Face Recognition. Sci Rep. 2022;12:43580. 12. She J, Hu Y, Shi H, Wang J, Shen Q, Mei T. Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021. 13. Zhang Y, Wang C, Ling X, Deng W. Learn From All: Erasing Attention Consistency for Noisy Label Facial Expression Recognition. In: European Conference on Computer Vision (ECCV). 2022. 14. Zeng D, Yan X, Wang F, Tang B, Liu L. Face2Exp: Combating Data Biases for Facial Expression Recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022. 15. Ruan D, Yan Y, Lai S, Chai Z, Shen C, Wang H. Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021. 16. Li Y, Wang Y, Li Z, Wang Y. Knowledge Transfer Network for Facial Expression Recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022. 17. Zhao X, Wang Y, Li Z, Wang Y. Learning Deep Global Multi-Scale and Local Attention Features for Facial Expression Recognition in the Wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021. 18. Li Y, Wang Y, Li Z, Wang Y. MVT: Mask Vision Transformer for Facial Expression Recognition in the Wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021. 19. Wang Y, Li S, Wang Y, Li Z, Wang Y. PSR: Progressive Self-Refinement for Facial Expression Recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022. 20. Zhang Y, Wang C, Ling X, Deng W. Relative Uncertainty Learning for Facial Expression Recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021. 21. Li Y, Wang Y, Li Z, Wang Y. TAN: Transformer-based Attention Network for Facial Expression Recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022. 22. Ma X, Zhang Y, Wang Y. Facial Expression Recognition with Visual Transformers and Attentional Selective Fusion. arXiv. 2022. Available from: https://arxiv.org/abs/2103.16854 [Accessed 05 February 2026]. 23. Liu Y, Zhang H, Wang Z. A Dual-Direction Attention Mixed Feature Network for Facial Expression Recognition. Electronics. 2023;12(17):3595. 24. Kim S, Lee J, Park H. Progressive Label Distillation for Facial Expression Recognition. Pattern Recognit. 2023;142:109579. 25. Wang J, Li Q, Zhao Y. POSTER: Pose-aware Spatial-Temporal Network for Facial Expression Recognition. Comput Vis Image Underst. 2023;223:103525. 26. Wang J, Li Q, Zhao Y. POSTER++: Enhanced Pose-aware Spatial-Temporal Network for Facial Expression Recognition. Pattern Recognit. 2023;142:109580. 27. Tan M, Le QV. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In: Proceedings of the 36th International Conference on Machine Learning. 2019;97:6105–6114.
|
