An Efficient Deep Learning based Real Time Facial Expression Recognition System

Sharmeen M.Saleem; Subhi R. M. Zeebaree; Maiwan B. Abdulrazzaq

doi:10.25212/lfu.qzj.9.4.55

Authors

Sharmeen M.Saleem Department of Information Technology, College of Informatics Akre, Duhok Polytechnic University, Duhok, Kurdistan Region, Iraq
Subhi R. M. Zeebaree Department of Energy Engineering, Technical College of Engineering, Duhok Polytechnic University, Duhok, Kurdistan Region, Iraq
Maiwan B. Abdulrazzaq Department of Computer Science, Faculty of Science, University of Zakho, Duhok, Kurdistan Region, Iraq.

DOI:

https://doi.org/10.25212/lfu.qzj.9.4.55

Keywords:

Deep Learning, CNN, Real Time, Facial Expression Recognition, RAF Dataset.

Abstract

Human-computer interaction, emotion analysis, and more depend on computer vision's face expression recognition. Real-time facial expression identification enhances decision-making in human-computer interaction, healthcare, marketing, and other fields. This study introduces a fast and accurate real-time facial expression identification system utilizing Convolutional Neural Network (CNN) and Mobile Network Version 2 (MobileNetV2). The proposed methodology involves the design and implementation of a CNN architecture tailored for facial expression recognition. We strategically arrange layers, fine-tune parameters, and leverage transfer learning techniques, particularly focusing on the optimization of model depth and complexity. The model is trained and evaluated using benchmark datasets to ensure robust performance across various facial expressions. Data-driven infrastructure helps the system manage real-world changes. Our approach identifies facial emotions in real time from live footage. This paper explores facial expression recognition employing a simplified CNN architecture, contrasting with more complex pre-trained networks like MobileNetV2. Despite its simplicity, the proposed CNN yields result close to the performance of MobileNetV2. The study emphasizes the viability of less intricate CNNs for facial expression recognition, offering a balance between model simplicity and competitive accuracy. High accuracy and computational efficiency make the system suitable for real-time applications and resource-constrained systems The experimental tests extracted that a very acceptable accuracy achieved when using CNN model with the AffectNet dataset with a ratio of (97.2%).

Downloads

Download data is not yet available.

References

Kong, Y., et al., Real‐time facial expression recognition based on iterative transfer learning and efficient attention network. IET Image Processing, 2022. 16(6): p. 1694-1708.

Hassouneh, A., A. Mutawa, and M. Murugappan, Development of a real-time emotion recognition system using facial expressions and EEG based on machine learning and deep neural network methods. Informatics in Medicine Unlocked, 2020. 20: p. 100372.

Ekman, P., Cross-cultural studies of facial expression. Darwin and facial expression: A century of research in review, 1973. 169222(1).

Ekman, P. and W.V. Friesen, Constants across cultures in the face and emotion. Journal of personality and social psychology, 1971. 17(2): p. 124.

Alom, M.Z., et al., A state-of-the-art survey on deep learning theory and architectures. electronics, 2019. 8(3): p. 292.

Khan, A., et al., A survey of the recent architectures of deep convolutional neural networks. Artificial intelligence review, 2020. 53: p. 5455-5516.

Li, S. and W. Deng, Deep facial expression recognition: A survey. IEEE transactions on affective computing, 2020. 13(3): p. 1195-1215.

Gill, R. and J. Singh. A deep learning approach for real time facial emotion recognition. in 2021 10th International Conference on System Modeling & Advancement in Research Trends (SMART). 2021. IEEE.

Ozdemir, M.A., et al. Real time emotion recognition from facial expressions using CNN architecture. in 2019 medical technologies congress (tiptekno). 2019. IEEE.

Pathar, R., et al. Human emotion recognition using convolutional neural network in real time. in 2019 1st International Conference on Innovations in Information and Communication Technology (ICIICT). 2019. IEEE.

Dukić, D. and A. Sovic Krzic, Real-time facial expression recognition using deep learning with application in the active classroom environment. Electronics, 2022. 11(8): p. 1240.

Talegaonkar, I., et al. Real time facial expression recognition using deep learning. in Proceedings of international conference on communication and information processing (ICCIP). 2019.

Singh, S.K., et al. Deep learning and machine learning based facial emotion detection using CNN. in 2022 9th International Conference on Computing for Sustainable Global Development (INDIACom). 2022. IEEE.

Dwijayanti, S., M. Iqbal, and B.Y. Suprapto, Real-time implementation of face recognition and emotion recognition in a humanoid robot using a convolutional neural network. IEEE Access, 2022. 10: p. 89876-89886.

Saleem, S.M., S.R. Zeebaree, and M.B. Abdulrazzaq. Real-life dynamic facial expression recognition: a review. in Journal of Physics: Conference Series. 2021. IOP Publishing.

Zhou, N., R. Liang, and W. Shi, A lightweight convolutional neural network for real-time facial expression detection. IEEE Access, 2020. 9: p. 5573-5584.

Minaee, S., M. Minaei, and A. Abdolrashidi, Deep-emotion: Facial expression recognition using attentional convolutional network. Sensors, 2021. 21(9): p. 3046.

Lee, D.-H. and J.-H. Yoo, CNN Learning Strategy for Recognizing Facial Expressions. IEEE Access, 2023.

Tiwari, T., T. Tiwari, and S. Tiwari, How Artificial Intelligence, Machine Learning and Deep Learning are Radically Different? International Journal of Advanced Research in Computer Science and Software Engineering, 2018. 8(2): p. 1.

Tiwari, T., T. Tiwari, and S. Tiwari, How Artificial Intelligence, Machine Learning and Deep.

Arsenov, A., et al. Evolution of Convolutional Neural Network Architecture in Image Classification Problems. in ITS. 2018.

Yamashita, R., et al., Convolutional neural networks: an overview and application in radiology. Insights into imaging, 2018. 9: p. 611-629.

Sakib, S., et al., An overview of convolutional neural network: Its architecture and applications. 2019.

Bambharolia, P. Overview of Convolutional Neural Networks. in Proceedings of the International Conference on Academic Research in Engineering and Management, Monastir, Tunisia. 2017.

Howard, A.G., et al., Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017.

Goodfellow, I.J., et al. Challenges in representation learning: A report on three machine learning contests. in Neural Information Processing: 20th International Conference, ICONIP 2013, Daegu, Korea, November 3-7, 2013. Proceedings, Part III 20. 2013. Springer.

Li, S., W. Deng, and J. Du. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.

Mollahosseini, A., B. Hasani, and M.H. Mahoor, Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing, 2017. 10(1): p. 18-31.

Demir, F., Deep autoencoder-based automated brain tumor detection from MRI data, in Artificial Intelligence-Based Brain-Computer Interface. 2022, Elsevier. p. 317-351