Multimodal emotion recognition from audio and video

Nithyasri S *, Hemavarthini B and Bharathi N. Gopalsamy

Computer Science and Engineering, SRM Institute of Science and Technology, Vadapalani, Chennai, India.
 
Review
International Journal of Science and Research Archive, 2024, 12(01), 142-149.
Article DOI: 10.30574/ijsra.2024.12.1.0723
Publication history: 
Received on 14 March 2024; revised on 26 April 2024; accepted on 29 April 2024
 
Abstract: 
As humans we want to interact with a machine as we would with a person, in a way that it understands us, advises us, and looks after us with no human supervision. Current systems lack empathy and user understanding in spite of very effective logical reasoning. By predicting the emotions of the users, we are able to identify their needs and cater to them as best as possible. Emotion recognition in video and audio has many potential applications, including conversational agents, recommendation systems as well as systems for smart homes, mental illness care, virtual reality games, remote physical training, education and car-hailing services. The aim of the project is to develop an automatic emotion detection system based on voice and facial expression. We propose a model that highlights contextual, multimodal information for emotion detection and recognition. If systems can understand emotions and respond accordingly to behavioral patterns, we can anticipate artificial agents becoming our cognitive-consulting partners in our daily lives. Additionally, the project can be expanded to make interactions more natural and better suited to handle complex situations.
 
Keywords: 
Multimodal Emotion Recognition; Computer Vision; Deep Learning; Audio and Video; OpenCV; Librosa
 
Full text article in PDF: 
 
Copyright information: 

Copyright © 2024 Author(s) retain the copyright of this article. This article is published under the terms of the Creative Commons Attribution Liscense 4.0