Uma abordagem para reconhecimento de emoção por expressão facial baseada em redes neurais de convolução

Abstract

Developing the emotional perception of computers is a technological trend. Emotion recognition composes cognitive systems with applicability in several areas. Facial expression is an effective way to recognize emotions, especially because it is less intrusive in data collection when compared to other methods, and because it is easy to obtain facial images in view of the popularization of cameras. Through facial expressions it is possible to classify the group of basic emotions (joy, fear, surprise, sadness, disgust and anger) and neutrality. Currently, convolution neural networks (CNN) have been the state of the art for image classification. Given this context, this dissertation presents an approach to recognize facial expression emotions using CNN called Single Shot Facial Expression Recognition (SSFER) and its use in a case study. Initially, an experimental study was conducted to evaluate four face detectors in facial expression bases and in VOC-2007. The MMOD-CNN method was the best, reaching 91.89% accuracy. Subsequently, another experimental study was conducted to compare five CNN architectures by alternating four classifiers in the last layer to classify facial expressions. The CNNs were: VGGNet, InceptionResNetV2, InceptionV3, MobileNetV2 and ResidualNet, and the classifiers: Softmax, SVM, Random Forest and KNN. The idea is for CNN to function as a feature puller by sending a one-dimensional vector for the classifier to define emotion. The best combination was VGGNet with SVM reaching 78.95% accuracy. Thus, the proposed approach (SSFER) outperformed the Microsoft Cognitive Services API by 9.74% in a comparison by evaluating facial expression bases. Overall, the joy and surprise emotions had the highest accuracy rates. In contrast, the fear and anger emotions achieved the lowest accuracy rates. A case study was performed in a real scenario focused on digital education. Twenty-seven high school students participated in order to answer an ENEM mock on a digital platform. During the test the students' facial expressions were collected, as well as all interactions with the platform. After the simulated, facial expressions were processed to correlate with click interactions and test performance. Data analysis suggests that neutrality may be related to the state of concentration and that students spend most of their time in the state of neutrality. The state of surprise can be confused with yawning allowing the recognition of drowsiness. And the students who scored the highest on the exam had the lowest surprise detection rate. Finally, the proposed approach has been shown to be positive for use in general applications and in particular in digital education.