On emotion detection and recognition using a context-aware approach by social robots : modification of faster R-CNN and YOLO v3 neural networks

Abstract

PURPOSE: This paper points out that it is not sufficient only to analyze the human face, but it is also necessary to know the context. This allows for a more accurate classification of emotions, and thus a more appropriate match between the robot’s behavior and the social situation in which it finds itself.

DESIGN/METHODOLOGY/APPROACH: Proper situation assessment through a social robot is a fundamental and necessary skill at this point. In order for such an evaluation to be correct, it is necessary to distinguish certain criteria whose fulfillment can be responsible for the robot’s better understanding of human intentions. One such criterion is the identification of the interlocutor’s emotions. For the analysis, Emotic image database has been used, whose unique character allows to identify 26 emotions, understood as discrete categories. This database is constructed in such a way that it allows to detect emotions from both the face or posture of a person, as well as from the context that occurs in the picture.

FINDINGS: The models chosen to solve the problem are Faster R-CNN and YOLO3 networks. In this paper a two-stage analysis is presented. Originally with no changes in the network structure along with the measurement efficiency. And then, as a next step, modifications to the aforementioned neural networks were proposed by introducing the possibility of an internal classifier that allowed for more satisfactory results.

PRACTICAL IMPLICATIONS: The analyzed solutions allow implementation in social robots due to the speed of operation, but show some hardware requirements. Nevertheless, they are an important support for social robots in social situations and have a chance to be the next step to their dissemination in everyday life.

ORIGINALITY/VALUE: Emotion detection and recognition is an essential part of the humanrobot relationship. Proper recognition increases the acceptance of robots by humans.