Uma estratégia para reconhecimento de sinais de Língua Brasileira de Sinais utilizando aprendizado profundo

Abstract

Sign languages are natural and living languages used for non-verbal communication between deaf, hearing impaired and hearing people. In Brazil, the sign language known as Libras is legally recognized as the language of expression and communication for this group of the population, composed of approximately 9 million people. To facilitate communication between deaf and hearing people, some assistive technologies for transcription of Libras to Portuguese have been developed, especially using vision-based techniques. In this scenario, convolutional neural networks are widely used due to achieving results considered state of the art in the area of gesture recognition. However, an effective solution to this problem has not yet been found, mainly due to: the high financial cost of implementation, such as the use of specific data acquisition devices; the intrusive aspect, since some solutions use portable sensors, such as gloves equipped with motion sensors; and technical limitations, because, despite the majority of Libras signs being executed with motion, the Libras signs recognition area is especially dominated by solutions that consider only static signs. Besides, few studies explore the impact of using techniques such as transfer learning, data augmentation and fusion different data channels. Thus, the development of this work is based on the need for a method to perform signs Libras recognition, which low cost, in a non-intrusive manner, and efficient in recognizing signs that are executed in motion. To accomplish this, this work aims to present a strategy for the recognition of static and dynamic Libras signs employing a three-dimensional convolutional neural network, fusion data from multiple channels and transfer learning.