Classificação automática de sinais visuais da Língua Brasileira de Sinais representados por caracterização espaço-temporal

Abstract

The automatic translation of the Brasilian Sign Languagem (LIBRAS) into Portuguese is a very complex problem, due to the peculiarities and characteristics of this sign language. Several researches have already been carried out and important results have been obtained. However, most of the proposed methods recognize only letters and numbers, or a reduced number of words. In addition, due to such limitations, the results of these researches are still insufficient to enable communication with deaf people without the dependency of interpreters, and basic services such as education and health need these professionals to meet the demand for care of the hearing impaired. Another problem faced on trying to envision solutions is the lack of a public database containing a significant number of signals, labeled by experts in this area. Finally, deep learning techniques have been used to solve many computer vision problems, but we have not found any work directly related to the automatic classification of LIBRAS. In light of these observations, this work uses a method based on deep convolutional 3D neural network, extracted spatiotemporal characteristics, strategy of transfer learning and depth data associated with RGB, to perform the classification of the most common LIBRAS signs used in the literacy of deaf people. In addition, another important contribution is the generated labeled database, composed of 510 instances, all representing dynamic signals, given that there is no LIBRAS database available with such an amount of samples.