Reconhecimento das configurações de mão de libras baseado na análise de discriminante de fisher bidimensional utilizando imagens de profundidade

Abstract

Deaf people communicate with other people using sign language. This communication is limited to people with knowledge in the language that, usually, are other deaf people. The fact is that there are too many people interacting with deaf people in education, health and leisure areas that are not proficient in sign language. Then, the inclusion of deaf people is seriously affected, because they are unable to make themselves understood. This study presents a methodology for automatic gesture recognition which represents hands settings from Brazilian Language of Signs - LIBRAS. The first approach consisted in a constructing of hands settings image database captured by depth camera, Kinect®. The region of interest, hands making gesture, was extracted using the following techniques: K-means and Distance Transformation. The recognition part was divided in two steps: feature extraction and gesture classification. This way, the dimensionality reduction technique was applied, 2D2LDA to obtain a features set, which was submitted to a classifier, k-nearest neighbor. The proposed system is able to segment image and recognize whole 61 settings of Sign Language. The average hit rate achieved was 96.10%. As the capture device is insensitive to light, background and colors of clothes and skin, the developed application adapts without modifications to any other capture environment.