Tradução automática de LIBRAS para triagem hospitalar: uma abordagem com visão computacional e aprendizado profundo de máquina

Abstract

This monograph presents the development and validation of an automatic translation system from Brazilian Sign Language (LIBRAS) to Portuguese, specifically focused on the context of hospital medical triage. The communication barrier between deaf patients and hearing healthcare professionals compromises the quality of care and patient autonomy. To mitigate this issue, a computer vision architecture based on Deep Learning is propo sed to interpret dynamic signs in real-time using conventional cameras. The methodology employs the MediaPipe Holistic library for the extraction of feature vectors based on ske letal coordinates of the pose, face, and hands, eliminating the dependence on raw image processing and ensuring computational efficiency. The temporal classification of signs is performed by a Gated Recurrent Unit (GRU) Neural Network, selected for its superiority over LSTM in preliminary tests. To validate the robustness and generalization capability of the system, a unique dataset was created with the participation of 30 volunteers, in cluding deaf and hard-of-hearing individuals. The evaluation used the rigorous statistical protocol Leave-One-Subject-Out Cross-Validation (LOSO-CV), simulating a real-world scenario where the system operates with unknown users. The obtained results demons trate a global average accuracy of 94% and an average F1-Score of 0.94. The system demonstrated practical viability for deployment in hospitals using accessible and easily available hardware, contributing to the democratization of assistive technology.