TY - GEN
T1 - Real-time Sign Language Recognition Using Computer Vision and AI
AU - Serrano, Gabriel
AU - Kwak, Daehan
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - Promoting inclusive communication is essential and sign language plays a crucial role in achieving this goal. In this research, a system is developed capable of making it easier for those who primarily communicate through sign language to interact with those who are unable or may not be knowledgeable in the language. This has the potential to help bridge the communication gap between those who are fluent in sign language, and those who may be struggling to learn or are not knowledgeable. In its current state, our system is capable of recognizing two forms of sign language, namely: American Sign Language and British Sign Language. The system also is capable of performing facial expression analysis to account for non-verbal inflections expressed by the user. These tasks are accomplished by making use of computer vision provided by the OpenCV Python library. It also uses various machine learning models and the MediaPipe library. We explore two approaches for sign language recognition: contour-based recognition and landmark-based recognition. Additionally, facial landmarks for facial expression analysis are investigated which can be used to detect expressions and inflections from a user's face alone. The next steps of this research will consist of working with more complex words and phrases and investigating gesture recognition.
AB - Promoting inclusive communication is essential and sign language plays a crucial role in achieving this goal. In this research, a system is developed capable of making it easier for those who primarily communicate through sign language to interact with those who are unable or may not be knowledgeable in the language. This has the potential to help bridge the communication gap between those who are fluent in sign language, and those who may be struggling to learn or are not knowledgeable. In its current state, our system is capable of recognizing two forms of sign language, namely: American Sign Language and British Sign Language. The system also is capable of performing facial expression analysis to account for non-verbal inflections expressed by the user. These tasks are accomplished by making use of computer vision provided by the OpenCV Python library. It also uses various machine learning models and the MediaPipe library. We explore two approaches for sign language recognition: contour-based recognition and landmark-based recognition. Additionally, facial landmarks for facial expression analysis are investigated which can be used to detect expressions and inflections from a user's face alone. The next steps of this research will consist of working with more complex words and phrases and investigating gesture recognition.
KW - Artificial Intelligence (AI)
KW - Computer Vision
KW - Facial Expression
KW - Image Recognition
KW - Machine Learning (ML)
KW - Sign Language
UR - http://www.scopus.com/inward/record.url?scp=85199969536&partnerID=8YFLogxK
U2 - 10.1109/CSCI62032.2023.00198
DO - 10.1109/CSCI62032.2023.00198
M3 - Conference contribution
AN - SCOPUS:85199969536
T3 - Proceedings - 2023 International Conference on Computational Science and Computational Intelligence, CSCI 2023
SP - 1214
EP - 1220
BT - Proceedings - 2023 International Conference on Computational Science and Computational Intelligence, CSCI 2023
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2023 International Conference on Computational Science and Computational Intelligence, CSCI 2023
Y2 - 13 December 2023 through 15 December 2023
ER -