Real-time Sign Language Recognition Using Computer Vision and AI

Gabriel Serrano, Daehan Kwak

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Promoting inclusive communication is essential and sign language plays a crucial role in achieving this goal. In this research, a system is developed capable of making it easier for those who primarily communicate through sign language to interact with those who are unable or may not be knowledgeable in the language. This has the potential to help bridge the communication gap between those who are fluent in sign language, and those who may be struggling to learn or are not knowledgeable. In its current state, our system is capable of recognizing two forms of sign language, namely: American Sign Language and British Sign Language. The system also is capable of performing facial expression analysis to account for non-verbal inflections expressed by the user. These tasks are accomplished by making use of computer vision provided by the OpenCV Python library. It also uses various machine learning models and the MediaPipe library. We explore two approaches for sign language recognition: contour-based recognition and landmark-based recognition. Additionally, facial landmarks for facial expression analysis are investigated which can be used to detect expressions and inflections from a user's face alone. The next steps of this research will consist of working with more complex words and phrases and investigating gesture recognition.

Original languageEnglish
Title of host publicationProceedings - 2023 International Conference on Computational Science and Computational Intelligence, CSCI 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1214-1220
Number of pages7
ISBN (Electronic)9798350361513
DOIs
StatePublished - 2023
Event2023 International Conference on Computational Science and Computational Intelligence, CSCI 2023 - Las Vegas, United States
Duration: 13 Dec 202315 Dec 2023

Publication series

NameProceedings - 2023 International Conference on Computational Science and Computational Intelligence, CSCI 2023

Conference

Conference2023 International Conference on Computational Science and Computational Intelligence, CSCI 2023
Country/TerritoryUnited States
CityLas Vegas
Period13/12/2315/12/23

Keywords

  • Artificial Intelligence (AI)
  • Computer Vision
  • Facial Expression
  • Image Recognition
  • Machine Learning (ML)
  • Sign Language

Fingerprint

Dive into the research topics of 'Real-time Sign Language Recognition Using Computer Vision and AI'. Together they form a unique fingerprint.

Cite this