American sign language and facial expression recognition using YOLO11 object detection model.

Abstract

This project addresses the critical need for effective communication solutions for the Deaf and hard-of-hearing community by focusing on the recognition of American Sign Language (ASL) gestures and facial expressions. Utilizing advanced deep learning techniques, specifically the YOLOv10 and YOLO11 object detection models, the study aims to develop a real-time system capable of accurately interpreting ASL signs and the associated facial cues. A custom dataset was created, consisting of high-resolution images that capture various ASL gestures along with corresponding facial expressions. These images were carefully manually annotated and preprocessed to ensure consistency and enhance model performance through data augmentation techniques. The dataset was then divided into training, testing, and validation sets for thorough model training and evaluation. The YOLOv10 and YOLO11 models were rigorously tested, demonstrating high precision and recall rates in ASL gesture recognition. Comparative analysis highlighted the advantages of each model, particularly in terms of their accuracy and computational efficiency. By offering a scalable and effective solution, this study significantly contributes to the fields of computer vision and communication accessibility, with the potential to enhance interactions between hearing individuals and the Deaf community. The outcomes of this research underscore the importance of technology in promoting inclusivity and improving communication for the Deaf and hard of hearing.