Devnagari Sign Language Recognition Using Machine Learning

Abstract

In the evolving landscape of Industry 4.0, enhancing communication for individuals with hearing and speech impairments is a crucial challenge, particularly in linguistically diverse countries like India. In India alone, over 63 million people rely on sign language for daily communication, yet technological support for regional sign languages such as Devnagari Sign Language (DSL) remains insufficiently explored. The primary objective is to develop a robust, vision-based sign language recognition system capable of identifying both individual DSL alphabets and complex Barakhadi characters. The thesis presents a comprehensive approach encompassing three major systems designed for recognizing both the DSL 47 alphabet and the complete Barakhadi set. Each system leverages different methodologies tailored to specific challenges in real-world recognition tasks. newlineInitially, a deep learning-based image classification model was developed to recognize static hand gestures corresponding to the 47 DSL alphabets. A custom dataset was curated using mobile phone cameras under varied lighting and demographic conditions. Pre-trained convolutional neural networks, including InceptionV3, AlexNet, VGG16, ResNet50, and Customized CNN, were fine-tuned and evaluated. Among these, InceptionV3 yielded the highest accuracy of 94.98%, highlighting its suitability for sign recognition tasks with limited training data. newline newline