Real-time Sign Language Interpretation Using Deep Learning

• Author(s):

  Dr. U. Nilabar Nisha | Livin Prajith VL | Anbuselvan A | Naveen S | Pandiyaraja P

• Keywords:

  Sign Language Recognition, Convolutional Neural Network (CNN), MediaPipe, Deep Learning, Hand Gesture Classification, Text-to-Animation, Computer Vision, Assistive Technology.

• Abstract:

  Sign Language Serves As The Primary Mode Of Communication For Millions Of Hearing-impaired Individuals Worldwide, Yet The Gap Between The Deaf Community And The Hearing World Remains A Significant Barrier. This Paper Presents A Comprehensive, Three-subsystem Deep Learning Framework For Real-Time Sign Language Interpretation, Encompassing Sign-to-Text Conversion, Text-to-Animation Synthesis, And Text-to-Image Generation. The Sign-to-Text Module Leverages A Convolutional Neural Network (CNN) Trained On MediaPipe-extracted Hand Landmark Vectors, Deployed Via A Flask Backend With OpenCV-based Real-time Video Capture, Achieving A Classification Accuracy Of 97.6% Across 26 American Sign Language (ASL) Gestures. The Text-to-Animation Subsystem Utilizes Angular 21 And Ionic 8 On The Frontend With Node.js/Express And Firebase Cloud Infrastructure, Employing MediaPipe Holistic For Body-pose-driven Skeletal Animation Rendering. The Text-to-Image Subsystem Is A React 19 Single-page Application Powered By A Generative Deep Learning Model Pipeline, Converting Descriptive Text Into Contextual Sign Language Visual Representations. Extensive Experiments Demonstrate Strong Cross-subsystem Performance, With MSE Of 0.043, RMSE Of 0.207, Precision Of 96.8%, And Recall Of 97.1% On The Test Set. The Proposed Integrated Framework Significantly Advances Assistive Communication Technology And Establishes A Replicable Architecture For Real-world Sign Language Translation Deployment.

---

Other Details

• Paper id:

  IJSARTV12I4104954

• Published in:

  Volume: 12 Issue: 4 April 2026

• Publication Date:

  2026-04-09

Download Article