Sign language is used by members of deaf community to communicate .Each hand gestures in the language corresponds to a meaning.
In India there are over 5-million deaf people but there are only 250 certified interpreters which is one Interpreter for every 20000 deaf-people .
It is a practically impossible to balance this ratio between Interpreters and Deaf people ,this is where our project comes into action.
We propose "Word Level Sign language Detector " for Indian Sign language by using INCLUDE Dataset which contains 2-3 second videos with the sign mentioned.
This can prove as a Game-Changer for deaf community people to interact with other people.
This detector device can be used in places like Information service centers in railway stations for deaf people to get interact and communicate with people and can get the required information with less effort.This device increases the inclusivity for the deaf people and makes them feel comfortable in public places.
First we are planning to extract key pose features points (body-positions ) and then we feed these videos in to Neural network architecture to find the spatial differences between the frames, with that we think we can build an model to classify the signs to words.
This Word Level Sign language Detector model is finally deployed in FPGA . A camera is connected in FPGA which is placed infront of the signer which captures the real time video of the sign and predicts the respective class of the sign.

Project Proposal

1. High-level project introduction and performance expectation

Sign language is used by members of deaf community to communicate .Each hand gestures in the language corresponds to a meaning.
In India there are over 5-million deaf people but there are only 250 certified interpreters which is one Interpreter for every 20000 deaf-people .
It is a practically impossible to balance this ratio between Interpreters and Deaf people ,this is where our project comes into action.
We propose "Word Level Sign language Detector " for Indian Sign language by using INCLUDE Dataset which contains 2-3 second videos with the sign mentioned.
This can prove as a Game-Changer for deaf community people to interact with other people.
This detector device can be used in places like Information service centers in railway stations for deaf people to get interact and communicate with people and can get the required information with less effort.This device increases the inclusivity for the deaf people and makes them feel comfortable in public places.
First we are planning to extract key pose features points (body-positions ) and then we feed these videos in to Neural network architecture to find the spatial differences between the frames, with that we think we can build an model to classify the signs to words.
This Word Level Sign language Detector model is finally deployed in FPGA . A camera is connected in FPGA which is placed infront of the signer which captures the real time video of the sign and predicts the respective class of the sign.

We are using Intel FPGAs because it offers good performance metric ,cost and flexiblity. It also optimizes the computation cost for implementing deep learning algorithms like Convolutional Neural Networks.

Intel FPGAs offers good performance at each stage of our vision pipeline.

2. Block Diagram

3. Expected sustainability results, projected resource savings

This pipeline mainly have to realize good working in Real time data, so as to achieve a good performance on real time data OpenVINO acceleration package provided by Intel will be more useful  and also very efficient in real time performance. The model should also achieve good accuracy with good performance metrics. to achieve good performance in a computer vision with lower computation cost without consuming a more memory is a challenging task.The working of a model should also be fast in a real time application.In order to meet these challenges we are using Intel FPGA accelerators like OpenVINO  for optimized performance. With its advantages of low power consumption, high performance and flexibility, Intel FPGA has gradually become an indispensable hardware part of deep learning.