PR034 » 基于DE10-Nano开发板的手语翻译系统
本手语翻译+语音交互系统是利用模块化的方式设计的，以DE10-Nano开发板作为开发平台，通过 flex 柔性传感器和高精度加速度电子倾角仪进行多维度的数据采集整理，并建立8维数据模型。采用基于贝叶斯理论的朴素贝叶斯分类算法，利用最大后验概率思想准确识别手语信息。语音交互系统搭载了科大讯飞的云端语音识别引擎，可以高效的实现双向交流，保证了正常人与聋哑人之间的无障碍交流。
Purpose:At present, there are nearly 200 million deaf people in the world. They are not born to speak or lose their ability to hear and distinguish language due to hearing impairment. They can only communicate with normal people through sign language, but the popularity of sign language is relatively high among normal people. low. In order to solve the problem of communication between deaf and normal people, a wearable wisdom sign language translation system was developed. The main functions include gesture gesture collection, gesture recognition and voice broadcast.
应用：聋人和普通人之间的手语沟通至关重要。 为了加快识别速度，FPGA用于开发智能手语翻译系统。 通过智能手语翻译系统，可以识别和表达聋人的手语。 让不懂手语的人明白他们想要表达什么。
Applications: Sign language communication between deaf and normal people is essential. In order to speed up the recognition, FPGA is used to develop a smart sign language translation system. Through the smart sign language translation system, the sign language of deaf people can be recognized and voiced. people who don't understand mute understand what they want to express.
Target User: The product is currently able to recognize sign language and sound through the speaker, which can be used for communication between deaf and normal people. At present, there is less research on the target language translation system. This product is designed with FPGA, which has great advantages over the single chip microcomputer.
System flow diagram
Wearable Wisdom Sign Language Translation System Work Diagram
(1) Using Bayesian algorithm to achieve sign language recognition. The real-time feedback optimization of the training model is realized by the idea of maximum posterior probability to realize the accurate recognition of sign language.
(2) Better wearability. Concentrating multiple sensors and core modules, communication modules, and functional modules on one glove greatly reduces the size and weight of the gloves, allowing users to have better comfort.
(3) Using FPGA efficient parallel computing unit, the operation speed and efficiency of gesture classification are improved.
The system design adopts DE10-Nano development board as the main control platform to complete the collection of gesture information, intelligent classification of sign language based on naive Bayesian classification algorithm and real-time broadcast of voice. The gloves are made of simple engineering gloves. Each of the five fingers uses a resistive bending sensor to collect the hand posture when making a mute action, and to obtain the posture and orientation of the gesture using a high-precision gyroscope. The naive Bayesian classification algorithm uses the eigenvalue distance to the nearest idea to realize the real-time feedback optimization of the training model to realize the accurate recognition of sign language. At the same time, the efficient parallel computing unit of FPGA is used to improve the operation speed and efficiency of gesture classification.
As shown in the figure, the whole system architecture consists of DE10-Nano development board, FLEX4.5 bending sensor, MPU6050 six-axis sensor and voice broadcast module. After each component is powered on, the system is initialized. After the initialization is completed, the user starts to respond. Action, when the action is made, there will be data acquisition, and then the collected data will be transmitted to the DE10-Nano development board through the serial port, and then the development board will analyze and pair the data multiple times, select the most correct action, and send the corresponding The data is converted into voice playback by the voice broadcast module.
As the control core of the whole system, DE10-Nano is mainly responsible for data collection and processing. The collected data is comprehensively analyzed and calculated. The K-nearest classification algorithm is used to calculate the minimum distance between the gesture data training model and the real-time acquired gesture. The minimum value is used as the classification result to achieve accurate recognition of sign language. At the same time, the efficient parallel computing unit of FPGA is used to improve the operation speed and efficiency of gesture classification.
The data acquisition part consists of a resistive bend sensor and a six-axis high precision gyroscope. The resistive bending degree sensor cooperates with the peripheral circuit to realize the change of the voltage value by using the change of the resistance value when the bending degree is different, thereby realizing the acquisition of the gesture data. The six-axis high-precision gyroscope is responsible for parsing and acquiring the orientation and posture of the gesture.
The voice broadcast module uses the YS-XFSV2 speech synthesis module of Keda Xunfei. The module supports the synthesis of any Chinese text and English text, and can analyze the text, such as common numbers, numbers, time, date, weights and symbols. The text can be correctly identified and processed according to the built-in text matching rules; for the general multi-phonetic words, the reading method can be correctly judged according to its context; and for Chinese and English texts at the same time, the Chinese-English mixed reading can be realized.
1、手势识别率 Gesture recognition rate
The recognition rate of the different gestures is basically the same. The research group analyzes the recognition of the translation system under different gestures. By accelerating the acceleration and angle, the difference of the features of different gestures is enlarged, so that the gesture matching path is better. The distance value is smaller, which reduces the misunderstanding and rejection rate of gestures during the matching process. The success rate of gesture recognition reflects the excellent performance of the gesture translation system. The average recognition rate of the measured gesture is currently 88.1%. This parameter is still in the optimization stage, and the optimized value will be given in the subsequent display.
2、手势识别反应时间 Gesture recognition reaction time
The recognition speed is the response time of the translation system when the user makes a gesture, and the value varies with the instability of the test data. The speed of gesture recognition reflects the overall performance of the game to some extent. Among the factors supporting the speed performance parameters of the gesture translation, the quality of the training template is important. The part of the subject that involves data comparison is processed in the FPGA. Compared with the traditional arm processor, this calculation process is greatly reduced in time. The average recognition time of the gesture is 125ms.