Annual: 2019

PR028 »
基于Openvino FPGA 的视频图像去雾系统)
📁Machine Learning
👤文君 吴
 (华南师范大学)
📅Oct 15, 2019
Regional Final



👀 3400   💬 1

PR028 » 基于Openvino FPGA 的视频图像去雾系统)

Description

在雾霾等恶劣天气条件下,大气中的微小颗粒会对光线产生大气物理散射作用,导致捕获的室外视频图像出现可见性差和对比度低等问题,这不仅对人类感知产生负面影响,而且对许多计算机视觉任务构成障碍,如视频监控、目标识别、图像分类等。为此,本方案目的是改善视频图像质量和提高计算机视觉系统性能,以及使用友晶公司的OpenVINO Starter Kit加快每帧图像处理的速度,达到视频图像去雾效果。
通过全卷积回归网络图像去雾的深度学习,训练识别目标对象的能力模块,用Model Optimizer把.prototxt和.cafemodel文件转化成.xml 和.bin文件,完成caffe模型到IR模型的转化;调用 OpenVINO™ 推理引擎API 在目标板做推理,运行目标程序Program,并将模块的所有参数输入FPGA中。

Demo Video

  • URL: https://v.youku.com/v_show/id_XNDM5OTcxMDgyOA==.html?spm=a2h3j.8428770.3416059.1

  • Project Proposal

    1. High-level Project Description

    Design Goals: The sharpness of the video image directly affects the accuracy of its information transmission, but it is often affected by the shooting environment, and the video is not clear.In the harsh weather conditions such as smog, the tiny particles in the atmosphere will cause physical scattering of light into the atmosphere, resulting in poor visibility and low contrast of captured outdoor video images.This not only has a negative impact on human perception, but also poses obstacles to many computer vision tasks, such as video surveillance, target recognition, image classification, and so on.When the smog is serious, it even affects people's daily life, leading to traffic accidents. Therefore, there are classes in primary and secondary schools, and motor vehicles are restricted.

      To this end, the purpose of this program is to improve the quality of video images and improve the performance of computer vision systems, and use the OpenVINO Starter Kit of the company to speed up the processing of each frame of image processing, to achieve the video image defogging effect.This will bring great convenience to people's daily life and reduce traffic accidents in smog days.

    Application areas and Target users:This design is used in the field of image video dehazing.For the smog in northern China and the harsh weather, it is difficult to see the sights of tens of meters away, which brings great inconvenience to our daily life, especially to the drivers.The design can effectively defogg, and avoid problems such as overexposure and color distortion after defogging, and retain complete details and have better defogging effect.

      
    The design reason of use Intel FPGA devices: 

    1.Intel FPGAs use well-trained models and take advantage of the Intel FPGA configuration to speed up usage, which greatly reduces development time and speeds time-to-market.

    2.The OpenVINO Toolkit can deploy applications and solutions that emulate human vision with Intel OpenVINO Toolkit. It implements the CNN-based deep learning by using heterogeneous execution acceleration. With the easy-to-use functions library for computer vision, the OpenVINO Toolkit speeds up time-to-market for the products.

       The Altera FPGA OpenVINO Toolkit also can improve performance,reduce power consumption and significantly improve FPGA utilization. Users can achieve double efficiency with half effort and open new design possibilities. The main features are:

    ⚫ Enable CNN-based deep learning inference on the edge;

    ⚫ Support heterogeneous execution across Intel's CV accelerators, using a common API for the CPU, Intel® Integrated Graphics, Intel® Movidius™ Neural Compute Stick, and FPGA;

    ⚫ Speed up time-to-market through an easy-to-use library of CV functions and pre-optimized Kernels;

    ⚫ Include optimized calls for CV standards, including OpenCV*, OpenCL™, and OpenVX*.

    2. Block Diagram

                                         

    Fig.1 system framework

     

     

     

     


     

                                  Fig.2 Full convolution regression network structure
     

    3. Intel FPGA Virtues in Your Project

    The design reason of use Intel FPGA devices: 

    1.Intel FPGAs use well-trained models and take advantage of the Intel FPGA configuration to speed up usage, which greatly reduces development time and speeds time-to-market.

    2.The OpenVINO Toolkit can deploy applications and solutions that emulate human vision with Intel OpenVINO Toolkit. It implements the CNN-based deep learning by using heterogeneous execution acceleration. With the easy-to-use functions library for computer vision, the OpenVINO Toolkit speeds up time-to-market for the products.

     

    performance parameters:

    1. 15fps or more;

    2. Intel FPGA can solve the problem that artificial intelligence processing is not fast enough, and the cost is reduced.

    4. Design Introduction

    Design Goals: The sharpness of the video image directly affects the accuracy of its information transmission, but it is often affected by the shooting environment, and the video is not clear.In the harsh weather conditions such as smog, the tiny particles in the atmosphere will cause physical scattering of light into the atmosphere, resulting in poor visibility and low contrast of captured outdoor video images.This not only has a negative impact on human perception, but also poses obstacles to many computer vision tasks, such as video surveillance, target recognition, image classification, and so on.When the smog is serious, it even affects people's daily life, leading to traffic accidents. Therefore, there are classes in primary and secondary schools, and motor vehicles are restricted.

       To this end, the purpose of this program is to improve the quality of video images and improve the performance of computer vision systems, and use the OpenVINO Starter Kit of the company to speed up the processing of each frame of image processing, to achieve the video image defogging effect.This will bring great convenience to people's daily life and reduce traffic accidents in smog days.

     

     

    Application areas and Target users:This design is used in the field of image video dehazing.For the smog in northern China and the harsh weather, it is difficult to see the sights of tens of meters away, which brings great inconvenience to our daily life, especially to the drivers.The design can effectively defogg, and avoid problems such as overexposure and color distortion after defogging, and retain complete details and have better defogging effect.

     

     

      
    The design reason of use Intel FPGA devices: 

    1.Intel FPGAs use well-trained models and take advantage of the Intel FPGA configuration to speed up usage, which greatly reduces development time and speeds time-to-market.

    2.The OpenVINO Toolkit can deploy applications and solutions that emulate human vision with Intel OpenVINO Toolkit. It implements the CNN-based deep learning by using heterogeneous execution acceleration. With the easy-to-use functions library for computer vision, the OpenVINO Toolkit speeds up time-to-market for the products.

       The Altera FPGA OpenVINO Toolkit also can improve performance,reduce power consumption and significantly improve FPGA utilization. Users can achieve double efficiency with half effort and open new design possibilities. The main features are:

    ⚫ Enable CNN-based deep learning inference on the edge;

    ⚫ Support heterogeneous execution across Intel's CV accelerators, using a common API for the CPU, Intel® Integrated Graphics, Intel® Movidius™ Neural Compute Stick, and FPGA;

    ⚫ Speed up time-to-market through an easy-to-use library of CV functions and pre-optimized Kernels;

    ⚫ Include optimized calls for CV standards, including OpenCV*, OpenCL™, and OpenVX*.

    5. Function Description

      The design function of this design is to process foggy video images based on the full roll machine regression network.Existing defogged image processing often has problems such as overexposure and color distortion, so a defogging algorithm based on full convolution regression network is proposed.

     

      Through the deep learning of the full convolution regression network image defogging, the module for identifying the target object is trained, and the model trained by the algorithm is converted into an IR model.Use Model Optimizer to convert the trained .prototxt and .cafemodel files into .xml and .bin files, and convert the caffe model to the IR model.Call the Intel OpenVINOTM Inference Engine API to infer the target board(Intel OPENVINO Starter Kit Board), run the target program, and input all the parameters of the module into the FPGA to process the wait output.

      It also show that Intel FPGAs use well-trained models and take advantage of the Intel FPGA configuration to speed up usage, which greatly reduces development time and speeds time-to-market.

     

     

      The figure below shows the comparison before and after defogging:

     

     

     

     

    fig1.Undefogging 

     

     

    fig2.Defogging 

    6. Performance Parameters

    The design reason of use Intel FPGA devices: 

    1.Intel FPGAs use well-trained models and take advantage of the Intel FPGA configuration to speed up usage, which greatly reduces development time and speeds time-to-market.

    2.The OpenVINO Toolkit can deploy applications and solutions that emulate human vision with Intel OpenVINO Toolkit. It implements the CNN-based deep learning by using heterogeneous execution acceleration. With the easy-to-use functions library for computer vision, the OpenVINO Toolkit speeds up time-to-market for the products. 

     

    performance parameters:

    1. 15fps or more;

    2. Intel FPGA can solve the problem that artificial intelligence processing is not fast enough, and the cost is reduced.

     

       It can be seen that after using Intel FPGA, the speed of defogging effect is obviously improved a lot, which lays a foundation for real-time dehazing, which is convenient for the daily life of people who are foggy.

        

    7. Design Architecture

    System Design Scheme:

       

     

     

     

    Hardware Design Block Diagram:

     

     

     

    Software Design Block Diagram:

                                         

     

     



    1 Comments

    Zhou Wenyan
    Have you completed the project? Let us know if you face any problem on doing it. It is really a good proposal.
    🕒 Jun 26, 2019 01:43 PM

    Please login to post a comment.