EM004 » Systolic fuzzy logic system for border surveillance using FPGA
Many regions, all over the world nowadays, are suffering severely from several contemporary issues that without a doubt are hindering the overall world progressions toward reaching a more prosperous economic status. Moreover, most of these issues are related directly to the failure of monitoring the borders thoroughly.
The main goal of this project is to propose an appropriate Artificial Intelligence (AI) based solution that could substantially increase the efficiency of border monitoring by creating a wireless network using ultra-low power nodes that could capable to detect any suspicious intruding behaviors. The design of the proposed project depends on three main stages: The sensory data fusion interface; The biological inspired Fuzzy Logic (FL) system controller; and the Wireless Sensor Network (WSN) interface stage.
Regardless to the massive and dynamic modernized our life became so far, no one could deny how the world become unstable and unsafe place to live in due to many severe and direct threats that mainly comes from terrorism. According to the prestigious Global Terrorism Database, we can clearly say that not a single vibrant city all over the world was save from this massive threat. Regarding to the same records, there are more than 180,000 terrorism accidents just from 1970 through 2017 that caused several thousands of innocent people to be killed, many historical buildings had been demolished, and many other types of mayhem that could easily being distinguished in our daily life. One of the main resources of such disease is the lack of security and monitoring of the border of a victimized country regardless to the different reasons and causes. I believe that science is the only key player which could eliminate such threat through providing an efficient and reliable solutions with an affordable cost.
The main goal of this project is to create a complete prototype of a robust Wireless Sensor Network (WSN) system that could help in detecting any illegal trespassing on a specific border or a protected area relying on variety of sensor types and improved Artificial Intelligence (AI) algorithm. The proposed design could be considered as a prototype of proving the concept and it has a lot of enhanced capabilities that it will be included in next stages.
Overall, the proposed project is depending on two sections. The first section represents the remote WSN node, which is equipped with the required sensors, Intel Terasic DE0 Nano FPGA board, and 2.4 GHz Zigbee wireless module. The main goal of this node is to perform the high-speed data acquisition processing for each of the integrated sensors using a 12-bit resolution Analog to Digital (ADC) unit, then transmitting the aggregating info. from these sensors to the main iHUB node for further processing and detection stages. The predefined transmitted data package from the remote data node has the essential info. which is required to be distinguished from other nodes.
The second section represents the iHUB node in which it is responsible for differentiating the received wireless data packages from the remote nodes and to apply an AI-based algorithm to detect the level of threat in each node using the OpenVINO Starter Kit.
Up-to-date pictures from the iHub monitoring display demonstration:
Figure: The remote node system diagram
Figure: The iHUB node block diagram
There will be many features that have been gained from using FPGA as the hardware platform for this project as:
Parallel computation ability: The proposed project is actually consisting of multiple of optimized systolic-based Fuzzy Logic units, in which all of them are operating in a parallel manner. Subsequently, the overall computational processing speed will be boosted in comparing with the similar solutions.
In general, any Fuzzy Logic System (FLS) is consists of five main functions. In this proposed project, only the first three functions had been designed in a complete- parallel manner. However, the last two functions, in each one of the proposed fuzzy cores, have been done using sequential design procedures. Which means that even the obtained high computational power could be enhanced in the soon future.
Figure: Fuzzy inference diagram
Low power consumption: The using of the VHSIC Hardware Description Language (VHDL) for describing this project is for its ability to achieve the maximum optimization levels to the targeted design and for balancing between the parallelism capability and the power dissipation of the code architecture.
Reconfigurability: The proposed iHub node could be reconfigure the operation manner of the remote nodes depending on the level of the threat that has been encounter (Future feature that easily could be integrated to the proposed system)
Scalability: It is an extremely simple process for the proposed iHub node to adopt new remote nodes to be integrated to its architecture if needed.
This project could be considered as an extension to the below two publications:  and 
Wireless sensor network (WSN) consist of large number of sensor nodes deployed densely in the environment. The deployed sensor nodes collect data and disseminate the collected data towards sink, this concept can be used for many crucial applications where manual involvement of humans is difficult.
One of the hottest applications of WSN nowadays is in Intrusion Detection System (IDS) for border monitoring to protect sensitive facilities or international borders. One of the applications of IDS is to observe any moving or stopped vehicles in a specific protected area, which may be used as a bombing vehicles or terrorist acts. Deploying such sensor nodes require the use of appropriate and suitable control system and sensors as well. Therefore, a suitable control technique is needed to trigger an invasion alarm level events in a way that sensor nodes can understand them. Most previous work on event detection in WSNs uses precise, also called crisp, values to specify the parameters that characterize an event. However, sensor readings are not always precise. In addition, different sensors, even if located close to each other, often vary in the values they registered, therefore wrong decision will be taken which can be classified as a false alarm. The situation becomes even more convoluted because more than two sensor measurements are involved often to raise the precision of the event. This makes determining the precise event thresholds an extremely hard task and can be clearly figured out that using crisp values to describe WSN events is not the most suitable approach. An Artificial Intelligence based Fuzzy logic System (FLS), on the other hand, might be able to better address the problems that are challenging for crisp logic.
The design of the hardware platform for an AI- based system for a border monitoring should guarantee behavior of these biologically-inspired algorithms and their high degree of parallelism when processing the input data. Therefore, depending on hardware platforms that permit such capability like the Field Programable Gate Array (FPGA) or the Application Specific Integrated Circuits (ASIC) should be the optimum solution especially due to their lower power dissipation capability that consider as an extremely important feature for such remote systems shat is often being deployed in a remote locations in which the continuity of supplying electrical power is seldom. FPGA-based hardware platform is preferred to be used over the ASIC-based solutions due to its unique reconfigurability feature.
The importance of using FPGA hardware platform for such applications is also obvious from a different perspective like the logic utilization efficiency, which will be extremely clear after introducing the proposed design and explaining its effect of applying some degree of optimization in each subblock in the architecture according to the exact need in the design. For instant, the blocks and subblocks in the architecture should not share the same fixed input and output bus types and widths if that is not mandatory. Subsequently, we could have a more efficient logic utilization in comparing with the CPU and GPU hardware platforms. Also, one of the powerful feature in the FPGA designs is that ability to even have an asymmetric input\ output bus widths in the structure of the subblocks since as an example, we have some of the blocks processing the input data with 14-bits while the others are using 12-bits,8-bits, and 7 bits bus widths.
The design of this project went through five stages as explained below:
Stage 1: Sensors selection
In which a survey has been made to select the proper and accurate sensors that will meet the requirements of the proposed project. After a few arguments, the selection went to equip each remote node by two SJ-PM-TFmini lidar sensors from Benewake company, two passive infrared sensors, and one acoustic sensor. Many perspectives have been taking into considerations for this selection such as the ease of use and interface, the accuracy and performance, the overall cost of the node, and the range of detection coverage.
Stage 2: Network package design
In this stage, package of nine bytes has been selected to transfer the sensory data from the remote node to main HUB station. The code section that has been designed to form the data in such package structure is completely configurable and adjustable for adding more bytes for any future upgrades. For instance, this project is considered as a proof-of-concept one since just one remote node is to be used and hence there is no need for example to add a node identification header in this peer-to-peer model. However, the expansion of either the number remote nodes or the number of integrated sensors in each node will affect the size of the transferred data package. Below is a brief description of the need of each byte in the designed data package.
Data frame structure description:
Generic byte structure description:
Stage 3: Fuzzy Logic design and stability testing
In order to design and implement the fuzzy logic processing cores as explained in figure 4 on the FPGA platform, we predesigned three different fuzzy logic models using MATLAB in order to understand and visualize the expected behavior of these control models and to measure to which extent they could fit the expected functionality in the proposed project. The first fuzzy logic model has been designed to visualize the predicted behavior of the fuzzy logic processing cores that responsible to be interfaced with the Benewake lidar sensors. While the second fuzzy logic model was for the acoustic sensor, and the last one was for mimic the operation of the fourth fuzzy logic processing core.
Stage 4: Digital design
The digital circuit design of the whole project has been done using VHDL without using any external IPs. Also, even the design itself is not depending on any embedded subblocks on the FPGA silicon fabric architecture like the DSPs or the embedded memory blocks. In the first stage, the logic design for each of the sensors has been finalized (i.e. the Benewake SJ-PM-TFmini, PIR sensor, and the Acoustic sensor. Then, the wireless interface logic blocks for both the transmitting node and the receiving node. The third logic design stage was related to build the corresponding codes for both the SK6812 24-pixel Ring RGB LED, and the SK6812 1-pixel RGB led.
The second last stage was about building the stages of each of the Fuzzy logic systems. This stage could be considered as the most demanding stage for the following reasons:
The last stage of the logic design, was related to the description for the rest of the blocks and how to write the top-level codes for both the Remote node and the iHub node.
Stage 5: Testing and validation
The most time-consuming process in the whole project is definitely the testing and validation stage. Many design issues and required modification process have been introduced while testing\ simulating whether a subblock nor a block in the system. The validation stage has been passed successfully using the help of ModelSim simulation software, practical hardware measurement using Oscilloscope, or the Signal Tap Logic Analyzer tool in Quartus Prime software.
For a clear and a wide-angle judgment of the proposed designs and their computational performance, an in-depth analysis has been introduced in this section. The analysis includes the three main aspects including area, power, and timing. The analyses, in this section, could be considered as the anchor point for further research-based studies and comparisons.
This section is divided into three subsections:
1) Remote node and iHub node analysis:
According to what has been explained before, the proposed system is mainly depends on two separate systems that are wirelessly cooperate together. The first system is the Remote node which is responsible for aggregating the data from the connected sensors and prepare these data in a predefined package to be sent to the iHub node for further processing operations. The Remote node is thus could be considered as a passive element since there is no any decisions could be taken by this remote node.
In this project, the implementation of the Remote node has been accomplished using the Intel Altera De0 nano board. As shown in TABLE 1, the remote node is only consumed about 2% of the total available logic resources in the EP4CE22F17C6 device. Furthermore, the Remote node could operate with a maximum operating frequency of about 481.93 MHz under the Fast 1200mV 0C Model. Also, the maximum Core Dynamic Thermal Power Dissipation is equal to only 9.10 mW under the 23 mm heatsink with 400 LFpM airflow cooling solution assumption.
Table 1: Design Summary Report for the Remote Node Architecture using Quartus Prime Tool
On the other hand, the implementation of the iHub node has been accomplished using the Intel Altera OpenVino board. The digital design of the iHub node includes:
In order to understand and analysis the iHub system, we compared the power, logic area consumption, and the timing performance of it as well as the main subblocks that it has to have a clear vision the expected futuristic performance for adding more fuzzy logic cores.
The overall logic utilization ratio for the iHub system is about 2% as elaborated in Table II. While the maximum operating frequency could reach be up to 300.12 MHz. Also, the maximum dynamic core power consumption is only 28.77 mW.
In order to have a better understanding of the time, power, and area distribution within the iHub node, a further analysis has been made for the fuzzy logic system of the Benewake lidar, beside the fuzzy logic system of the acoustic sensor as illustrated in Table III, and Table IV. In which we could easily realize that the Fuzzy logic-based sub cores are the main dominant components in the iHub design.
Table II DESIGN SUMMARY REPORT FOR The iHub ARCHITECTURE USING QUARTUS PRIME TOOL
Table III DESIGN SUMMARY REPORT FOR The Lidar Fuzzy Core ARCHITECTURE USING QUARTUS PRIME TOOL
Table IV DESIGN SUMMARY REPORT FOR The Acoustic Fuzzy Logic ARCHITECTURE USING QUARTUS PRIME TOOL
2) Remote node vs node comparison:
In the below two figures, we can observe that the Remote node has a faster operating frequency of about 482 MHz in comparing with only 300 MHz for the iHub node, while the digital logic area is about 1789 ALMs in the iHub node in comparing with only 518 LEs for the Remote node system. Regarding to the power, we can observe the minimum overall core power consumption in all the block and subblocks of the proposed design due to the high-level of optimization coding techniques that have been adopted in the design process. As it was expected, the iHub node is drained much more power than the Remote node due to the intensive computational parallel operations that it has due to the AI fuzzy logic cores. Also, we could observe that almost 99.79% of the consumed power in the iHub node is caused from the three FLS engines that it has.
Also, the below Figure illustrate one of the most important features in this project proposal and the main reason of depending on a systolic architecture of the design of the FLS. Furthermore, the systolic design in our architecture added a massive degree of parallelism through the division of the main units into micro elements. Thus, we could find that the overall number of parallel operations per seconds (GOPS) that the Acoustic FL core and the Lidar FL core could achieved are equal to 5.5 TOPS and 6.4 TOPS respectively.
3) Proposed iHub node vs other contributions:
In this part a brief comparison has been obtained between the proposed iHub system and other similar contributions , , and . In Table V, we can observe that the proposed FLS engine in this project has three cores (one for the acoustic sensor, and two for the right and left Benewake lidar sensors) in comparing with the others contributions that have either two or one FLS cores. However, we can clearly observe the efficient power consumption rate from the proposed FLS engine in this project, which is about 28.77 mW, in comparing to the other contributions which presents more that triple the mentioned amount. Also, the table showed that despite of having variable bus widths for processing the fuzzy input crisp values in the proposed FLS engine, it achieved a noticeable boost in the overall computational speed by roughly 300MHz, while the other contributions showed only 192.1 MHz and 75.1 MHz only. The promising results that obtained from the proposed FLS engine for the iHub node is mainly achieved due to the optimization strategies that have been adopted. For instance, the design of the FLS cores in the iHub engine depends on asymmetric bus widths for each subblock that it has depending on the required bits which required to perform its specific function within each FLS core. Also, we observe from the table that the designed FLS cores are not depending on any embedded DSP blocks for performing any required dot-product operations. Also, the high-speed performance of the proposed FLS engine can be related the parallel-fashion of firing the output data for the majority of the subblocks in each of the FLS cores.
Table V DESIGN SUMMARY REPORT FOR The Acoustic Fuzzy Logic ARCHITECTURE USING QUARTUS PRIME TOOL
Figure: The iHUB node system diagram
Figure: The hardware elements of the iHUB node
Figure: The remote node system diagram
Figure: The hardware elements of the remote node without enclouser
Figure: The hardware elements of the remote node with enclouser
Finding an optimized AI-based solution for solving one of the problems for the border surveillance has attracted the attentions recently. Many aspects have been taken into consideration in this proposed solution like: the overall high-speed processing of the data, relatively low power consumption rate, the majority of the subblocks operates in a concurrent manner to close the gap with the biologically inspired similar models. The achieved maximum frequency of the iHub system is about 300 MHz and the total dynamic core power consumption nearly 28.77 mW. Also, the proposed design is showing a magnificent adaptability performance for either integrating more sensors or integrating more FLS cores upon the need.
The proposed system has a wide range of aspects that could be improved. One of these aspects is to adapt the FLS parameters for allowing the system to be trained for increasing its efficiency of detection. Also, many other important features could be included as the jamming detection principle, which already has been introduced in the proposed design but not in more details. Finally, the proposed remote node could be implemented using the ASIC flow to verify the enhancement in the power consumption in comparing with the FPGA hardware platform. Finally, we can observe that the system is adopting simplex transmission mode for transferring the data from the Remote node to the iHub node. Such transmission mode could be accepted in simple systems. However, the need for more sophisticated duplex transmission mode will be essential as the number of the integrated sensors and\or the number of the Remote nodes increased. Moreover, The number of data bytes in each packages beside the number of different packages will be increased as well. Also
Many thanks to the Benewake company for their technical support to make this self-funded hobby project reach this point and their endless trust.
 Ahmed, H.O., et al., Design and Implementation of Fuzzy Event Detection Algorithm for Border Monitoring on FPGA. International Journal of Fuzzy Systems, Springer, 2014.
 Ahmed, H.O., et al., Intelligent Fuzzy Event Detection for Border Monitoring in Noisy Environment, in ICEENG-9. 2014: Cairo, Egypt.
 KHELlFI, F., et al., Fuzzy Logic-Based Hardware architecture for Event Detection in Wireless Sensor Networks, in Computer Applications & Research (WSCAR), 2014 World Symposium on. Jan. 2014, IEEE: Sousse. p. 1 - 4.
 Al-Aubidy, K.M. FPGA Implementation of Fuzzy Inference System for Embedded Applications. The Dean, Faculty of Engineering, Philadelphia University 2010.