“Farm at ease” (Technology to farming):
To meet the growing population of the world, some serious reforms are required in the agricultural sector to ensure that our food system is ready to meet the upcoming challenges. This pushes us to shift from traditional/conservative agricultural practices to sustainable practices. Further, these sustainable practices are integrated with technological advancements, the major challenges of the agriculture/farming across the world can be resolved in an efficient and effective manner.
The following are some of the key challenges posed by the agricultural sector:
1. There is a considerable gap between the adoptions of technology in the agricultural sector when compared to the non-agricultural sectors especially in third world countries.
2. Mismanagement in crop planning without the proper analysis of soil.
3. Overutilization/underutilization of fertilizers.
4. Mismanagement in irrigation and thereby the water scarcity/wastage may occur.
5. Delay in identification of weeds/pests.
6. Unable to adopt the best practices of agriculture in the other regions.
Proposed solution:
To address above challenges, we are proposing for a design of a system (IOT based with Mobile/Web application development/Message alert system) which will aid the farmer in the following way:
1. Crop recommendation based on the soil condition, climate and water availability of that region.
2. Automatic irrigation system depending upon the crop requirement.
3. Recommendation of organic means of agriculture practice in place of fertilizers to the most possible extent.
4. Disease detection in crops using image processing.
5. Recommendation/alert system will be made in a simple manner and if possible, voice instructions will be given in regional languages.
Design idea:
The above-proposed idea will be designed initially as a real-time prototype device where FPGA board will be integrated with sensors such as NPK sensor, humidity sensor, temperature sensor, water level monitoring sensor, etc., Further, a communication shall be established between the FPGA board and mobile phone using FPGA virtues, Azure cloud, etc., This prototype device will be tested in a real-time agricultural field and based on the feedback/recommendations of the user (farmers) a robust system may be developed in future.

Demo Video

Project Proposal

1. High-level project introduction and performance expectation

SMART AND SUSTAINABLE AGRICULTURE USING FPGA

Growth in world population, increasing urbanization, and changing consumption habits means an increase in demand for food production and this must be achieved despite the challenges. To meet the food requirements of the growing population of the world, some serious reforms are required in the agricultural sector to ensure that our food system is ready to meet the upcoming challenges. This pushes us to shift from traditional/conservative agricultural practices to sustainable practices.

Further, these sustainable practices are integrated with technological advancements. The major challenges of agriculture/farming across the world can be resolved in an efficient and effective manner. The idea is to enable farmers more reliable monitoring their fields remotely, allowing the operation of several facilities at one time

Farmers don't need to repeat the same crop cultivation always instead, they can choose the crop as per their farm soil nature/condition. Farmers no longer have to apply water, fertilizers, and pesticides excess across their fields. Instead, they can use the minimum quantities required and target very specific areas, or even treat individual plants differently.

DEMO VIDEO OF OUR PROJECT:Demo Video

https://youtu.be/y86IUrFhL6M

DESIGN IDEA:

The above-proposed idea will be designed initially as a real-time prototype device only for paddy crop, where FPGA board will be integrated with sensors such as NPK sensor, pH sensor, EC sensor, Moisture sensor,  humidity sensor, temperature sensor, water level monitoring sensor, camera sensor, etc., Further, a communication shall be established between the FPGA board and mobile phone using FPGA virtues, Azure cloud, etc., This prototype device will be tested in a real-time agricultural field and based on the feedback/recommendations of the user (farmers) a robust system may be developed in future.

Our model workflow goes in 3 stages.

1. DATA READING

This Data reading is a two-step process.

Initial sensor data is used to perform soil testing and crop recommendation, here we collect data from NPK sensor, pH sensor, EC sensor, etc.

 Secondary sensor data is collected from sensors such as from NPK sensor, pH sensor, EC sensor, Moisture sensor, humidity sensor, temperature sensor, water level monitoring sensor, camera module, etc., is used for nutrient management (fertilizer recommendation), automatic irrigation and crop monitoring (including Weed detection). And this data is taken continuously throughout the crop from the field.

Further, all this sensor data is passed to the FPGA board.

2 DATA PROCESSING AND DECISION MAKING

We fix threshold values (for Paddy crop) from the Research data collected, this data is stored and further used to compare with real-time sensor data (Input data) in FPGA DE Nano-10 / Intel Azure Cloud.

After data processing, the system makes decisions in crop recommendation, fertilizer recommendation, automatic irrigation, weed detection, and pests/disease identification (when the farmer uploads a picture of the affected part of paddy).

3 CONTROLLING and OUTPUT DISPLAYING

As per decisions made by our system; it controls the irrigation system automatically. All the commands and results will be displayed in the Mobile/ Web application. Our system will guide and assist farmers through voice commands (Region language).  Farmer can able to monitor crop(plant growth) by receiving crop pictures regularly.

FPGA is used as a large computational data processing platform, and the time-consuming part of the image processing algorithm is transplanted into the FPGA.

FPGA can parallelize the tasks while the size consumption of such a system is less than the consumption of CPU and GPU. Especially for the image processing techniques, FPGA takes less time as compared with a microcontroller because of high-speed memory.

We need instantaneous processing for all the inputs. Also, we require a high computing power device. That can be done with the DE-10 Nano kit. Therefore, FPGA is the best choice for the development of our device.

APPLICATION AREAS:
Our System has a wide range of application areas in the modern agricultural context.
Some of those areas are:

1. Agricultural farmers.
2. Home Gardens.
3. The plant science Research centers.
4. Green House Manufacturing areas.

TARGET USERS:

Our target users are, those who mainly works on agriculture-based work:

1. Farmers
2. Agriculture firms and companies.
3. Home Gardeners.
4. Environmentalists.
5. Plant Science Researchers.
6. Automated Green manufacturers.

2. Block Diagram

3. Expected sustainability results, projected resource savings

Expected Sustainability results

1.Higher crop productivity.

2.Decreased use of water, fertilizer, and pesticides, which in turn keeps food prices down and reduces food crisis.

3.Reduced impact on natural ecosystems and maintains Soil Health.

4.Less runoff of chemicals into rivers and groundwater.

5.Increased crop sustainability.

         Thereby, Agriculture becomes cost-effective, smart, and sustainable.

Projected resource savings

1.Our project will help farmers in performing soil tests and in choosing suitable crops for the soil condition based on Soil test results within an affordable price range.

2.Our system tests every detail of soil parameters, recommends farmers the required amount and type of fertilizer, and also when to supply. Indeed saves money and also keeps the soil fertile.

3.Our system is automated to supply a sufficient amount of water in the proper way without any human involvement, which saves the surplus of water and farmers Time.

4.Our model identifies weeds present in the crop field. Which helps in crop growth.

5.Our model detects types of disease/pests attacked on crops. Which indeed saves at most half of the yield, by taking care of the crop at an earlier stage.

4. Design Introduction

Purpose of Design:

The main purpose of the design is to address the routine issues in conventional farming practice such as lack of awareness in crop selection, inefficient water management, disproportionate use of fertilizers, and delay in identification of weed and disease detection.

To address the above,we developed a prototype with the following features:

1.Providing a comprehensive solution to address the issues of agriculture using      technological advancements of sensors, cloud connectivity, IoT platforms and FPGA platforms.

2.This design includes monitoring of various key parameters in farming such as NPK (Nitrogen - Phosphorus - Potash) values, Ph values, Electrical Conductivity (EC) values, temperature, water level, moisture.

3.This design includes weed detection system also through image processing and Convolution Neural Networks (CNN). 

4.This design includes crop recommendation system based on above parameters.

Application Scope:

1.This design/prototype can be extensively used in agriculture across the world.

2.This can also be used in Crop Research Centres for complete for further customization of the design based on the local area farming practices etc.,

3.This design is suitable in water shortage areas

4.Development of a farming practice/system with less fertilizers

Targeted Users:

1.Farmers

2.Agriculture based Industries

3.Agriculture Universities / Research Centres

4.Soil health monitoring agencies/industries

Reasons for utilization of Intel FPGA Devices in the Design:

1.Reconfigurability of FPGA Devices

2.Pipelined and Parallel architectures helps in execution of CNN algorithms in less execution time

3.First party support for OpenCL directly from Intel

4.Smooth integration with Linux using the DE10 Nano kit. This allowed us to use the traditional SoC for common things while using FPGA for OpenCL code and parallelization.

5.DE10 allows using all the advantages of SoC, combining the productive ARM HPS core and the flexibility of FPGA. HPS core makes it possible to locally implement powerful data processing algorithms and parallelize their execution at the hardware level using FPGAs. HPS core allows easily connecting to WiFi and Ethernet to communicate with an external server. In addition, the use of IP cores permits quickly implementing any additional hardware module on the FPGA logic. FPGA facilitates the implementation of any interfaces connecting different peripherals. Power supply (5V) permits implementing a safe and autonomous system with low power consumption.

6.The DE10-Nano presents a robust hardware design platform built around the Intel System-on-Chip (SoC) FPGA, which combines the latest dual-core Cortex-A9 embedded cores with industry-leading programmable logic for ultimate design flexibility. It also has a sufficient number of outputs for connecting modules

5. Functional description and implementation

The functional description and implementation of the entire project is illustrated with the help of a flow chart as follows:

A. Functional description of the project:

The following are the various functions involved in the project: 

1.Sensor data collection

2.Crop recommendation system

3.Weed Detection

4.Disease Prediction

5.Smart Irrigation

6.Fertilizer recommendation

7.Overall crop monitoring system

Detailed description of above functions:

1.Sensor data collection:

Sensor data collection takes place in two parts.

Part 1: Data collection through serial communication from Arduino to DE-10Nano. All the analog sensors i.e. NPK Sensor, pH Sensor, Water level sensor, Soil Moisture sensor, Electric Conductivity Sensor and Soil Temperature sensor are connected to the Arduino Uno Microcontroller and read the data from them. This Arduino board is connected to DE-10Nano board using USB hub to OTG (USB-II mini) This sensor data is transferred through Serial Communication to FPGA Board. This Data will be sent to Azure Cloud for every 10 minutes and stored in Azure Containers.

Part 2:  Data collection from Temperature and Humidity sensors which are on RFS Daughter card is transferred to Azure IoT HUB as messages this data is being redirected to Azure Blob Storage.

All the data stored in Azure Cloud is being processed and displayed on Website.

Hardware Integration of the project:

Sensors we used in our project:

Implementation:

     1. Data collection through sensors:

Data collection through sensors is done with microcontroller (Arduino) and RFS Daughter Card. We collected the Analog sensor data from NPK Sensor, pH Sensor, Water level Sensor, Soil Moisture Sensor, Electric Conductivity Sensor and Soil temperature Sensor through Serial Communication to DE-10 Nano. We collected Temperature and Humidity data from Humidity sensor and Temperature Sensor presented in the RFS Daughter Card.

    2 .Integration of sensors data with DE-10 Nano:

Temperature sensor and Humidity Sensors present in RFS Daughter Card are connected to DE-10 Nan0 ( 2X20 GPIO 0 (JP1) pins).

NPK Sensor is connected to Arduino through  MAX 485 module, PH sensor and EC sensor is connected to arduino through BNC Probes. Likewise water level , soil moisture and soil temperature is connected to ArduinoUno; is connected to DE10-Nano board using USB Hub.

2.Crop recommendation system:

For Crop recommendation the sensor values from NPK, EC, and PH are taken as inputs. These parameters are observed previously and then it is stored in the database. The received data sets of soil and climatic conditions have applied to the machine learning algorithm. The support vector machine and linear regression are sensitive learning models. We have modelled four different learning algorithms, such as linear regression, decision tree, k-nearest neighbors, and XGBoost. In this study, we have chosen XGBoost for its better performance. We trained a Tree based ML model (XGBoost) to predict the recommended crop using features like NPK, PH, EC. The dataset was collected from Kaggle(Publicly available dataset).The algorithm checks the original data of the soil (DSoil) with the test data of the soil (DTest) and estimates the crop suitable for that area. a. Moreover the algorithm checks the humidity, temperature, and water level present in the field. The agricultural field is monitored by picturing the field once in a day using a digital camera. Achieved 95% accuracy on test data.

 Implementation of crop recommendation:

Crop Sample data:

Confusion Matrix:

Crop results:

Our website recommending crop to the user :

Note: As it is not a season  to cultivate paddy crop we have cultivated corn for the sake of demonstration.

3.Weed Detection:

1,For weed detection, we have collected our own custom paddy weed detection dataset from our village  using a mobile.

2.The dataset contains 300+ crop images with weed plants.

3.We manually labeled the dataset and trained a state-of-the-art object detection model (YOLOv5) to detect the weeds in a given image.

4.Developed a server in python Flask to host the model and deployed it using DOCKER and azure app service.

 Implementation of weed detection:

Prediction of Validation Data:

F1 Curve:

Results:

Our website showing the detection of weed present in the field:

4.Disease Prediction:

1.Disease prediction is done when a farmer uploads the affected part of a plant. we Collected diseased images of several crops like rice, tomato, potato etc.

2.Each crop contains more than 1K images with different diseases.

3.We trained a convolutional neural network model to predict the given infected plant image and recommend respective fertilizers.

4.Achieved 85% accuracy on all disease prediction models.  

Implementation of disease prediction:

Training model for rice crop prediction:

Training loss:

Training Accuracy:

Rice Confusion Matrix:

Disease prediction page to upload the image:

Chemical control , Biological control and Cultural control suggestion for tested disease:

5.Smart Irrigation:

An automatic irrigation system is proposed based on the water level of the farm and the moisture available in the soil. Our model performs irrigation based on water requirements in stages wise. Soil moisture acts as an essential role in agriculture. Water level sensor is used as key sensor in smart irrigation. Level concept is useful in rainy and high temperature conditions. We modelled a conditioned based algorithm to turn the motor on and off. 

Implementation of smart irrigation system:

* Based on the sensor data from Moisture sensor and Water level sensor.  We supply       water to the crop in stage wise (4 stages).

* Since from the first day of the cropping we start counting days and according to the crop stages we supply water in an appropriate manner.

*We supply water into the field through INLET motor based on the stage and level of the water required for the crop.

* And we also remove excess water which comes into the field in abnormal situations for the crop like heavy rains and etc.… which help in better yield production.

All the sensor data and pictures from camera is been sent to the cloud, performs computation there and displayed over Web Application.

Case 1: INLET  and OUTLET motors remains in OFF, when there is a adequate amount of water. 

Case_2:  INLET motor ON when there is a requirement of water.

Case_3: OUTLET motor ON when there is a excess water in feild.

6.Fertilizer recommendation:

Fertilizer recommendation is done based on the type of soil, kind of plant, and the result of the soil test. The data regarding the fertilizer is being displayed to farmers directly in a web page along with the fertilizer brand and description about that. We have developed an application for the selection of suitable crops and fertilizers appropriate for the field. The application shows all the data related to the Name of the crop, duration, fertilizer is seen through the current crop, and the fertilizer for the corresponding plant will display for the particular time of agriculture. Based on the received data from the field, the controller decides the type and amount of fertilizers suitable for the crop.

 Implementation of fertilizer recommendation system:

i). we collected required nutrient’s data for the crop in stages wise.

ii). Based on the field NPK values we compared the sensor data with the researched      data.

iii). whenever a change is noticed in the sensor NPK values, our system will recommend required Macro nutrients to the farmer by comparing with collected data based on the threshold values of the crop in that particular stage.

iv). we also display respective nutrients brands to the farmer

Fertiliser recommendation with their brands in Website when there is a defeciency in nutrinets:

7.Azure cloud/Website functionality:

* We collected input from various sensors using DE_10 NANO and for serial communication we have used Arduino.

* The collected data is then redirected to AZURE IOT HUB as messages using AZURE IOT client for the inference.

* The collected data from FPGA like sensor data, RFS data and daily crop image is sent to AZURE STORAGE ACCOUNT.

* We have developed a custom YOLOV5 model on AZURE APP SERVICE which receives a crop image from FPGA and sends the image with bounding boxes if any weed exists.

* The data stored in AZURE STORAGE ACCOUNT is simultaneously redirected to the web server to display the data on website.

* We have displayed XG-BOOST model for crop recommendation and CNN model for Disease Prediction as APIs using AZURE CONTAINERS in APP SERVICE.

* Farmer will interact with these APIs through website and the expected results like recommended crop and disease cure control are displayed to farmer in the website.

Azure Cloud/Website implementation:

We have used azure storage accounts to store the rfs and sensor data as blobs. we are sending the data from FPGA board with help of python sdk and azure-blob-storage package.

We have used the Azure container registries to store the docker containers of model APIs and with the help of Azure App services we've deployed our APIs as web apps.

We also integrated the IOT Hub with the de10 nano to see messages for the inference.

Overall crop monitoring system:

We come up with solution for the problems as mentioned earlier. We divided our system into 3 parts.

a). Hardware, b). Cloud and c) Web Application

* We recommended best yield obtained crop based on NPK, pH, Temperature, Humidity, EC sensor values using XG Boost Algorithm.

* We collected crop pictures for every 3 hours for Crop Monitoring.

* We performed Weed Detection to detect weeds present in the field using YOLO v5 Algorithm.

* We also recommended fertilizers whenever there is a nutrients deficiency for the Crop. According to the crop stage we also recommend fertilizers (Micro and Macro nutrients).

*We implemented Smart Irrigation system in our field in stages wise (4 stages), here we use 2 motors for INLET (for irrigation from ground water into the field) and OUTLET (to removes excess water from the field in abnormal situations for the crop like heavy rains…).

*Here the user can perform disease prediction by capturing the disease effected part of the plant by uploading the picture through Web Application. Here we used CNN Algorithm.

6. Performance metrics, performance to expectation

7. Sustainability results, resource savings achieved

Sustainable Results:

 We desinged our system only for paddy and some other crops in performing disease prediction. 

We taken a 5.8 sq.m of land and planted Corn (since its not the time for Paddy) and planted corn crop, we have take data for 15 days (14- march-2022 to 3- April-2022) based on the data there is a need of 2 irrigation (first 9 days for Emergency VE and next 5 days and further for 4 leaves V4 stage  maintaining  of 0.8inch and 1 inch of water level respectively) and note down our results.

System Design Scheme:

8. Conclusion

We have successfully done soil analysis of samples at different locations. This proposed framework is useful for farmers for getting real time soil parameters at fingertips, without waiting for soil testing lab results .  Field is being monitored by the farmer at regular intervals of time. Weed is identified without a farmer in the field.  A successful disease detection model is being built where a farmer uploads a pic (disease affected part) is being processed. And the main task in agriculture, irrigation now becomes automatic and successfully a crop gets sufficient water through automatic irrigation at intervals of time. Farmer can able to cultivate his crop with greater yield.

Refernces:

1. Smart Irrigation based on Crops using IoT (IEEE Xplore): https://doi.org/10.1109/ICIIS51140.2020.9342736

2. Tamil Nadu state Agriculture Dept:-  https://agritech.tnau.ac.in/agriculture/agri_nutrientmgt_rice.html

3. Science Direct:- https://doi.org/10.1016/j.cropro.2018.01.007

Resources:

1.https://www.innovatefpga.com/portal/support.html

2.https://github.com/intel-iot-devkit/terasic-de10-nano-kit

3.https://rocketboards.org/

4. Data sets: www.kaggle.com