AP034 » DIGITAL FARMING USING MACHINE LEARNING
Linking technology with agriculture is one of the key areas that is to be given at most concern.
Agriculture is the main stem of Indian economy. But the crops are facing potential problems namely pests, fungal diseases, water scarcity and thereby causing a huge loss to the crops. That’s why farmers are monitoring the crops day-by-day.
It is observed that, in most of the cases farmers will diagnose the disease by identification and will take the proper action to overcome those problems. For the farmers, those who are aware of these problems will easily handle the situation but in the case of farmers, who don’t have proper knowledge about of it, will take improper actions. Finally, it will destroy the entire crop and cause a huge loss to farmer.
This results into giant waste of human work, loss of time and money.
The primary solution for these problems is ‘Digital Farming’.
Digital Farming is applying precision location methods and decision quality agronomic information to illuminate, predict and affect the continuum of cultivation issues across the farm. This farming works on the basis of Digital image processing, Convolutional Neural networks and Machine Learning techniques. By using these techniques our system will work as a guide, to assist the farmers by giving information about Water level, Atmospheric humidity, Temperature and about plant growth, plants affected by pests in the field for the entire life span of the crop. Simultaneously it will suggest immediate measures to be taken to overcome those problems and will suggest the best pesticides to the farmer.
This complete setup of image processing and machine learning will be performed on FPGA DE Nano-10 board to use the features of parallelism and pipelining architecture for getting high speed and accuracy in assessment.
We mainly have Two Blocks for this Project:
a. HPS (Hard Processor System):
Our system will take multi spectral images from the camera and give those images as input to the HPS automatic training module (as initial field images). For certain time the process of taking multi spectral images will be repeated. These images also given as an input to the HPS automatic training module (as new images). All these initial and every new field images will automatically trained in HPS.
Along with that we will train the HPS with the Disease affected images of all plants related to all species (Ex: Paddy, wheat, cotton, chilly, all pulses etc…,). The trained data will give as input to the FPGA part.
b. FPGA (Field Programmable Gate Array):
By taking the new images and previous images data in HPS as input to the FPGA part of the board, the system will pre-process the data and undergoes Image segmentation. Then, it compares both the data and make the result. The result will be sent to the farmer mobile as a message
Our main aim of the Digital Farming is to improve agricultural yield and reduce potential environmental risks, while benefits are:
1. Water Level Sensing:
An advanced water level sensing sensor will be fixed in the field for monitoring the water level. Based on the information given to the system, our system will alert the farmer along with the water level percentage through message.
2. Atmospheric Temperature sensing:
A temperature sensing device will be fixed at the field for checking the temperature of the present atmosphere. Our employed FPGA will decide whether the atmospheric temperature is sufficient for field growth or not and send the information through message.
3. Atmospheric Humidity sensing:
An atmospheric humidity sensing sensor will be placed at the field for checking the humidity present in the atmosphere. The output of the sensor will be given as input to FPGA for checking humidity is high or normal for crops growing. Our system will send this Atmospheric humidity level to the farmer through message.
4. Detection of disease affected plants in the field:
Here also it checks the initial images and new images of the plant to give information about the disease affected to those plants in the field.
5. Diagnosis and Suggest some measure to be taken by farmer:
After detecting the disease affected to those plants, our system will send the pesticides have to be taken and suggest some precautionary measures to the famer as shown below.
Contemporary Mission vision is getting dominated day-by-day by Deep learning (With Convolution Neural Network’s being the state of art recognition system). Our ALTERA FPGA is sufficient for all the tasks mentioned above.
Recognition of different diseases and training of CNN using a collected dataset will be treated as a classification task in our entire project. We will train separate CNN’s for each type of diseases for different plants and they will be able to recognize each type of disease and gives the details of that disease (name of disease, symptoms) along with that, it will suggest list of best pesticides to eradicate the disease.
The inference using this Neural Network will be done based on FPGA as real-time performance is expected for this computationally heavy task.
We are going to design our system in the form of a single-board computer device using FPGA for complex computations. We can assure that this is possible in real time, as an inference done in CNN’s using FPGA.
Our digital farming system has a wide range of application areas in modern agriculture context.
Some of those areas are:
Our target users are, those who mainly works on agriculture based work:
Fig1:Blok Diagram of Digital diagram
SYSTEM FLOW DIAGRAM:
Fig2: System Flow Diagram of Digital Farming
Agriculture assumes a place of special significance in Indian economy. India is the only country which provides all the cultivated species of both commercial and food crops with a good number of ancestral and hybrids. Presently hybrids occupy 26% of area and contribute to 42% of total production.
Nevertheless, the production of crops like cotton in India is low on account of significant losses accrued due to pests and diseases. Ironically, the crops receive 52% of total pesticides used and these needs to be scaled down to minimum. The management of pests and diseases in an eco-friendly manner assumes paramount importance and strategy essentially requires knowledge about the pests, their ecology behavior and identification. The manipulation of ecological niche, greater reliance on bio-agents and bio pesticides is possible, if the weak links in the developmental cycle of the pests and pathogen city of diseases are properly understood by researchers and scientific personnel, so that the upgrading technology could be effectively transferred to the farming community.
Every time it is not possible for the farmers to consult agriculture researcher’s .In this situation to achieve an accurate plant/crop disease diagnostics a farmer/pathologist should possess great observation skills so that one can identify characteristic symptoms. Variations in symptoms indicated by diseased crop may lead to improper diagnosis since amateur gardeners or farmers could have more difficulties in determining it than a professional pathologist.
Then we made an attempt named “Digital Farming” to overcome those conditions in crop.
“Digital Farming” is applying precision methods and decision quality agronomic information to illuminate, predict and affect the continuum issues across the farm. This farming works on the basis of Digital image processing, Convolutional Neural networks and Machine Learning techniques. By using these techniques our system will work as a guide, to assist the farmers by giving information about Water level, Atmospheric humidity, Temperature and about plant growth, plants affected by pests in the field for the entire life span of the crop. Simultaneously it will suggest immediate measures to be taken to overcome those problems and will suggest the best pesticides to the farmer.
This complete setup of Image processing and Machine learning will be performed on FPGA
DE Nano-10 board.
Fig3: Practical setup for Digital Farming.
Paper work beforw we start the main process:
After considering the reviews and suggestions of pathologists we made a small research work to know about details of both seasonal and unseasonal diseases that affect the crops. Finally we realized that the diseases will spread through air, water and also because of worst climatic conditions.
Here we got some of the reasons we sorted out that makes a path to affect the crop
Seed decay is due to adverse weather conditions at the time of harvest or poor storage conditions which often lead to contamination by micro-organisms. The micro-organisms involved with seed decay either contaminate the seed prior to harvest or invade the seed from soil. If the seed is contaminated before planting, then significant seed decay occurs in wet conditions which often results in slow or no germination of seeds.
Fungi most often associated with deteriorated seed are as follows:
(These are very common fungi for every type of seed)
Management of Seed decay:
1. In order to prevent seed-borne diseases, ensure that seed does not remain on the plant for long period after the boll has opened.
2. Seeds should be stored in a well-aerated, dry atmosphere at 20-25 c and at moisture level of <10%.
Fungal Foliar Diseases:
Some of the crops had a large leaf surface area. The dense canopy encourages a humid micro-climate within the crop, creating an environment where large numbers of micro-organisms flourish. Many of these organisms are non-pathogenic or secondary invaders of necrotic or insect damaged tissues.
Alternaria leaf spot(Example):
Alternaria leaf spot incited by Alternaria macrosporia , is a common disease in all the cotton growing areas of the country.
Fig4: Leaf Spot disease
Soil borne diseases
Root and Stem Diseases:
Majority of pathogens affecting root and stem. However vascular diseases which are able to affect the root and lower stem of 45-55 days old plants. Some of the diseases affect plant at seedling stage but due to non-availability of conductive conditions remain latent and express full symptoms at some stage or the other.
Despite this ambiguity, there are five major diseases which are of economic importance.
Crops are affected by bacterial blight at all stages of the crop development starting from seedling, The pathogens is seed-borne and diseases is transmitted from cotyledons to leaves, followed by the main stem and bolls. Symptoms at each stage has been given different descriptive nature which is based on plant organ or the growth stage affective viz., seedling blight, angular leaf spot, vein blight, black arm and boll lesions.
Fig5:Bacterial blight affected plant
Viral Diseases in cotton are not common in India. Cotton leaf crumple virus disease was reported for the first time in India. Subsequently, another virus disease i.e., leaf curl virus was reported in Northern parts of India in 1994. Both the diseases are transmitted by whitefly. The affected plants can be distinguished from healthy plants by abnormality in leaves mostly in the form of upward or downward curling of leaf margins. In some cases, the affected plants remains stunted and give poor yields if the infection occurs in early stages of crop growth.
List of Viral Diseases:
Above all the diseases have given us a new challenge to face all the conditions at a time and this made our work possible by "DIGITAL FARMING".
By considering the feedback of the pathologists and professionals we decided to design a prototype device which can detect the soil moisture level, temperature, and humidity values along with the different types of diseases with the help of DE10-NANO board.
For this we have used both FPGA and HPS parts of DE10 board to design Digital Farming System.
By using FPGA part we have connected sensors named soil moisture, temperature and humidity to the Arduino header.
We have used three sensors to detect the temperature, humidity and soil moisture.
1. Temperature Sensor- LM35
2. Humidity Sensor - DHT 22
3. Soil Moisture Sensor - FC 28
1. Temperature Sensor - LM35:
Features of LM35 Temperature Sensor:
2. Humidity Sensor - DHT 22:
Features of DHT22 Humidity Sensor
Resolution : 0.1°C
Accuracy : ±0.5°C
Measuring range : -40°C ~ 80°C
3. Soil Moisture Sensor - FC28:
Features of FC28 Soil moisture Sensor:
Fig 8: Internal connections for Arduino to ADC
We connect the sensors directly with the analog pins (A0-A6) of Arduino header which is already embedded in the DE-10 NANO BOARD. The sensors will give the values in analog form .The output values of the sensors are in the range of 0 to 5V and its digital equivalent is 0 to 4095 (which is 12 bits i.e.., 0 to 212-1). We can’t deal with analog values in FPGA directly. So, we need an Analog to Digital Converter. To make our work easier, in our DE10 board the analog pins of Arduino header are directly connected with the pins of 2*5 ADC HEADER.
The ADC HEADER have 10 pins i.e.., 1) VCC 2) ADC_IN0 - ADC_IN7 10) GND.
These pins are directly connected with the ADC (LTC2308) which is already embedded in the DE-10 NANO FPGA Board.
Fig 9: Internal connections for Arduino Header to FPGA
1.We have to generate the control signals to access ADC (LTC 2308) which is inbuilt in DE10
NANO FPGA Board.
2.The control signals are:
To access the ADC we must have a clock with a frequency less than 40 MHZ, but the FPGA (DE-10 NANO FPGA) is having a clock frequency of 50 MHZ .So, we have generated a clock frequency of 25 MHZ from the board clock which is given to the ADC with the help of ADC_SCK. The time period of the ADC_SCK is 40ns. ADC_SCK is generated only after a time t_convst (conversion time) and it is of 480 ns (12 clock cycles).
In 12 clock cycles:
The ADC (LTC 2308) has 8 channels. To select those channels we need a channel selector and this channel selection is done using ADC_SDI. This ADC_SDI receives 6 bits for 6 clock cycles serially (i.e.., each bit is received per each clock cycle). We stored all these channel selector values in temporary registers. We are using 4 different sensors, so we have to select 4 different channels (Each separate channel for each sensor). The channel selection is done for different values of ADC_SDI and these values are obtained using a single variable x (i.e.., different channel selection values for different values of x).
Here, we have used 1, 3 and 5 channels for 3 different sensors:
Channel 1 for Moisture sensor
Channel 3 for Humidity sensor
Channel 5 for Temperature sensor
ADC_SDI values for different channels depending on X are
X=0, 110011 for moisture sensor
X=1, 110111 for humidity sensor
X=2, 111011 for temperature sensor
The ADC will take the analog input signal from the selected channel depending on the ADC_SDI value and it requires an acquisition time of 240 ns after the ADC_SDI value is given and before the next conversion starts.
The ADC (LTC 2308) has a typical conversion time (t conv) of 1300 ns to a maximum conversion time of 1600ns. We have two methods for applying ADC_CONVST to ADC.
a) Long conversion pulse
b) Short conversion pulse
We have used short conversion pulse method. Here, The ADC_CONVST must be low within 40ns after the conversion starts. The ADC will start the conversion whenever a positive edge detected in ADC_CONVST pulse. The difference between the two positive edges of the ADC_CONVST has a typical time period (t cycle) of 2000ns. The ADC_CONVST go high whenever the count begins (i.e.., count==0). The ADC_SCK, ADC_SDI will be given as inputs to ADC and ADC_SDO will be taken as output after the typical conversion time (t conv) only.
ADC takes the ADC_SCK, ADC_CONVST and ADC_SDI values from FPGA and generates a 12 bit resolution output of previous ADC_SDI value in 12 clock cycles. Single bit for each clock cycle (i.e.., serial data comes from the ADC). SDO transition occurs on the falling edge of each SCK pulse. These outputs of ADC (ADC_SDO) are stored in a 12 bit temporary register .In this manner all the sensor values are stored in moisture_o, humidity_o and temperature_o temporary registers respectively.
Fig 10: Timing diagram for Short Conversion table
We considered some reference values of temperature, humidity and soil moisture level for a plant to grow well. The obtained sensor values (values in temporary registers) are compared with these reference values and the conditions like good, low and high are checked and two status signals are generated for each sensor .So there are a total of 6 status signals will come as output from FPGA .These status signals are given to digital pins of Arduino using GPIO pins of the DE10 Nano board. The Arduino receives the status signals from the FPGA through GPIO pins. Based on these status signals, the Arduino sends message to Farmer about the field condition through GSM.
Arduino Uno Technical Specification:
• Microcontroller : AT MEGA 380P , 8 bit AVR family microcontroller
• Operating Voltage: : 5V
• Recommended Input Voltage : 7-12V
• Input Voltage Limits : 6-20V
• Analog Input Pins : 6 (A0 – A5)
• Digital I/O Pins : 14 (Out of which 6 provide PWM output)
• Frequency (Clock Speed) : 16 MHz
Fig 11: Connections for DE10 NANO Board with Arduino UNO
With the help of GSM we sent the final output values of sensors to Farmer’s mobile.
Global System for Mobile Communications (GSM):
GSM is a mobile communication modem. We use SIM900A GSM Modem. The module supports communication in 900MHz band. It is used for sending and receive text message. Here we are using 3 pins Transmitter, Receiver and ground.
1. Single Supply Voltage: 3.5 - 4.5V
2. Features keypad interface
3. Features display interface
4. Features Real Time Clock
5. Supports UART interface
6. Supports single SIM card
7. Communication by using AT commands
Booting the GSM Module
1. Insert the SIM card to GSM module and lock it.
2. Connect the adapter to GSM module and turn it ON!
3. Now wait for 1 minute and see the blinking of ‘status LED’ or ‘network LED’ (GSM module will take some time to establish connection with mobile network)
4. Once the connection is established successfully, the status/network LED will blink continuously every 3 seconds
Connection between GSM and Arduino:
The communication between Arduino and GSM module is serial. In general, we need only three connections to make connections between the GSM module and the Arduino. To wire for this mode, the transmitter (TX) pin of the GSM module has to be connected to the receiver (Rx) pin of the Arduino. Similar, the receiver (Rx) pin of the GSM module has to be connected to the transmitter (TX) pin of the Arduino. The grounds of both GSM and Arduino are connected with common ground.
GSM Tx –> Arduino Rx and GSM Rx –> Arduino Tx.
By properly communicating with the GSM module from the Arduino according to its standard protocol, it is possible to sending and receiving text message. So we have to used AT Commands, which are required for establishing communication with the GSM module
1. AT+CMGF=1, this command used for set the GSM module to text mode first.
2. AT+CMGS=\ “mobile number\ “\r, this command for we have to send the mobile number to which we want to send SMS
Fig 12: Block Diagram for sending sensors information from using Arduino through GSM
Now, with the help of HPS part of the board we designed a network to detect the Plant leaf disease by using Deep learning.
Deep learning becomes a modern technique for image processing and data analysis, with a better potentiality and accurate results. Unfortunately, deep learning has become a very successful domain and started entering into the every domain as a part of this it entered into agricultural domain. In present scenario, Convolutional Neural Network can be considered as the leading methods for object detection.
Fig 13: Block Diagram for Training of Image dataset
By importing python libraries we started to implement a convolution network.
An appropriate dataset is required at all stages of object recognition research, starting from training phase to evaluating the performance of recognition algorithms. (CNN requires a large amount of data to be trained for the Network.) As a first step of our practical work we collected image data set of different plant leaves which were affected by different diseases. In order to distinguish healthy leaves we just added some of the healthy leaves to the data set. Finally data base containing maximum number of images for training and for evaluation was created.
Pre-processing and Labeling Images:
Preprocessing images commonly involves removing low-frequency background noise, normalizing the intensity of the individual particle images, removing reflections, and masking portions of images.
The acquired dataset consists of images with minimal noise and hence noise removal was not a necessary preprocessing step. The images in the dataset were resized to minimum resolution to speed up the training process and make the model training computationally feasible.
The process of standardizing either the input or target variables tends to speed up the training process. This is done through improvement of the numerical condition of the optimization problem. It also made sure that the several default values involved in initialization and termination are appropriate. For our purpose, we normalized the images to get all the pixel values in the same range.
Before resizing the images’ we will convert them RGB to the grey model.
For the purpose of Plant disease detection we made experiments with deep learning architectures like Alex Net, Google Net but the best results could be seen with LeNet architecture with accuracy in training data. Here we considered LeNet architecture.
Convolutional neural networks (CNNs) consist of multiple layers of receptive fields. These are small neuron collections which process portions of the input image. The outputs of these collections are then tiled so that their input regions overlap, to obtain a higher-resolution representation of the original image; this is repeated for every such layer. Tiling allows CNNs to tolerate translation of the input image. Convolutional networks may include local or global pooling layers, which combine the outputs of neuron clusters. They also consist of various combinations of convolutional and fully connected layers, with point wise nonlinearity applied at the end of or after each layer. A convolution operation on small regions of input is introduced to reduce the number of free parameters and improve generalization. [One major advantage of convolutional networks is the use of shared weight in convolutional layers, which means that the same filter (weights bank) is used for each pixel in the layer; this both reduces memory footprint and improves performance. The convolutional neural network is also known as shift invariant or space invariant artificial neural network (SIANN), which is named based on its shared weights architecture and translation invariance characteristics. The convolutional layer is the essential building block of the convolutional neural network. The layer’s parameters are comprised of a set of learnable kernels which possess a small receptive field but extend through the full depth of the input volume.
For this purpose we have taken the help of Tensorflow in python to train and test our data from camera.
TensorFlow offers multiple levels of abstraction so you can choose the right one for your needs. Build and train models by using the high-level Keras API, which makes getting started with TensorFlow and machine learning easy.
Fig 14: Visual Idea for flow of Convolution
Fig 15: Convolution steps in HPS
There are many important sateps in the Convolution Neural Network. These include the following operation.
Depth: Depth corresponds to number of filters we use for the convolution operation.
Stride: Stride is the number of pixels by which we slide our filter matrix over the input matrix.
Zero-padding: Sometimes, it is convenient to pad the input matrix with zeros around the border, so that we can apply the filter to bordering elements of our input image matrix.
Non-Linearity: ReLU is an element wise operation (applied per pixel) and replaces all negative pixel values in the feature map by zero. The purpose of ReLU is to introduce non-linearity in our ConvNet, since most of the real-world data we would want our ConvNet to learn would be non-linear.
(Convolution is a linear operation element wise matrix multiplication and addition, so we account for non-linearity by introducing a non-linear function like ReLU).
And these Rectified Linear Units (ReLU) are used as substitute for saturating nonlinearities.
Spatial Pooling: Spatial pooling can be called as sub sampling or down sampling. It reduces the dimensionality of each feature map but retains the most important information.
It can be of different types: Max pooling, summing etc…
Activation: This activation function adaptively learns the parameters of rectifiers and improves accuracy at negligible extra computational cost. In the context of artificial neural networks, the rectifier is an activation function defined as:
f (x)=max(0,x), where x is the input to a neuron.
This is also known as a ramp function and is analogous to half-wave rectification in electrical engineering.
Dropout: Dropout regularization randomly drops neurons in the network during each iteration of training to reduce the variance of the model and simplify the network which aids in the prevention of overfitting. Finally, the classification block consists of two sets of fully connected neural network layers. The second dense layer is followed by a softmax activation function to compute the probability scores for the entire classes.This was performed on HPS part of DE10 –NANO FPGA board.
Fig 16: Block Diagram of Image taken for Tesing
For testing a given input image of leaf from camera, we started to import python libraries.
Tkinter: Tkinter is the standard GUI library for Python. Python when combined with Tkinter provides a fast and easy way to create GUI applications. Tkinter provides a powerful object-oriented interface to the Tk GUI toolkit.
By using this toolkit we designed a Graphical user interface to visualize how our data test the image whether it is affected by any infectious disease or not.
Here we have shown “what are the remedies to be taken to eradicate those diseases?” and how to save the crop from spreading that disease.
As like of Data training we convoluted the input image and then tested with samples of trained data.
ENTIRE SETUP TO PERFORM SENSOR PART ON FPGA:
CONDTIONS FOR SOIL IN DIFFERENT MOISTURE LEVELS:
1. WHEN SOIL IN WET CONDITION AND MEESAGE SENT TO MOBILE:
2. WHEN SOIL IN DRY CONDITION AND MESSAGE SENT TO MOBILE:
Trained data results in HPS:
Here we can see the trained results of our data set for different disease affected plants.
1. For Epoch1:
2. For Epoch2:
3. We can see loss reduction during training:
4. For Epoch 3,4,5,6:
5. For Epoch 5, 6, 7:
6.For Epoch 6,7,8:
GUI has been created when the testing was started:
Click “GET PHOTO” button to browse a photo taken from the camera.
Browse the image taken from the camera as shown below
Here the image was loaded and to check the health status of crop ‘Click on Analyze Button’
Status of crop health and Its Disease name was displayed on GUI.
If it was affected by disease “Click on Remedies button to check for Remedies.”
Finally, the list of remedies was displayed on GUI.
Press the EXIT button to exit from it.
Here we successfully designed a small device that that can detect the plant diseases in our crops and will suggest the best pesticides to eradicate those diseases along with that our system send the information about of soil moisture, temperature and humidity (climatic conditions in field) to the farmers mobile.
In present scenario AI and Deep learning will become a sovereign for every domain. And our project ultimate goal is to help the farmers those, who are oblivious about of plant diseases and the different type of climatic conditions at crops. We hope that the result of our projects will make a small path to design AI assistants to guide the farmers in upcoming years. And it leads to a low-cost portable device that can track and monitor the farming areas.