Our project will be a self-driving boat that collects garbage floating on water. The boat will have a camera mounted on it, which sends image captures to the DE10-Nano board to analyze where, in relation to the boat, the garbage is. The board then controls the motors of the boat to drive it to the waste and picks it up. There will be a Wi-Fi system built-in so that the board can send live video feedback to a connected PC, while also receive commands from the PC to control the boat, in case it goes out of control. The garbage collection is, under most circumstances, fully automated. It uses sensors to detect if the collection is approaching its max capacity, and will return to a set location near land if it does, to dump out the trash and prepare for the next round of garbage collection. We aim to utilize edge computing for image recognition on objects floating on water, to distinguish whether they are actually garbage or not, confirm their location, and compute how to get there.

Demo Video

Project Proposal

1. High-level project introduction and performance expectation

Every year there are 115 million to 241 million tons of plastic  garbage flowing from rivers to the ocean. For the purpose of solving this   kind of garbage pollution that severely affects our environment, we have decided to design a self-propelled boat that can pick up garbage automatically on rivers, ponds, reservoirs and even oceans. Our project is targeted at any parties who wish to rid the water of floating waste, environmental organizations, or governments of countries that have ample amounts of rivers, reservoirs, or long coastlines. This project can be easily extended into larger scales of garbage collection on bodies of water, but in the limited time, we estimate that it will be able to operate in a radius of 50 meters.

There are plenty of choices when it comes to choosing a     development environment, each of them has their own advantages. In this competition, we decided to use Intel FPGA devices for the following reason. First of all, the FPGA devices are generally considered to be the most efficient way to develop designs. The FPGA devices can be programmed, debugged, reprogrammed, and repeated operations readily, as the result, it can fully carry out design development and verification with relatively less risk. These characteristics make us easily get start on our design and offer the design flexibility in the development. We can implement any idea through FPGA and add any function to our project without any concern. What's more, DE10-Nano board has enabled us to use image recognition through programming inside the board without buying another additional board, which saves a lot of money and work. Intel FPGA devices have strong calculating abilities that help us recognize pictures to identify garbage in a short period time and find out its location. DE10-nano also board also gives a stable environment for doing image recognition, it can consistently recognize the object, which solves one of biggest problem in many image recognition projects. In this case, it raises the efficiency and helps the boat collect more garbage in the same period. We do not want to see our device take its sweet time to finish its calculation.

It is also super easy to upload our code onto FPGA devices, which enables us to check whether our codes work well in real situations, saved a lot of time and effort. DE10-Nano has a lot of Processor I/O which enable us to plug multiple devices and use it at the same time, which saves us a lot of time during designing the project and enhance the board's practicability. For example, we thanks to the HPS Gigabit Ethernet, we connect wifi to the board and make communication through wifi to send different data and successfully design the manual way to control our boat, for the purpose to control our boat in some special conditions.

Due to time constraint, we did not implement the part about connecting board to the cloud and use Microsoft Azure IoT provided by the board to record how many garbage we collect every day. Then, we can analyze the data and monitor whether the area is polluted and need more actions to improve the environment.

2. Block Diagram

                                                               Fig. 1

                                                               Fig. 2

As Fig. 1 implies, the DE10-Nano board is connected to the motor driver board and the camera via cables. The power bank feeds into the board and the batteries feed into the motor driver board. The driver board connects to the motors at the bottom of the boat using some wires.

Fig. 2 is the actual appearance of our boat for the project.

3. Expected sustainability results, projected resource savings

Every year there are 115 million to 241 million tons of plastic garbage flowing from rivers to the ocean, significantly affecting the ocean environment since about 800 kinds of marine species are suffering from the pollution. To solve this environmental problem, we designed a small boat to collect garbage. We estimate that our boat can collect up to 5 kilograms of garbage per hour, and by working 10 hours a day, it can pick up about 50 kilograms of garbage a day. This result is only based on one boat, if multiple boats are working at the same time, there will be a tremendous amount of garbage collected. What’s more, the boat is totally based on electricity and it can achieve zero carbon emission. To solve the problem of electricity usage we plan to place solar panels on the boat, which can provide power for a mobile power bank which will then provide power for our boat and motors.

Our goal is to clean up floating garbage, we expect to reach the goal of cleaning up 10 million to 15 million garbage if our project can be implemented to larger scale, such as a larger boat or multiple boats. Intel FPGA devices provide us with the ability to analyze pictures quickly and find out whether there is garbage, which increases the efficiency to collect garbage and improves the amount of garbage that we can collect in the same period. Furthermore, since it is entirely automated, we do not need to hire people to clean up the water, which saves around $70,000~$80,000 NTD a month, and this money can be spent in other applications that can benefit our environment.

4. Design Introduction

Every year there are 115 million to 241 million tons of plastic  garbage flowing from rivers to the ocean. For the purpose of solving this   kind of garbage pollution that severely affects our environment, we have decided to design a self-propelled boat that can pick up garbage automatically on rivers, ponds, reservoirs and even oceans. Our project is targeted at any parties who wish to rid the water of floating waste, environmental organizations, or governments of countries that have ample amounts of rivers, reservoirs, or long coastlines. This project can be easily extended into larger scales of garbage collection on bodies of water, but in the limited time, we estimate that it will be able to operate in a radius of 50 meters.

There are plenty of choices when it comes to choosing a     development environment, each of them has their own advantages. In this competition, we decided to use Intel FPGA devices for the following reason. First of all, the FPGA devices are generally considered to be the most efficient way to develop designs. The FPGA devices can be programmed, debugged, reprogrammed, and repeated operations readily, as the result, it can fully carry out design development and verification with relatively less risk. These characteristics make us easily get start on our design and offer the design flexibility in the development. We can implement any idea through FPGA and add any function to our project without any concern. What's more, DE10-Nano board has enabled us to use image recognition through programming inside the board without buying another additional board, which saves a lot of money and work. Intel FPGA devices have strong calculating abilities that help us recognize pictures to identify garbage in a short period time and find out its location. DE10-nano also board also gives a stable environment for doing image recognition, it can consistently recognize the object, which solves one of biggest problem in many image recognition projects. In this case, it raises the efficiency and helps the boat collect more garbage in the same period. We do not want to see our device take its sweet time to finish its calculation.

It is also super easy to upload our code onto FPGA devices, which enables us to check whether our codes work well in real situations, saved a lot of time and effort. DE10-Nano has a lot of Processor I/O which enable us to plug multiple devices and use it at the same time, which saves us a lot of time during designing the project and enhance the board's practicability. For example, we thanks to the HPS Gigabit Ethernet, we connect wifi to the board and make communication through wifi to send different data and successfully design the manual way to control our boat, for the purpose to control our boat in some special conditions.

Due to time constraint, we did not implement the part about connecting board to the cloud and use Microsoft Azure IoT provided by the board to record how many garbage we collect every day. Then, we can analyze the data and monitor whether the area is polluted and need more actions to improve the environment.

5. Functional description and implementation

• General description

Our boat uses image recognition to find the location of garbage floating above water level, plans the best route and drives itself toward the garbage, then picks up the garbage automatically.

• Image recognition

To implement the image recognition, we needed to set up the environment on the DE10-nano board. We set up Ubuntu, OpenCV 4.4, Python, and used Mobilenet-SSD to implement deep image recognition. We had also tried Yolo4-Full and Yolo4-Tiny, but found out that Mobilenet-SSD is the best choice since it provides the best performance. The training set we used is Microsoft Common Objects in Context.

The board is trained to find the target object, a.k.a. garbage, in the environment. When it detects an object, it boxes up the target in a rectangle, and sends back the x-y coordinates of the object by taking the average of the coordinates at its upper left and lower right corner, which will give us the coordinates at the center of the target. The center coordinates are put into a virtual 3-by-3 grid like the one in Fig. A to determine which square it is in. After finding which section it is in, the action for the motors can be determined.

Fig. A

For example, the picture below is captured by the camera:

The image is sent to the de10-nano board to analyze, more specifically, it uses OpenCV and Mobilenet-SSD to detect the target object. The board then computes its coordinates as shown below:

Then, the board finds out where the center of the target is in the grid:

The target is in the top middle part of the grid. With this information, the board can the corresponding control signals to the motors.

• Motor control

Each section has a corresponding motor movement. Using Fig. A as reference, for section 1 and 4, right motor will move full speed ahead, and left motor will stop. For section 7, right motor will also move full speed ahead, but left motor will stop, since it is a sharp left. For section 3,6, and 9, the motor actions are the same but reversed left and right. And for section 2,5,8, both motors will move full speed ahead. If there is no object in view, the boat will spin on the spot until it sees trash to pick up, and then repeat the above process again.

 Fig. B

We later discovered that the electric circuit for controlling motor may cause negative distractions to FPGA and it didn't work well if we wanted to implement manual controlling of the motors. Therefore, to allow manual control from a remote device and seperate motor control system from FPGA with network, we used an Adafruit Feather HUZZAH ESP8266 Wifi connected to a motor driver board, which allows wifi connection. The wifi board is connected to a router, which is connected to the De10-nano via an Ethernet cable. This not only allows wireless connection between the wifi board and the De10-nano, but also remote access with other devices such as phones or computers. Fig. C is the wifi board attached to  the motor driver board. Fig. D is the interface of our manual controller.

 Fig. C                  Fig. D

• Boat

The boat we used is a custom designed catamaran boat that we 3D printed out. Catamarans have less hull volume, smaller displacement, and shallower draft than monohulls of comparable length, which is why we decided on this specific boat structure. The tubes and walls on both side are made hollow so that we can connect the motors and the de10-nano board easily. The propeller blades are designed in consideration of the conservation of angular momentum, where one spins clockwise and the other spins counterclockwise.

6. Performance metrics, performance to expectation

Every year there are 115 million to 241 million tons of plastic garbage flowing from rivers to the ocean, significantly affecting the ocean environment since about 800 kinds of marine species are suffering from the pollution. To solve this environmental problem, we designed a small boat to collect garbage. We estimate that our boat can collect up to 5 kilograms of garbage per hour, and by working 10 hours a day, it can pick up about 50 kilograms of garbage a day. This result is only based on one boat, if multiple boats are working at the same time, there will be a tremendous amount of garbage collected. What’s more, the boat is totally based on electricity and it can achieve zero carbon emission. To solve the problem of electricity usage we plan to place solar panels on the boat, which can provide power for a mobile power bank which will then provide power for our boat and motors.

Our goal is to clean up floating garbage, we expect to reach the goal of cleaning up 10 million to 15 million garbage if our project can be implemented to larger scale, such as a larger boat or multiple boats. Intel FPGA devices provide us with the ability to analyze pictures quickly and find out whether there is garbage, which increases the efficiency to collect garbage and improves the amount of garbage that we can collect in the same period. Furthermore, since it is entirely automated, we do not need to hire people to clean up the water, which saves around $70,000~$80,000 NTD a month, and this money can be spent in other applications that can benefit our environment.

7. Sustainability results, resource savings achieved

8. Conclusion

Thanks to this contest, we had the opportunity to use Intel FPGA devices and DE10-nano to design a project which can really help the earth. However, we are not able to implement every idea about the project due to limit of time, there are some more ideas that we are sure we can add to our project and extend the efficacy of our project which can then help the environment more. We would like to optimize our strategy to pick up the garbage since the method we used in the demo video is a rough version that cannot collect too many garbage. What’s more, we can use Wi-Fi and Microsoft Azure IoT to record how many garbage we collect every day and analyze the data, monitor whether the area is polluted and need more actions to improve the environment.

FPGA devices gave us the chance to change the world, we personally experienced that our idea can really make an affect and help solve some of the tough questions. We have responsibility to take good care of the environment, so we are really excited that we can use what we have learned and explore solutions that can reduce environmental impact.