AI for Conservation: Aerial Monitoring to Learn and Plan against Illegal Actors

1 Introduction

Conservation of our planet's natural resources is of the utmost importance and requires constant innovation. This project focuses on innovation for one aspect of conservation: the reduction of wildlife poaching. Park rangers patrol parks to decrease poaching by searching for poachers and animal snares left by poachers. Multiple strategies exist to aid in these patrols, including adversary behavior prediction and planning optimal ranger patrol strategies. These research efforts suffer from a key shortcoming: they fail to integrate real-time data, and rely on historical data collected during ranger patrols.

Recent advances in unmanned aerial vehicle (UAV) technology have made UAVs viable tools to aid in park ranger patrols. There is now an opportunity to augment the input for these strategies in real time using computer vision, by (i) automatically detecting both animals and poachers in UAV videos, (ii) using these detections to learn future poaching locations and to plan UAV patrol routes in real time, and (iii) using poaching location predictions to determine where to fly for the next patrol. In other words, detection is done on realtime data captured aboard a UAV. Detection will then be used to learn adversaries' behaviors, or where poaching may occur in the future, in future work. This will then be used to plan where to fly in the long term, such as the next mission. Finally, planning where to fly next during the current flight will depend on the long term plan and the real-time detections.

This proposed system directly improves wildlife security. Through our collaboration with Air Shepherd, a program of the Charles A. and Anne Morrow Lindbergh Foundation, we have already begun deploying poacher detection prototypes in Africa and will deploy further advances there in the future (Fig. 1). Furthermore, this also applies to similar surveillance tasks, such as locating people after natural disasters.

2 Motivation and Previous Work

In order to protect wildlife, park rangers patrol wildlife parks to find poachers, or traps placed by poachers. During these patrols, rangers record signs of poaching activity, such as snares, evidence of campfires, etc. These data are collected infrequently, only during the day, and are not analyzed in real time since they must be entered into a database after patrols. UAVs can provide image data more frequently and in real time, and can even provide data at night when poaching typically occurs through the use of thermal infrared cameras. Because raw imagery does not provide the same data as a park ranger, detection of poachers and animals in the video is needed to automatically generate similar data. Automatic detection in thermal infrared videos captured aboard UAVs is extremely difficult, since (i) the varying altitude of the UAV can sometimes lead to extremely small humans and animals, possibly less than 20 pixels in the images, (ii) the motion of the UAV makes stabilization, and consequently human and animal motion detection, difficult, and (iii) the thermal infrared sensor itself leads to lower resolution, singleband images, much different from typical RGB images (Fig. 1). Existing computer vision algorithms such as motion detection, shape detection, and temperature segmentation using automatic thresholding techniques, all struggle due to the difficulties of thermal infrared images. Given these limitations, we show that the great strides in object detection using convolutional neural networks are still applicable to this unique domain, and provide three major contributions: (i) we provide a novel video labeling application, VIOLA, to efficiently acquire labeled data to train these convolutional neural networks, (ii) we provide SPOT, the first (to our knowledge) aerial thermal detector for wildlife and poachers; (iii) we analyze the effect of simulated data on detection performance.

Because thermal infrared imagery is different from the photos used to train algorithms like Faster RCNN [Ren et al., 2015], labeled thermal infrared imagery is required to use these models for our detection. As a result, we developed VIOLA [Bondi et al., 2017], an application that assists in labeling objects of interest such as wildlife and poachers in thermal infrared imagery. After labeling 70 videos of varying altitude and resolution over the course of 6 months, we then fine-tuned Faster RCNN to locate and classify objects of interest as poachers or animals to create a system, SPOT [Bondi et al., 2018b], with efficient processing on Microsoft Azure.

We evaluated SPOT based on both historical videos captured at different altitudes and a test run by Air Shepherd in the field (Fig. 2). We perform better than Air Shepherd's current application, EyeSpy, in both precision and recall for large-sized poachers and animals, and in the field test video. For example, for the historical video containing large poachers, SPOT achieves 0.3977 precision and 0.0188 recall, whereas EyeSpy achieves 0.0052 precision and 0.0159 recall. Additionally, SPOT significantly reduces the burden on human operators since it is fully automated, whereas EyeSpy requires expert tuning of 14 parameters. We also use simulated data to augment the current dataset using AirSim-W [Bondi et al., 2018a], a simulator for UAVs, by converting RGB images into thermal infrared images with simple physical models. With simulated data, SPOT achieves 0.7799 precision and 0.0374 recall on the same historical video.

3 Research Plan

To reiterate, UAV-based imagery can be utilized for antipoaching by detecting objects of interest automatically, learning where adversaries may be in the future and planning accordingly in real time, and using adversary location predictions to determine where to fly for the next mission. We have already deployed a detection prototype. Now, we will follow three key steps: (i) improving the existing detection system, (ii) creating a flight planning framework, and (iii) integrating detection data into a poaching prediction system.

To improve detection results, a new model will be developed that works better for smaller objects, takes advantage of motion, and is tailored to thermal infrared imagery. Additional data at more altitudes or with rare content can be generated with AirSim-W. Once more accurate detection has been achieved, we plan to consider flight planning. No previous work handles real-time, adaptive planning for wildlife security. These new flight strategies will need to make new decisions in real time in order to track detected poachers, and reason about uncertainty in detection, for example by flying lower if there is a possible poacher. These extensions will involve partially observable Markov decision processes (POMDPs), in which the uncertain observation is a detection that would lead to some belief in the presence of a poacher. They will also draw upon behavioral models, such as in [Nguyen et al., 2016], in order to make real-time decisions based on detections, and incorporate data from previous flights, for example how poachers evaded capture in the past.

Finally, we will develop techniques for integrating data into poaching predictions. A starting point will be models such as [Gholami et al., 2017], which show promising results on historical park ranger patrol data. We will incorporate realtime data and assign higher weights to newer sightings.

4 Conclusion

We will utilize UAVs to acquire data in real time for antipoaching, with the goal of improving planning and prediction in these domains. In particular, we will (i) detect objects of interest such as poachers automatically, (ii) learn where adversaries such as poachers may be in the future and plan accordingly, and (iii) use adversary location predictions to plan future missions. In doing so, we hope to both advance computer science knowledge, and to tackle real-world problems.

5 Acknowledgments

This was supported by Microsoft AI for Earth, NSF (CCF-1522054), UCAR N00173-16-2-C903, primary sponsor Naval Research Laboratory (Z17-19598), and the Viterbi School of Engineering Ph.D. Merit Top-Off Fellowship.