--- title: "Strategies for Cooperative UAVs Using Model Predictive Control" authors: - "Amith Manoharan" doi: "10.24963/ijcai.2020/738" url: "https://ijcai.org/proceedings/2020/738" pdf: "https://ijcai.org/proceedings/2020/0738.pdf" venue: "Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence" year: 2020 pages: "5196-5197" track: "Doctoral Consortium" keywords: - "Robotics: Motion and Path Planning" - "Robotics: Multi-Robot Systems" - "Robotics: Localization, Mapping, State Estimation" - "Agent-based and Multi-agent Systems: Coordination and Cooperation" --- # Strategies for Cooperative UAVs Using Model Predictive Control Amith Manoharan *Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence*, pages 5196-5197. DOI: [10.24963/ijcai.2020/738](https://doi.org/10.24963/ijcai.2020/738) ## Abstract Unmanned aerial vehicles (UAVs) have reached significant maturity over several years for safe civilian operations like mapping, search and rescue. The operation performance can be significantly improved by deploying multiple cooperating UAVs and optimal decision making. In this work, we present the use of nonlinear model predictive control (NMPC) for two different applications involving cooperative UAVs. ## Full text ## 1 NMPC Based Approach for Cooperative Target Defence The multi-agent pursuit-evasion problem is an application that appears in several domains. An interesting subset of the multi-agent pursuit-evasion problem is the three agent case, involving target, attacker and defender [Garcia et al., 2015]. If the target and the defender cooperate, then they can maneuver such that defender intercepts the attacker while target maximizes the chances of escape. There are many applications like aerial surveillance, communication relaying, where the target may need to stationary. In such situations, a defender tries to defend the target from any attacking agents. The limitations with the earlier methods considering the three-body interception scenario are that (i) the attacker states are assumed to be known accurately and (ii) the target has constant speed. In this study [Manoharan et al., 2019b], we propose a control strategy that combines a Nonlinear Model Predictive Control (NMPC) scheme with state estimation using an Extended Kalman Filter (EKF) while addressing (i) and (ii). The formulation also considers the case in which the target can be stationary unless strictly necessary. ## 1.1 NMPC Formulation We propose a strategy wherein the target-defender team will use NMPC to compute the control commands so that the target evades the attacker, while the defender intercepts the attacker before it could capture the target. The formulation allows the target to move or stop based on the engagement geometry. The objective of the target is to stay still so as to capture images of the area of interest, and to move only when necessary. This condition can be achieved by defining a threshold, e which is the safe distance of the target from the attacker. The target should always try to keep this safe distance, e from the attacker. The NPMC problem is defined as: where X is the state vector, U is the control vector, ux, uy are the velocity components of the target in x and y directions respectively, R and r represents the distance between the attacker and the target and the distance between the defender and the attacker respectively (see [Manoharan et al., 2019b] for more details). ## 1.2 Results The performance of the proposed MPC formulation was evaluated through numerical simulations and hardware experiments performed using ground rovers. A comparison with previously studied CLOS formulation [Ratnoo and Shima, 2011] was also carried out to show the efficacy of the proposed scheme. The experimental results are shown in fig. 1. A video of preliminary results can be seen at: https://youtu.be/IuvKP1vifh0. Figure 1: Trajectory of Unmanned Ground Vehicles (UGV) ![Image](https://ijcai.org/proceedings/2020/fulltext/738/image_000000_d82047282d4ca8c3ad62a32b1798683d51f715682452d0e75493ef4d210c11ab.png) ## 2 Nonlinear Model Predictive Control to Aid Cooperative Localization We propose a nonlinear model predictive control (NMPC) scheme to tackle the problem of localization and path planning of a group of unmanned aerial vehicles (UAVs) in global positioning system (GPS) denied environments [Manoharan et al., 2019a]. It is assumed that the UAVs can cooperate by sharing information among themselves. It is also assumed that the area under consideration contains some landmarks with known locations. The NMPC computes the optimal control inputs for the vehicles such that the vehicles cooperate to transit from a source location to a destination while choosing a path that will cover enough landmarks for localization. An Extended Kalman Filter (EKF) is used to estimate the vehicle positions using only relative bearing measurements. ## 2.1 Proposed NMPC Formulation is the cost associated with minimizing the distance between the vehicle and the destination, Xi is the state vector of ith vehicle, nv is the number of vehicles and C2i is the cost associated with maintaining the connection with the landmarks which is defined as: where η is a constant representing the minimum number of required connections and λi is defined as the maximum eigenvalue of the Laplacian matrix. The weight W in equation (3) is equal to W, if 3σp ≥ σc, else W = 0. σp is the standard deviation in the estimated position of the vehicles and σc is the critical value of standard deviation. This adaptive weight formulation enables determination of the optimal path for the vehicle while satisfying the estimation error within bounds (see [Manoharan et al., 2019a] for more details). ## 2.2 Results Monte-carlo simulations were performed using three UAVs with different number and configurations of landmarks. One such scenario is shown in fig. 2 for 3 UAVs and 20 landmarks. ## 3 FutureWork Involving AI Techniques Determining optimal multi-agent control theoretic strategies for large number of agents is difficult due to the sensing and computational limitations. However, AI techniques have shown promise with reduced computation time at the run time while providing solution close to optimal. Reinforcement Learning (RL) is a promising candidate that has been applied for multi-agent systems [Panait and Luke, 2005] and shepherding problem [Go et al., 2016]. SARSA and Q-learning are two well-known examples for learning algorithms in reinforcement learning that are also promising approaches. The main difference between the classical dynamic programming methods and the reinforcement learning algorithms is that the latter do not assume knowledge of an exact mathematical model of the Markov Decision Process (MDP) and thus can be used for large MDPs where exact methods become infeasible. Figure 2: Multi-UAV trajectories ![Image](https://ijcai.org/proceedings/2020/fulltext/738/image_000001_fbf7c25b07c1e7055cf4d4f285791c3cee2ed6764cd9689e9937b4f42739069b.png) Therefore the multi-vehicle cooperative localization problem can be extended to use AI techniques for finding the best path with less computational effort and time. Also, for the extension of the single target-attacker-defender problem to a pursuit-evasion problem with multiple number of agents, AI techniques are the best suited option. ## References 1. [Garcia et al., 2015] Eloy Garcia, David W Casbeer, and Meir Pachter. Cooperative strategies for optimal aircraft defense from an attacking missile. Journal of Guidance, Control, and Dynamics, 38(8):1510-1520, 2015. 2. [Go et al., 2016] Clark Kendrick Go, Bryan Lao, Junichiro Yoshimoto, and Kazushi Ikeda. A reinforcement learning approach to the shepherding task using sarsa. In Inter- national Joint Conference on Neural Networks (IJCNN), pages 3833-3836. IEEE, 2016. 3. [Manoharan et al., 2019a] Amith Manoharan, Rajnikant Sharma, and PB Sujit. Nonlinear model predictive control to aid cooperative localization. In International Conference on Unmanned Aircraft Systems (ICUAS), pages 26-32. IEEE, 2019. 4. [Manoharan et al., 2019b] Amith Manoharan, Mandeep Singh, Andrea Alessandretti, JG Manathara, SC Prusty, Nishant Mohanty, IS Kumar, Ashutosh Sahoo, and PB Sujit. Nmpc based approach for cooperative target defence. In American Control Conference (ACC), pages 5292-5297. IEEE, 2019. 5. [Panait and Luke, 2005] Liviu Panait and Sean Luke. Co- operative multi-agent learning: The state of the art. Au- tonomous agents and multi-agent systems, 11(3):387-434, 2005. 6. [Ratnoo and Shima, 2011] Ashwini Ratnoo and Tal Shima. Line-of-sight interceptor guidance for defending an air- craft. Journal of Guidance, Control, and Dynamics, 34(2):522-532, 2011.