Cooperative control strategies for autonomous agents using nonlinear model predictive control

Abstract

Autonomous aerial vehicles (AAVs) are extensively used in civilian and military applications like aerial surveying, search and rescue, transportation, border patrolling, etc. In most applications, achieving the objectives using a single AAV is difficult. Hence, multiple cooperative AAVs are used to accomplish the mission quickly and efficiently. However, achieving cooperation is challenging in real-world scenar- ios due to the uncertainties in obtaining other vehicle states (position, velocity, etc.) and measurement information. The constraints, such as limited sensor range and availability, noises, and environmental disturbances, must be handled properly to obtain an efficient system. In this thesis, we provide solutions for a three-agent and four-agent pursuit-evasion problem, path planning under localization constraints, and tracking ground vehicles for cinematography purposes. The optimal control commands for the coop- erative agents in each of these problems are found using the nonlinear model predictive control (NMPC) framework. We analyze the theoretical properties of the proposed solutions and show the performance through numerical simulations. Brief explanations of the proposed solutions are given in the following paragraphs. Chapter 2 presents a cooperative target defense guidance strategy using a nonlinear model predictive control (NMPC) framework for a target-attacker-defender (TAD) problem. The TAD problem consists of an attacker and a cooperative target-defender pair. The attacker’s objective is to capture the target, whereas the target-defender team acts together such that the defender can intercept the attacker and ensure target survival. We assume that the cooperative target-defender pair do not have perfect knowledge of the attacker states, and hence the states are estimated using an extended Kalman filter (EKF). The capture analysis based on the Apollonius circles is performed to identify the target survival regions. The efficacy of the NMPC-based solution is evaluated through extensive numerical simulations, and hardware experiments performed using ground rovers. We also compare our approach against previous studies, iv which use the command to line of sight (CLOS), and augmented command to line of sight (A-CLOS) guidance strategies. Chapter 3 presents a cooperative target defense strategy using nonlinear model-predictive control (NMPC) framework for a two–target two–attacker (2T2A) problem. Each attacker needs to capture a designated target individually. The objective of the two targets is to cooperate such that they lure the two attackers into a collision. We assume that the targets do not have perfect knowledge of the attacker states, and hence they estimate the attacker states using EKF. The NMPC scheme computes closed-loop optimal control commands for the targets while satisfying state and control constraints. Theoretical analysis is carried out to determine escape regions that will lead to the targets’ survival or capture. Numerical simulations are carried out to evaluate the performance of the proposed NMPC-based strategy for different scenarios validating the theoretical results. From Chapters 2 and 3, we observe that the NMPC-based solution offers robustness to the different unknown attacker models and has better performance than the CLOS and A-CLOS based strategies. With the help of escape zone maps, it is now possible to identify the outcome of the games beforehand. Also, the experimental results proved that the proposed online-feedback scheme is a suitable alternative to conventional optimal control techniques for real-world scenarios. In Chapter 4, we solve a joint cooperative localization and path planning problem for a group of autonomous aerial vehicles (AAVs) using nonlinear model predictive control (NMPC). The vehicles do not have access to global satellite navigation systems (GNSS). A moving horizon estimator (MHE) is used to estimate the states with the help of relative bearing information to known landmarks and other vehicles. The goal of NMPC is to devise optimal paths for each vehicle between a given source and destination while maintaining desired localization accuracy. The localization error covariance of the vehicles for the NMPC prediction window was calculated using an approximate analytical expression based on the relation between the covariance and path lengths to the landmarks. We show that a ve- hicle’s position accuracy is inversely proportional to the path length to the landmarks. The use of this analytical expression reduces the computation requirement of NMPC compared to the traditional method of propagating and estimating covariances using an extended Kalman filter (EKF). Finally, we present numerical simulation results that validate the proposed approach for different numbers of vehicles and landmark configurations. The proposed framework is useful in the area of urban mobility, where many v autonomous aerial vehicles fly through urban areas with buildings and other structures performing tasks such as cargo delivery. The proposed framework allows these AAVs to localize themselves cooperatively in the absence of GNSS signals since, in urban canyons, the accuracy of GNSS is affected by phenomena such as multipath. In Chapter 5, we introduce a learning-based nonlinear model predictive control (L-NMPC) scheme for the iterative task of filming race cars using gimbaled cameras mounted on autonomous aerial vehicles (AAVs). The controller is capable of avoiding inter-vehicle collisions and the environmental obstacles that block the path of the AAVs. It also ensures that the cars always lie in the field of views (FOVs) of the cameras while satisfying the control and state constraints. The controller is able to learn from the previous iterations and improve the tracking performance with the help of reinforcement learning (RL). Simulation results are given to demonstrate the efficacy of the proposed learning-based control scheme. The proposed scheme helps reduce manual effort in tuning weights for the cost components of the NMPC. Also, with the help of RL-tuned weights, the NMPC scheme gives tight tracking of the cars even in environments containing obstacles. Finally, in Chapter 6, we conclude the thesis by providing the inferences from the various experi- ments and simulations for different problems discussed in Chapters 2-5. The main extensions for the NMPC framework discussed in the thesis are to broaden the framework to three dimensions taking the terrain map and urban environments into account. The current NMPC solver suffers from a high value of computation time for such complex environments with a large number of agents. Hence, for real- time implementation, we would like to drive our approach toward the use of fast-MPC based solvers for speeding up the computations. In the future, we would like to use the actual dynamics for the agents instead of point-mass models. Also, environmental challenges like wind disturbances could be taken into effect when formulating the control law for the agents. A detailed description of future directions for each problem is given at the end of this chapter. In summary, this thesis focuses on developing solutions for different applications – pursuit- evasion games, path planning under uncertainty, and cinematography using the NMPC framework. We analyze the theoretical properties of the proposed solution and show the performance through simulations and experiments.