Airborne visual tracking and re-identification & obstacle avoidance using deep reinforcement learning for UAV's

Abstract

Visual tracking and person re-identification has gathered a lot of attention in computer vision due to its challenging nature and importance in surveillance applications. In this work, we explore the on-board visual tracking and re-identification of person from moving camera such as a drone. We first detect the humans in the scene, then re-identify potential target and then track. In order to achieve real time tracking, we use Kernelized Correlation Filter (KCF) tracker. To perform long term tracking, we apply re-identfiication algorithm using deep 3DConvNet. The 3D ConvNet model is reliable in learning the spatio-temporal features with high discriminative power. We develop an android application to realize this system. The application receives frames as input in real time and transmits roll, pitch and vertical upthrust instructions to drone to follow the object being tracked.In addition, our system can handshake with multiple drones and transfer the tasks between them.Our system is fully autonomous In an extreme surveillance scenario, we need obstacle avoidances for drone following the target.To this end, we detect an obstacle and perform actions to avoid it by using deep reinforcement learning. (The algorithm is currently being trained and we expect the results to be obtained before the BTP presentation