Face detection and verication in unconstrained videos: challenges, detection, and benchmark evaluation

Abstract

With increasing security concerns, surveillance cameras are playing an important role in the society and face recognition in crowd is gaining more importance than ever. For video face recognition, researchers have primarily focused on controlled environments with a single person in a frame. However, in real world surveillance situations, the environment is unconstrained and the videos are likely to record multiple people within the eld of view. Surveillance videos encompass multiple challenges for face detection and face recognition. For instance, detection algorithms may be a ected due to size of a face image, occlusion, pose, illumination, and background while recognition algorithms may be a ected due to low resolution, occlusion, pose, illumination, and blurriness. State-of-the-art approaches for both face detection and face recognition in such challenging scenarios are currently in nascent stages. Moreover, due to the unavailability of such databases, it is difficult for researchers to pursue this important challenge. This thesis attempts to ll the gap in unconstrained face recognition in two ways:develop a large unconstrained video face database, and create a benchmark protocol and perform baseline experiments for both face detection and recognition. As the rst contribution of this thesis, a large video database of 384 videos consisting of 258 subjects is prepared. Each video generally contains multiple subjects in unconstrained settings. Further, ground truth for face and landmark (eye and mouth) detection is manually annotated. As the second contribution of this thesis, we design a benchmark protocol for face detection and recognition evaluation. Using the protocols, we evaluate existing face detection and face recognition approaches, including commercial systems. Poor face detection and veri cation results showcase the challenging nature of the problem and the database.