Cross-spectral cross-resolution face recognition in videos

Abstract

One of the challenging applications in face recognition is video surveillance, where unconstrained low-resolution video data is captured both in day and night time (visible and near-infrared) with multiple subjects in frames, which are matched with high resolution gallery images. Due to the lack of an existing database for such a cross spectral cross resolution video-to-still face recognition application, this still remains an open research problem. The aim of this research is to come up with robust face recognition algorithms in surveillance scenarios. For this, we first present a video database which can be utilized to benchmark face recognition algorithms addressing cross- spectral and cross-resolution matching. The proposed Cross-Spectral Cross-Resolution Video dataset version 2 (CSCRV-v2) contains 460 videos pertaining to 252 subjects with an open-set protocol. We then focus on the first step in the face recognition pipeline i.e., face detection and propose an algorithm – Face Finder. Face Finder addresses shortcomings (like high false positive rate) of existing face detectors by making use of human body segmentation results of a trained Convolutional Neural Network model specifically designed for semantic segmentation. Experimental results on CSCRV-v2, are compared with that of two off-the-shelf face detection algorithms to show the efficacy of the proposed algorithm. We then focus on face recognition and present results with two commercial matchers for two experimental scenarios on the data. It is our assertion that our research will further facilitate the research community to develop robust face recognition algorithms to handle real world surveillance scenarios