Pushing boundaries of face recognition : adversary, heterogeneity, and scale

Abstract

Due to the unconstrained nature of data capture and non-cooperative subjects, automatic face recognition is still a research challenge for application scenarios such as law enforcement. We observe that challenges of face recognition are broadly rooted into two facets: (1) the non-ideal and possibly adversarial face image samples and (2) the large size and incremental/streaming availability of data. The first facet encompasses various challenges such as intentional or unintentional obfuscation of identity, attempts for spoofing system, user non-cooperation, and large intra-subject variations for heterogeneous face recognition. The second facet caters to challenges arising due to application scenarios such as repeat offender identification and surveillance where the data is either large scale or available incrementally. Along with advancing the face recognition research by addressing the challenges arising from both the aforementioned facets, this dissertation also contributes to the pattern classification research by abstracting the research problems at the classifier level and proposing feature independent solutions to some of the problems.

The first contribution addresses the challenge of face obfuscation due to usage of disguise accessories. We collect and benchmark IIIT In and Beyond Visible Spectrum Face Dataset (I2BVSD) pertaining to 75 subjects, which has various types of disguises applied on different individuals. It has become one of the most used disguise face dataset in the research community. Since disguised facial regions can lead to erroneous identity prediction, a texture based algorithm is designed to differentiate between biometric and non-biometric facial patches. The proposed approach is embedded with local face recognition algorithm to address the challenge of disguise variations. The approach is further enhanced with the use of thermal spectrum imaging. As the second contribution, the dissertation addresses the challenge of heterogeneous face matching scenarios, such as matching a sketch against a mugshot dataset of digital photographs, cross-spectrum, and crossresolution matching, that arise in a wide range of law enforcement scenarios. Heterogeneous Discriminant Analysis (HDA) is designed to encode multi-view heterogeneity in the classifier to obtain a projection space more suitable for matching. Further, to extend the proposed technique for nonlinear projections, formulation of kernel HDA is proposed. Focusing on application such as identification of repeat offenders, as the third contribution, we develop an approach to efficiently update the face recognition engine to incorporate incremental training data. The proposed Incremental Semi-Supervised Discriminant Analysis (ISSDA) provides mechanism to efficiently, in terms of accuracy and training time, update the discriminatory projection directions. The proposed approach capitalizes on offline unlabeled face image data, which is inexpensive to obtain and generally available in abundance. The fourth contribution of this dissertation is focused on designing a face recognition classifier that can be efficiently learned from very large batches of training data. The proposed approach, termed as Subclass Reduced Set Support Vector Machine (SRS-SVM), utilizes the subclass structure of training data to effectively estimate the candidate support vector set. This candidate support vector set facilitates learning of nonlinear Support Vector Machine from large-scale face data in less computation time.