Harnessing auxiliary information : new methods to improve person identification

Abstract

Large scale biometric identification systems still lack the versatility to handle challenging situations such as adverse imaging conditions, missing or corrupt data, and non-conventional operating scenarios. It is well understood that in different operating conditions, evidence of identity obtained from different sources is disparate. In such cases, additional ‘situational’ cues can be utilized to improve the performance and robustness. The primary emphasis of this thesis is the formulation of new methods to utilize situational cues such as quality of input biometric samples, social cues of co-occurrence, and other background information towards more inclusive biometric systems. Biometric sample quality assessment during capture and its integration into the recognition system improves performance and reduces the failure-to-enroll rates. The first contribution of this thesis is an in-depth survey along with statistical evaluation of different concepts and interpretations of biometric quality in multiple biometric modalities. The thesis also investigates the effectiveness of holistic representations of faces for classifying them into different quality categories that are derived from matching performance. The experiments on the CASPEAL and SCFace databases containing covariates such as illumination, expression, pose, low-resolution, and occlusion, suggest that the representations can efficiently classify input face images into relevant quality categories and be utilized in face recognition systems. An assessment based quality enhancement framework is also presented that showcases the effectiveness of quality assessment metrics for parameter selection in a denoising method to enhance performance and reduce computational time. Multi-modal biometric recognition systems combine evidence from multiple sources of information for improving the recognition performance. Existing multi-modal biometric recognition techniques are, however, unable to provide required levels of accuracy in uncontrolled noisy capture environments. Such algorithms do not adequately scale to variations in data distribution that occur due to changing deployment conditions. The second contribution of this thesis is an adaptive context switching algorithm coupled with online learning to address both these challenges of multimodal biometrics. The proposed framework uses the quality of input images to dynamically select the best biometric matcher or fusion algorithm to verify the identity of an individual. The proposed algorithm continuously updates the selection process using online learning to address the scalability and accommodate the variations in data distribution. The results on the WVU multimodal database and a large real world multimodal database obtained from a law enforcement agency show the efficacy of the proposed framework. Humans are efficient at recognizing familiar faces even in challenging conditions by deducing social context between individuals in group photos. The identity of the person in a photo, in such cases, is inferred based on other individuals present in the same photo; using the known or deduced social context between them. The third contribution of the thesis is a novel algorithm to utilize co-occurrence of individuals as the social context to improve face recognition. Association rule mining is utilized to infer multi-level social context among subjects from a large repository of social transactions. The results are demonstrated on the G-album and on the real-world SN-collection pertaining to 4675 identities that is prepared for the purpose of this research from a social networking website. An anonymized version of the dataset with match scores from a commercial system is also made available. The results of the proposed approach show that association rules extracted from social context can be used to augment face recognition and improve the identification performance. The availability of a large number of unlabelled images from various sources facilitates semi-supervised approaches to improve the performance and robustness of recognition systems. As the fourth contribution, this thesis introduces a novel learning based approach to face recognition towards an affordable and friendly biometric for newborns. Biometric recognition of newborns is an opportunity for the realization of several useful applications such as improved security against swapping and abduction, accurate census and effective drug delivery. The proposed approach couples learning based encoding method via deep neural networks with a one shot similarity distance metric formulated with an online SVM to match effective features with low semantic gap. To evaluate the approach, the largest publicly available database of 96 newborns is collected from various hospitals to study face recognition and is also made available to other researchers. Several existing face recognition approaches and commercial systems are also evaluated on a common benchmark protocol. The proposed approach provides state-of-the-art identification and verification performance on the newborns database.