Learning representations for matching fingerprint variants

Abstract

Fingerprint recognition has evolved over the decades, providing innumerable applications for improving the modern day security. Based on the method of capture, fingerprints can be classified into four variants: inked, live-scan, latent, and fingerphoto. Extensive research has been undertaken for inked and live-scanned fingerprints. However, research on latent fingerprints and fingerphoto matching is still in nascent stages. These two capture methodologies are semi-controlled or uncontrolled which pose significant variations in the feature space and therefore warrant further exploration. The key research challenges involved in building an automated system for latent fingerprint and fingerphoto matching are as follows: (i) lack of publicly available large scale datasets with diverse variations to motivate reproducible research, (ii) segmentation of foreground regions from the complex background surface, and (iii) lack of robust feature models to represent the noisy and partial finger ridge information. Currently, there are limited end-to-end automated systems for latent fingerprint and fingerphoto matching. This thesis primarily focuses in contributing towards building a completely automated “lights-out" matching system for these two fingerprint variants. There are four contributions ranging from creating large databases to designing algorithms for segmentation and feature extraction for these two fingerprint variants. First, we create two benchmark datasets with diverse acquisition methods: (i) Multi-sensor Optical and Latent Fingerprint (MOLF) dataset containing 19,200 fingerprint images with large intraclass and capture variations and (ii) IIIT-D SmartPhone FingerPhoto Dataset version 2 (ISPFD-v2) containing 16,800 images from 300 classes captured under different environmental setup. The second contribution is designing an automated latent fingerprint segmentation algorithm that segments the fingerprint regions from background by distinguishing between ridge and non-ridge patterns. Latent fingerprint segmentation is usually affected by the texture of the surface and smudges are introduced during lifting. The proposed learning-based algorithm is generalizable and can accommodate for unseen texture noises. Further, a novel Spectral Image Validation and Verification based metric is proposed to measure the effect of the segmentation algorithm. Third, a minutiae extraction algorithm is proposed as a major contribution towards the “lights-out" latent fingerprint matching. A novel group (or class) sparsity based ℓ2,1 regularization method is proposed to improve the unsupervised features learnt using stacked autoencoders and Restricted Boltzmann Machines. Latent fingerprint minutiae extraction is then posed as a binary classification problem to classify patches as minutia or non-minutia. To the best of our knowledge, this is the first algorithm in literature for automated minutia extraction from latent fingerprints. The fourth contribution is towards fingerphoto recognition, in which a novel end-to-end fingerphoto matching algorithm is proposed that is invariant to the environmental factors such as background noise, illumination variation, and camera resolution. The ridge-valley pattern present in a fingerphoto in not as distinct as VII a fingerprint image, thus making minutia extraction highly noisy. The matching pipeline consists of a segmentation technique to extract the fingerphoto region of interest from varying background, followed by enhancement to neutralize the illumination imbalance and increase the ridge valley contrast. For feature extraction, a deep scattering network based representation is introduced. The resultant fingerphoto features are robust and invariant to environmental variations. By addressing these challenging problems, this thesis improves the understanding and performance of automated matching systems for forensic latent fingerprints and fingerphoto images.