Learning algorithms for multi-tasks of unconstrained fingerprint recognition

Abstract

Attributed to extensive acceptance and research, lives can and inked fingerprints are successfully used for human recognition. However, applications such as contactless biometrics amidst the pandemic and crime scene identification call for newer covariates of fingerprints. These new unconstrained/semi-constrained covariates consist of contactless fingerprint1, multi-view contactless fingerprints (contactless 3D or finger videos), and latent fingerprints. These fingerprints are prone to acquisition variations, resulting in a non-optimal performance with traditional fingerprint algorithms. These unconstrained fingerprints require dedicated algorithms that can yield state-of-the-art performance under different challenges. Hence, this thesis presents a six-fold contribution of recognition using semi/un-constrained fingerprints (finger photos, finger videos, and latent fingerprints). Structured in two parts, the first part discusses aspects of contactless fingerprints, while the second focuses on latent fingerprints. The thesis provides large-scale databases for different applications of unconstrained fingerprints. Using the databases, dedicated end-to-end recognition algorithms are presented for contactless fingerprints, fingervideos, and latent fingerprints. The recognition algorithm for latent fingerprints is further backed by scientific learning of the process followed by forensic examiners. The details of each of the contributions are listed below. The first three contributions in Part I of the thesis are towards contactless fingerprint recognition. As the first contribution, we create databases to establish large-scale contactless fingerprint recognition under various acquisition challenges and vulnerabilities. Primarily, the proposed IIIT-D Smart Phone Finger-selfie Database v2 (ISPFD-v2) accounts for 19,456 images in total. These databases understand the behaviour of finger selfie recognition under various vulnerabilities and challenges. As the second contribution, we present a novel algorithm for matching fingerselfies that performs multiple sub-tasks associated with end-to-end finger-selfie matching. The algorithm comprises of segmentation and enhancement methods to handle illumination, background, and resolution variations. A discriminative feature representation is extracted from the pre-processed finger-selfie using Deep Scattering Network (DSN). The DSN features show their efficacy towards recognition when matched against a gallery of Livescan and finger selfies. We also understand the behavior of finger-selfie recognition under vulnerabilities of spoofing, highly unconstrained acquisition, and uninformed recognition using social-media posted finger-selfies. Third, the Multi-Resolution Frame Selection Algorithm (MultiRes) is presented, which performs efficient frame selection from a finger video. The proposed MultiRes algorithm processes a finger-video stream frame-wise at four different spatial resolutions: 3⇥3, 5⇥5, 7⇥7, and 13⇥13. Processing at different spatial resolutions, the frame selection becomes invariant to varying camera resolution and finger scaling (as introduced by various movements). The subsequent three contributions in Part II of the thesis are towards latent fin1contactless fingerprint, finger photos, and finger selfies are used synonymously throughout this thesis. fingerprint recognition. As the fourth contribution, we create the Multi-Surface Multi Technique (MUST) Latent Fingerprint Database. The MUST Latent Fingerprint Database has nearly 21,000 impressions, 16,000 of which are latent fingerprints. The latent fingerprints span over 35 different surfaces, development techniques, and corresponding exemplar fingerprints (live scan, inked, and rolled). The fifth contribution aims to understand and infer the manual latent fingerprint comparison process followed by forensic examiners. Latent fingerprint examiners perform multiple tasks such as comparison, determining value, and annotation of minutiae during manual comparison of impressions. However, different examiners discern fingerprint details differently based on their proficiency and ability to perceive details. We infer perceptual behavior by collecting examiners’ eye gaze while they mark minutiae and perform a comparison. We empirically find examiners’ Region of Interest (ROI) on the impressions using the eye gaze fixation to show the patterns observed across different forensic examiners and understand their search strategies using ROI. The strategy can be incorporated into an automated system to improve comparisons and help train novice examiners. As the sixth and the last contribution, we achieve efficient joint optimization of varied multi-tasks associated with latent fingerprint recognition. Using the MUST database, the algorithm jointly solves exemplar and latent fingerprint segmentation, exemplar, and latent fingerprint orientation classification, and recognizes latent fingerprints against fingerprints. With the varying nature of the tasks, the performance of a subset of tasks may get compromised (known as negative transfer). We propose a Dropped-Scheduled Task (DST) algorithm, which probabilistically “drops” specific tasks during joint optimization while scheduling others to reduce negative transfer. The DST algorithm shows the minimum negative transfer and overall least errors. Hence, this thesis understands the challenges of semi/un-constrained fingerprint recognition. With challenging databases and optimized algorithms backed by real-world use case scenarios, the thesis improves automated algorithms’ performance for finger-selfies, finger videos, and latent fingerprints.