Exploring geometric constraints for learning representations for visual data

Abstract

Representation of visual data is a connecting link between the perceptual world and machine based processing. Over the decades, the computer vision community is dedicated to improving these representations, so that it can assist humans in a wide range of applications from medical imaging to visual search and face recognition systems to name a few. In this thesis, we explore geometric constraints to aid in learning representations for various computer vision applications that either have access to only limited amount of labeled training data, abundant unlabeled training data or a combination of two. We investigate two types of geometric constraints: manifold and semantic. The contribution in this thesis can be categorized into two parts based on the geometric constraints used. In the first category, we use the geometry of manifolds. First, we use the geometry of Stiefel manifold to learn a linear transformation of feature representations in the supervised setting, and we show improved generalization in low training-data settings. We also show the same manifold constraint to be effective in the unsupervised learning of disentangled representations, which can help improve the interpretability of deep networks. The third problem is that of defense against adversarial attacks on deep networks. Using the geometry of the Grassmann manifold, we show that our subspace based representations of an adversarially perturbed input sample are sufficiently close to their clean counterparts, and can serve as adefense strategy without the need of any retraining or fine-tuning of the network. In the second category, we make use of semantic constraints and derive a loss term that leverages the statistical manifold, i.e., the space of probability distributions. We apply this loss term in two learning scenarios. First, we use it to combat over-fitting in supervised representation learning in case of limited labeled training data for visual animal biometrics task. We show that it improves the robustness and generalization of the representations for primate face recognition as well tiger re-identification problem. Secondly, we use it for learning cluster able representations in a semi-supervised setting, where it has access to limited labeled data along with abundant unlabeled data. In this thesis, based on the improvements across different applications and settings, we conclude that the geometric information is useful for visual data representation learning regardless of the level of supervision.