Generative and adversarial learning for object recognition

Abstract

Generative modeling and adversarial learning have significantly advanced the field of computer vision, particularly in object recognition and synthesis, unsupervised domain adaptation, and adversarial attacks and defenses. These techniques have enabled the creation of more accurate and robust models for critical applications. In particular, we develop algorithms for fine-grained object recognition (Re-ID) and classification tasks. Re-ID involves matching objects across non-overlapping cameras, which is challenging due to visual recognition hurdles like pose change, occlusion, illumination variation, low resolution, and modality differences. On the other hand, object classification is another aim to categorize input data into pre-defined classes, using patterns learned from training data. In this context, our thesis is motivated by the potential of generative modelling to synthesize novel human views, which can be used for unsupervised learning of Re ID models. Unsupervised Re-ID suffers from domain discrepancies between labeled source and unlabeled target domains. Existing methods adapt the model using aug mented samples, either by translating source samples or assigning pseudo labels to the target. However, translation methods may lose identity details, while label assignment may give noisy labels. Our approach is distinct from other methods in that it decou ples the ID and non-ID features in a cyclic manner, which promotes better adaptation to pose and background, thereby resulting in richer novel views. This approach could improve the accuracy of Re-ID models for the unlabeled target domain, thus enhancing their robustness in real-world settings. Furthermore, we aim to analyze the robustness of Re-ID and classification models and propose adversarial attack and defense methods to enhance their reliability. Adver sarial attacks are a malicious technique that manipulates input data to cause machine learning models to make incorrect predictions or classifications. Adversarial defense methods, including adversarial training, certified defense, and detection mechanisms, are used to protect models from such attacks. By integrating adversarial attack and de fense methods into model development and deployment, the risk of incorrect Re-ID and ii misclassification can be minimized, leading to robust models. This is especially impor tant in critical applications such as surveillance and security systems. Our thesis aims to propose adversarial attack and defense mechanisms for Re-ID models and certify the robustness of classification models in both white-box and black-box settings. Specifically, we address the limitations of conventional adversaries that consider Euclidean space and ignore the geometry of the pixels. We propose a stronger at tack by incorporating geometry using the Wasserstein metric attack. To defend against such adversarial attacks, we propose a stochastic neural network that uses isotropic and anisotropic Gaussian noise to parameterize stochasticity. These parameters are learned under a meta-learning framework to make our defense more effective and scalable. Finally, in order to provide a provable guarantee of a black-box model robustness, we propose a certified black-box defense via zeroth-order (ZO) optimization for image classification tasks. Previous works suffer from high model variance and low perfor mance on high-dimensional datasets due to inadequate denoiser design and limited uti lization of ZO techniques. To address these limitations, we introduce a robust UNet denoiser (RDUNet). RDUNet enables the model to learn intricate details while main taining low reconstruction error, surpassing the performance of previously developed custom-trained denoisers. We extensively evaluate our proposed generative and adversarial techniques using publicly available Re-ID and classification datasets - Market-1501, DukeMTMC-ReID, MSMT17, CUHK03, Veri-776, CIFAR-10, CIFAR-100, STL-10, Tiny Imagenet, and MNIST.