Deep clustering

Abstract

The traditional way of clustering is first extracting the feature vectors according to domain-specific knowledge and then employing a clustering algorithm on the extracted features. Deep learning approaches attempt to combine feature learning and clustering into a unified framework which can directly cluster original images with even higher performance. Therefore, deep clustering approaches rely on deep neural networks for learning high-level representations for clustering. Auto-encoders are a special instance of deep neural networks which are able to learn representations in a fully unsupervised way. Majority of the prior works on deep clustering are based on auto-encoder framework where the clustering loss is embedded into the deepest layer of an auto-encoder. The problem with auto-encoder is that they require training an encoder and a decoder network. The clustering loss is incorporated after the encoder network; the decoder network is not relevant for clustering. The need of learning an encoder and a decoder network leads to learning twice the number of parameters as that of a standard neural network. This may lead to overfitting especially in the cases where the number of data instances are limited. Moreover, the current state-of-the-art deep clustering approaches are not able to capture the discriminative information in the learned representations due to the lack of supervision [1]. To alleviate the aforementioned problems, we have proposed deep clustering approaches based on Dictionary Learning, Transform Learning, and Convolutional Transform Learning (CTL) frameworks. We have embedded two popular clustering algorithms – K-means clustering and Sparse Subspace clustering. The limitation of unsupervised learning in existing deep clustering approaches is mitigated by incorporating contrastive learning in CTL framework. The proposed deep clustering approaches are evaluated using datasets from multiple domains including computer vision, hyperspectral imaging, text and multiview datasets. The results demonstrate the superiority of the proposed approaches over the current state-of-the-art deep clustering approaches.