Deep transform learning

Abstract

Conventional dictionary learning is a synthesis formulation; it learns a dictionary to generate/synthesize the data from the learned coefficients. Transform learning is its analysis equivalent. The transform analyzes the data to generate the coefficients. Dictionary learning had been popular in both signal processing and machine learning communities. However, transform learning is largely unknown outside the signal processing research community. So far, transform learning has been primarily used for solving inverse problems.

The objective of the thesis is to build a completely new machine learning framework out of transform learning. It has already been shown how the basic transform learning has been used as an unsupervised feature extraction tool.

This work aims at proposing a supervised version of transform learning with a plug-and-play approach. The supervised version is general enough to perform classification without the need for any external classifier. The kernelized version of supervised transform learning and stochastic regularization on transform learning are also proposed. Based on the proposed supervised transform learning framework, problems on computer vision, bioinformatics, hyperspectral image classification, and arrhythmia classification are solved.

This work also focuses on an unsupervised greedy deep transform learning problem, where each of the layers was solved separately. This was a solution for unsupervised feature extraction using deep transform learning. But the greedy solution for deep transform learning was sub-optimal. Then work has been done on proposing an optimal solution to learn all the layers jointly. It was used to solve classification, clustering and inverse problems.

Another problem discussed in this work is the supervised version of deep transform learning. The supervised version is general enough to perform single-label classification and multi-label classification. Proposed supervised deep transform learning for multi-label classification has been used for solving a practical problem of non-intrusive load monitoring. Another contribution of this work is to propose a deeply transformed subspace clustering framework. In this work, two techniques are introduced: transformed locally linear manifold clustering and transformed sparse subspace clustering. Next, a deeper architecture for the same is proposed.

Then, the idea of convolutional transform learning is introduced. Here, a set of independent convolutional filters are learned that operate on the images to produce representations (one corresponding to each filter). The kernels learned from this method have a close relationship with that of convolutional neural networks.

Finally, a semi-coupled transform learning framework is introduced. Given training data in two domains (source and target), it learns a transform in each of the domains such that the corresponding coefficients are (linearly) mapped from the source to the target. Since the mapping is in one direction (source to target) but not the other way round, It is called semi-coupled. This work is the analysis equivalent of (semi) coupled dictionary learning.