Deep dictionary learning

Abstract

Currently there are three basic frameworks in deep learning - stacked autoencoders (SAE), deep belief network (DBN) and convolutional neural network (CNN); SAE and DBN can be applied to arbitrary inputs but CNN can only be applied to natural signals having local correlations (speech, image, ECG, EEG etc.). I am working on developing a new framework for deep learning – deep dictionary learning (DDL). Just as SAE uses autoencoders as basic units and DBN uses restricted Boltzmann machines, DDL uses dictionaries as the basic unit. In lay man’s terms, DDL is formed by stacking one dictionary after another such that the output (features) from the shallower layer feeds into the next (deeper) layer as input.

The initial work on DDL was a greedy sub-optimal solution, i.e. each of the layers were solved separately. My first work has been on proposing an optimal solution to jointly learn all the layers. This was a solution for unsupervised feature extraction using DDL. Later I worked on a supervised version of deep dictionary learning with a plug-and-play approach. The supervised version is general enough to perform classification, multi-label classification and regression. Proposed supervised deep dictionary learning for multi-label classification has been used for solving a practical problem of Non-Intrusive Load Monitoring (NILM). We also proposed a technique called deep blind compressed sensing which combines the analytic power of deep learning with reconstruction ability of compressed sensing. The objective of this work was to classify biomedical signals from their compressive measures.

Next, we work on Siamese DDL networks. These are usually required for verification problems in biometrics. We applied them on face verification problem in disguise detection and kinship verification. It can also be applied in a variety of other situations. For example in estimating dense depth from an image and sparse depth with a complete learning based approach. Or in problems arising in BP estimation from multiple sources (multi-channel ECG and PPG).

Finally, we worked on deeply coupled dictionary learning. These networks are used to generate a linear mapping between samples of different domains. Existing approaches learn a linear map between the source and target domains. We propose to use deep dictionary learning to solve for complex mapping. Such networks can be used in applications like image denoising, image super-resolution, image reconstruction etc.