Deep dictionary learning

Abstract

This Thesis focuses on combining the two well researched concepts of representation learning – Dictionary Learning and Deep Learning. These two learning paradigms have been known for long. Ever since, plethora of papers have been published for both the paradigms for solving inverse problems and for prediction problems. While dictionary learning focuses on learning “basis’’ and “features’’ by matrix factorization, deep learning focuses on extracting features via learning “weights’’ or “filter’’ in a greedy layer by layer fashion. While dictionary learning is shallow learning, deep learning methods are data hungry. This Thesis merges these two learning methodologies and proposes a new learning method referred to as Deep Dictionary Learning which tries to eliminate the disadvantages of the two. The proposed learning method learns multiple levels of representations using dictionary learning that correspond to different levels of abstraction; the levels form a hierarchy of linear/non-linear transformations. On the above described strategy, two deep dictionary learning methods have been proposed in this thesis. Algorithm 1 is the deep dictionary learning method while Algorithm 2 is the robust version of algorithm 1, referred to as Robust Deep Dictionary Learning. Algorithm 2 consists of an additional denoising layer at the top most level for the purpose of generating useful representations even with noisy data. The proposed techniques are compared with other learning approaches such as Stacked Auto Encoder, Deep Belief Network, LCKSVD1 and LCKSVD2 on benchmark datasets in the presence and absence of impulse noise of varying amounts. The classification performance of the proposed technique achieves higher or at par accuracies for both the cases. On a real world problem of hyperspectral image classification, it is observed that the proposed deep dictionary learning methods performs at par with other learning techniques with much less complexity and time in both, the absence and the presence of shot noise. Also, the effect of going deep in dictionary learning versus shallow learning is discussed and shown experimentally.