Wavelet transform learning and applications

Abstract

Transform learning (TL) is currently an active research area. It has been explored in several applications including image/video denoising, compressed sensing (CS) of magnetic resonance images (MRI), etc. and is observed to perform better than the existing transforms. However, TL involves non-convex optimization problem with no closed form solution and hence, is solved using greedy algorithms. A large number of variables (transform basis as well as transform coefficients) along with the greedy-based solution makes TL computationally expensive. Also, TL requires a large amount of training data for learning. Hence, it may run with challenges in applications where only single snapshots of short-duration signals such as speech, music or electrocardiogram (ECG) signal are available. Thus, one uses existing transforms that are signal independent. This motivates us to look for a strategy to learn transform in such applications.

Among existing transforms, discrete wavelet transform provides an efficient representation for a variety of multi-dimensional signals. Owing to this, wavelets have been applied successfully in many applications. In addition, wavelet analysis provides an option to choose among existing basis or to learn new basis. This motivates us to learn wavelet transform from a given signal of interest that may perform better than the fixed transforms in an application. The learned wavelet transform is, hereby, called signal-matched wavelet transform. Since the translates of the wavelet filters associated with discrete wavelet transform form the basis in l2-space, wavelet transform learning implies learning wavelet filter coefficients. This reduces the number of parameters required to be learned with wavelet learning compared to the traditional transform learning. Also, the requirement of learning fewer coefficients allows one to learn basis from a short single snapshot of signal or from the small training data. We also show that closed form solution exists for learning the wavelet transform unlike traditional transform learning.

Although the problem of signal-matched wavelet design/learning has been explored in the literature, there are a number of limitations. Firstly, existing methods require full original signal to learn wavelet transforms and hence, these methods cannot be used in inverse problems, where one has access to only the degraded signal and not to the original signal. Secondly, signal-matched wavelet transform learning is not explored for rational wavelets, although rational wavelets are observed to be more effective than dyadic wavelets in audio and speech signal processing. Thus, we note that there is a need for methods to learn signal-matched wavelets that are modular, have compactly supported filters for dyadic or rational wavelet systems, are easily implementable in DSP hardware, and can also be learned from degraded signals. This thesis is motivated to address these limitations and proposes a number of methods along with their utility in applications.

Specifically, we propose methods to learn dyadic as well as rational wavelet transform using the lifting framework. The proposed method inherits all the advantages of lifting, i.e., the learned wavelet transform is always invertible, method is modular, learned transform has compactly supported filters and hence, is DSP hardware friendly, and the corresponding wavelet system can also incorporate nonlinear filters, if required. We show that closed form solution exists for learning the wavelet transform with the proposed method. Also, wavelet transform can be learned using the proposed method even when a small amount of data is present. Since the wavelet transform is being learned from the signal itself, one may use the learned wavelet transform in applications instead of struggling to choose from the existing wavelet bases.

For dyadic wavelet transform learning (DWTL), we propose three methods in different scenarios. Particularly, we propose methods to learn dyadic wavelet transform (DWT) from 1) original signal, 2) degraded signal in inverse problems, and 3) a class of signals. We use the learned DWT as the sparsifying transform in the application of 1) Gaussian denoising of speech and music signals, 2) CS based reconstruction of speech, music, and ECG signals, 3) impulse denoising of images, and 4) CS based reconstruction of images. Extensive simulations have been carried out that demonstrate that the learned transforms outperform the standard dyadic wavelet transforms.

We also extend the existing theory of lifting framework from dyadic to rational wavelets and use the extended lifting theory to learn critically sampled signal-matched rational wavelet transform (RWT) with generic decimation ratios from a given signal of interest. We introduce the concept of rate converters in predict and update stages to handle variable subband sample rates. So far, signal-matched rational wavelet learning have remained limited in use because design methods are in general cumbersome. Since our proposed methodology exploits lifting framework, we provide modular, compactly supported, DSP hardware friendly rational wavelet transform learning (RWTL) methods. This may enhance the use of RWT in applications which is so far restricted. We use the learned RWT as the sparsifying transform in CS based reconstruction of 1-D and 2-D signals. The learned RWT is observed to perform better than the existing dyadic as well as rational wavelet transforms.

Apart from the wavelet transform learning methods, we propose a new multilevel wavelet decomposition strategy for images, named as L-Pyramid wavelet decomposition. L-Pyramid wavelet decomposition is observed to perform better in CS based image reconstruction. In addition, we also propose weighted non-convex minimization for CS based recovery. Detailed experiments are provided using the weighted non-convex minimization and the learned wavelet transform for CS based ECG signal recovery with various sensing matrices. The learned wavelet transform along with the proposed weighted non-convex minimization method is observed to provide much better ECG signal reconstruction as compared to existing wavelet transforms as well as existing methods.