Robust autoencoders

Abstract

An autoencoder is an artificial neural network used for learning efficient codings. The aim of an autoencoder is to learn a representation of data, which can then be used for better classification or any such application. Recently, the autoencoder is being used for learning generative models of data. Usually the Euclidean distance (l2-norm) is used as data fidelity constraint for training the autoencoder. The Euclidean distance is optimal when the noise is normally distributed but not so in the presence of outliers. In inverse problems like impulse denoising and de-aliasing, the noise/artifact is sparse but large. Hence a more robust data fidelity cost function is desired. So we use the lp-norm asit makes the problem robust to outliers. Both linear and non-linear activation functions are used at the output end of autoencoder. The linear problem is solved using alternating minimization and Iterative Reweighted Least Squares (IRLS). The non-linear formulation is solved using split-Bregman technique. Experimental results show that our proposed method yields considerably better results for impulse denoising and de-aliasing compared to previous formulations. Also, stacked linear autoencoder gives a representation for images which significantly improves the results for classification problem even using simple classification techniques like KNN or SVM. This performs much better than complex methods like deep belief networks or non-linear autoencoders. Yet another application of this robust autoencoder is for inverse problems like signal recovery, especially for sparse signals. This produces results almost at par with compressed sensing techniques like Basis Pursuit, Lasso or Iterative Soft-thresholding and is also much faster.