Analysis dictionary learning for classification

Abstract

Data classification is the core of leading technologies today. With the explosion of data through mobility and growth of the Internet, analysis and classification of data is the immediate process after acquisition. Most of the information is hidden within the acquired data and needs to be extracted for further processing. Thus, feature extraction is an important pre-processing task to classification. Till recently features were hand crafted which though accurate, were time consuming to generate and required human intervention. With the explosion of deep learning (since 2006), automatic feature generation has become the norm for most applications. Algorithms learn from the data and generate the required features adapted to various tasks, such as classification, reconstruction, denoising, sentiment analysis, data mining and the like. The most popular algorithms for automatic feature generation are Deep Belief Networks, Autoencoders and Convolutional Neural Networks(CNN) which also provide classification capabilities. All these algorithms and architectures derive motivation from the fact that human visual and audio cortex are compositional in nature and activate at various level of abstraction. Similarly, the aforementioned machines learn from raw data at multiple levels of abstraction with growing complexity. Such algorithms have proven to be very efficient and have also provided various benchmark tools and applications in the industry like Google Photos and Facebook's deep Face. However, the major limitation to such tools are their enormous training times and humongous amount of data that is required to train them. In this thesis, a computationally simpler model for generating features through an Analysis Dictionary Learning approach is presented. In contrast with the synthesis dictionary learning approaches where the features are generated by solving an inverse problem via an iterative procedure, the analysis approach has the advantage of generating features from the data with minimal preprocessing by directly operating the data with the dictionary. The dictionary operates on the data and generates features, hence the framework is named as the analysis framework. Significant improvement in test time feature generation is obtained as compared to other dictionary learning methods. Also, the learning procedure is computationally inexpensive and is flexible as any prior knowledge can easily be incorporated into the framework depending on the task the learnt features will be put to. To prove the versatility of the framework, the proposed approach is applied to various real world scenarios like digit recognition, speech recognition and Non Intrusive Load Monitoring (NILM). We have been able to establish state of the art results under varied conditions comparable to more complex deep learning techniques.