Dictionary based time series modelling

Abstract

Time series analytics is the practice of determining future values of correlated signals. In seminal works, time series were modeled using classical techniques such as ARMA (autoregressive moving average), and its variants ARIMA (auto-regressive integrated moving average). ARMA and its variant models often fail to perform well with non-stationary time series data. State Space Models (SSMs) have risen in the last few decades because they can overcome drawbacks of ARMA systems and provides an uncertainty quantification, which is crucial in time series point estimates and gives better-informed decisions. But SSMs are well suited for applications where the structure of the time series is known and understood in advance. They require the incorporation of structural and statistical information on the model. Real-time problems involve noisy samples with diverse sources; it can become difficult for SSMs to assume the model’s structural details in advance. In the last decade, various machine learning and deep learning techniques have also gained attention in solving time series forecasting problems. The structured neural network models, namely recurrent neural network (RNN) and long short-term memory (LSTM), 1D-CNN, DeepAR, TFT, N-Beats, MFNN are now considered as state-of-the-art in the resolution of stock forecasting problems, due to their inherent ability for processing varying length sequences and predicting future trends with no structural assumption in advance. It is worth mentioning that the deep learning models require a rather large dataset to learn parametric functions to forecast efficiently for unseen data. Moreover, those techniques usually provide pointwise estimates without any measure of uncertainty. Both approaches have their benefits but lack in one or other essential aspects to dynamically model the time-series signals. This thesis has two main objectives: 1) Propose more efficient algorithms to forecast future time stamp signals dynamically, requiring no prior explicit information on model parameters and a less data-hungry approach. 2) Estimate uncertainty score for each future pointwise prediction for a time-series signal. The work focuses on devising efficient implementation strategies for practical use of the method in the context of stock market time series analysis. The Thesis will walk you through the investigation conducted based on experiments on actual financial data of the performance of the novel proposed approaches. The first contribution proposes new modeling and inferential tool for dynami-cal processing of time series. The approach is called recurrent dictionary learning(RDL). The proposed model reads as a linear Gaussian Markovian state-space model involving two linear operators, the state evolution and the observation matrices that we assumed unknown. These two unknown operators (that can be seen interpreted as dictionaries) and the sequence of hidden states are jointly learnt via an expectation-maximization algorithm. The RDL model gathers several advantages: online processing, probabilistic inference, and a high model expressiveness, which is usually typical of neural networks. RDL is particularly well suited for stock forecasting. Its performance is illustrated on two problems:next-day forecasting (regression problem) and next-day trading (classification problem), given past stock market observations. Second, we propose sequential transform learning (STL). The proposed work is a linear Gaussian Markovian state-space model involving state evolution, observation matrices, and an exogenous control input. The resultant formulation resembles loosely based on transform learning. The proposed work is made recurrent, introducing a feedback loop where learnt transform coefficients for the t th instant is fed back along with the t+1 st sample. Furthermore, the formulation is made supervised by the label consistency cost. The approach differs from RDL due to presence of an exogenous input, which helps to establish more informed prior for state-space evolution. Our approach keeps the best of two worlds, marrying the interpretability and uncertainty measure of signal processing with the function approximation ability of neural networks. We have carried out experiments on one of the most challenging problems in dynamical modeling –stock forecasting. Third, we propose Deep recurrent dictionary learning (DRDL). The proposed work is developed to cater to bottlenecks experienced in recurrent dictionary learning approach. The work overcomes the limitations of RDL and also dives into multi-linear Gaussian state space. In this work, we combine the benefits of both approaches(signal processing and neural network) by introducing a multi-linear Gaussian SSM whose state and evolution operators can be learnt from the data. We propose factorized forms for the state and evolution operators to cope with possible non-linearity in the observed data and the hidden state. We also introduced expectation-maximization method combined with an alternating block strategy to estimate each involved factor while jointly performing the state inference, generalizing our previous work (Recurrent Dictionary Learning). Fourth, we propose Deep sequential transform learning. The proposed method is a deep network to model multi-linear Gaussian state space in the presence of an exogenous input. In this work, we propose to model non-linearity using a deep factor model. Thus the proposed approach is a multi-linear Gaussian state space involving state evolution, observation matrices, and an exogenous control input modeled as a deep latent factor model. The model is also made recurrent by introducing a feedback loop where learnt deep factor model parameters for the t th instant is fed back along with the t + 1 st sample. The method is developed to overcome the limitations of the Sequential transform learning model and explore the multi-linear Gaussian state-space model in the presence of exogenous input, which differentiates it from the DRDL approach. The method keeps the best of both worlds – the interpretability and ability of SSMs to yield uncertainty estimates with the flexibility of RNNs to learn the underlying operators from the data. This work takes up one of the most challenging problems of today’s age –predicting the prices of cryptocurrencies. The motive of the work presented in this thesis is to propose efficient al-gorithms to forecast future time stamp signals dynamically, simultaneously estimating uncertainty score with each pointwise prediction to offer a more informed prediction for future time-series signals.