A comparative study of correlation measures for the application of anomaly detection in data streams

Abstract

We study two measures namely maximal information coefficient (MIC) and distance correlation (dCor) for the application anomaly detection. MIC is based on the concept of information theory and measures the mutual information between two variables. dCor uses distances between observations for its calculation. Both of these measures are considered better than the classical Pearson Correlation coefficient that has been the main measure for many decades. The main reported issue with the Pearson correlation measure is to not guaranteeing independence between variables even in the case of coefficient value being zero. Furthermore, it is very sensitive to outliers and assumes continuous normally distributed data for its working. We experimentally study the efficacy of MIC and dCor measures over energy data for different experimental settings, i.e. windows size, overlapping threshold and anomaly threshold. We observe that MIC consistently outperforms dCor.