Video inpainting detection using inconsistencies in optical flow

Abstract

In recent years due to advancement in video and image editing tools it has become increasingly easy to modify the multimedia content. The doctored videos are very di cult to identify through visual examination as artifacts left behind by processing steps are subtle and cannot be easily captured visually. Therefore, the integrity of digital videos can no longer be taken for granted and these are not readily acceptable as a proof-of-evidence in court-of-law. Hence, identifying the authenticity of videos has become an important eld of information security. In this thesis work, we present a novel approach to detect and temporally localize video inpainting forgery based on optical ow consistency. The pro- posed algorithm comprises of two stages. In the first step, we detect if the given video is inpainted or authentic and in the second step we perform tem- poral localization. Towards this, we rst compute the optical ow between frames. Further, we analyze the goodness of t of chi-square values obtained from optical ow histograms using a Guassian mixture model. A threshold is then applied to classify between authentic and inpainted videos. In the next step, we extract Transition Probability Matrices (TPMs) by modelling the optical ow as rst order Markov process. SVM based classi cation is then applied on the obtained TPM features to decide whether a block of non-overlapping frames is authentic or inpainted thus obtaining temporal localization. In order to evaluate the robustness of the proposed algorithm, we perform the experiments against two popular and e cient inpainting techniques. We test our algorithm on public datasets like PETS and SULFA. The results show that the approach is e ective against the inpainting techniques. In addition, it detects and localizes the inpainted frames in a video with high accuracy and low false positives.