AI based sketch animation

Abstract

Animation of hand-drawn sketches can be a challenge for artists with little to no dedicated experience in animation, even when working with reference videos for movement. information. An inaccurate animation can make the final animation look too unnatural. We propose a framework to automate the task of applying animation to an input sketch using an input reference video. Our framework handles significant challenges, such as mapping a skeleton from one 2D silhouette to another, motion extraction from the video, motion transfer, and resolving occlusions. Given a skeleton on either the sketch or a video frame, we first map it to the other. We then track the skeleton vertices of the video across frames using a sparse point tracking model and transfer the motion to the sketch skeleton. This method allows artists to create skeletal animations without having to animate character motion manually.