Neural Puppet: Generative Layered Cartoon Characters

Winter Conference on Applications of Computer Vision (WACV'20)

Published March 1, 2020

Omid Poursaeed, Vladimir (Vova) Kim, Eli Shechtman, Jun Saito, Serge Belongie, Eli Shechtman

We propose a learning based method for generating new animations of a cartoon character given a few example images. We express pose changes as a deformation of a layered 2.5D template mesh, and devise a novel architecture that learns to predict mesh deformations matching the template to a target image. In addition to coarse poses, character appearance also varies due to shading, out-of-plane motions, and artistic effects. We capture these subtle changes by applying an image translation network to refine the mesh rendering. Our generative model can be used to synthesize in-between frames and to create data-driven deformation. Our template fitting procedure outperforms state-of-the-art generic techniques for detecting image correspondences.

Learn More