FaceStyle / Puppetron: Example-Based Synthesis of Stylized Facial Animations

Jakub Fišer

CTU in Prague, FEE

Ondřej Jamriška

CTU in Prague, FEE

David Simons

Adobe

Eli Shechtman

Adobe Research

Jingwan (Cynthia) Lu

Adobe Research

Paul Asente

Adobe Research

Michal Lukáč

Adobe Research

Daniel Sýkora

CTU in Prague, FEE

Adobe Max 2017 (#ProjectPuppetron)

Siggraph 2017

We introduce a novel approach to example-based stylization of portrait videos that preserves both the subject’s identity and the visual richness of the input style exemplar. Unlike the current state-of-the-art based on neural style transfer [Selim et al. 2016], our method performs non-parametric texture synthesis that retains more of the local textural details of the artistic exemplar and does not suffer from image warping artifacts caused by aligning the style exemplar with the target face. Our method allows the creation of videos with less than full temporal coherence [Ruder et al. 2016]. By introducing a controllable amount of temporal dynamics, it more closely approximates the appearance of real hand-painted animation in which every frame was created independently. We demonstrate the practical utility of the proposed solution on a variety of style exemplars and target videos.

Project page.  Online demo.

FaceStyle / Puppetron in the news:

DigitalArtsOnline “The 7 best highlights from Siggraph 2017

DigitalTrends “Stylized Facial Animation project Turns Video Into Realistic-looking Hand-drawn Art

TheVerge “This face filter technology can turn you into a statue or an oil painting

PopularScience “Adobe is training AI to be a better photo and video editor than you

 

Project Publications

Example-Based Synthesis of Stylized Facial Animations

Fišer, J., Jamriška, O., Simons, D., Shechtman, E., Lu, J., Asente, P., Lukáč, M., Sýkora, D. (Jul. 30, 2017)
ACM Transactions on Graphics (Proc. of SIGGRAPH 2017)