Adobe Research scientists, in collaboration with researchers from Georgia Tech and UC Berkeley, have taken on the problem of how to generate realistic imagery using deep convolutional networks. The result: Project Scribbler.
Recognizing the limits of prior approaches, the researchers created a new image synthesis architecture to structure their network. Employing machine learning techniques, they then trained the system using tens of thousands of photos, including a diverse range of human faces.
What resulted is a remarkable interactive image generation system. Scribbler takes an input image—either a photo or a grayscale sketch—and quickly generates a full-color output image. The network also allows users to “scribble” over a sketch to indicate a preferred color. It was tested on faces, cars, home interiors, and more.
Scribbler is designed to operate in real time, giving users immediate feedback—which is clear from this Adobe MAX conference Sneak. Jingwan (Cynthia) Lu, research scientist at Adobe Research, demonstrates a great use for Scribbler: it can rapidly and richly colorize grayscale images. She also shows how the system—with cues from the user—can propagate specific colors and textures across an entire sketch.
Contributors:
Jingwan (Cynthia) Lu and Chen Fang, research scientists, and Daichi Ito, senior technical artist (Adobe Research)
Patsorn Sangkloy, Wenqi Xian, Amit Raj, Varun Agrawal, and James Hays (Georgia Tech)
Fisher Yu (University of California, Berkeley)
By Meredith Alexander Kunz, Adobe Research
