Wondering how artificial intelligence (AI) will transform creativity and help build the metaverse? We got some big hints at this year’s Adobe MAX, where Sneaks from the Adobe Research team showed new tools for everything from video editing to designing beautiful, immersive 3D worlds.
Comedian, entrepreneur, and Adobe MAX Sneaks co-host Kevin Hart was on hand to help Researchers present their latest projects, along with Adobe’s Bria Alexander, senior design program manager.
Here’s everything you need to know about the Sneaks from Adobe Research this year.
Project Clever Composites
Usually, compositing images is a tedious project. Users carefully cut out an object and manually edit it to blend into another image. But Project Clever Composites makes compositing images as easy as dragging-and-dropping. Using AI, the tool can automatically identify a suitable object and cut it out, adjust its color and size, and generate shadows based on the new background lighting — all in a few simple clicks.
Presenter: Zhifei Zhang
Collaborators: Zhe Lin, Scott Cohen, and Jianming Zhang
Project Instant Add
Editing videos and adding VFX effects in post-production can be complex and time consuming. That’s why Project Instant Add uses AI and machine learning to let users edit video content more easily — just the way they edit images. Users can choose an element in a video and Project Instant Add automatically maps text or graphics directly to the chosen element.
Presenter: Joon-Young Lee
Collaborators: Oliver Wang, and Gabriel Huang (university-based contributors: Leonid Sigal and Kwang Moo Yi)
Project Motion Mix
Project Motion Mix lets users create high-quality looping human animations, complete with realistic human movements, from a still image. Usually, video creators need to capture each unique motion before adding it to an animation. But Project Motion Mix uses AI-based motion generation and human rendering technologies to create high-quality, realistic 3D movements automatically. Creators can also make changes to the video’s background, including adding a friend to join in the animation.
Presenter: Jae Shin Yoon
Collaborators: Chengan He, Yi Zhou, Duygu Ceylan, Jun Saito, Tuanfeng Wang, James Zachary, Jimei Yang, Zhixin Shu, and Cynthia Lu
Project Blink
Project Blink is a video editing tool that uses AI to help users find and quickly pull highlights from video content, making video editing as easy as text editing. To create clips, users just search for specific words, objects, sounds, or even types of activities in the video, then select the portion of the video transcript they’d like to use. Adobe AI automatically transforms that section of the video into a new clip.
Project Blink is already available in beta.
Presenter: Mira Dontcheva
Contributors:
Current Blink Core Members: Joel Brandt, Aseem Agarwala, Jovan Popovic, Joy Kim, Valentina Shin, Dingzeyu Li, Xue Bai, Jui-Hsien Wang, Hanieh Deilamsalehy, Mira Dontcheva, Ailie Fraser, Justin Salamon, Karrie Karahalios, Haoran Cai, Pankaj Nathani, Anh Truong, Esme Xu, Kim Pimmel, Muj Syed, and Ray Ma
Past Blink Core Members: Seth Walker, Cuong Nguyen, Chelsea Myers, Morgan Evans, Najika Yoo, Leo Zhicheng Liu, Tim Ganter, and Aleksander FicekContributors from outside the Blink Core Team: Ali Aminian, Aashish Misraa, Xiaozhen Xue, Nick Bryan, Yu Wang, Xiaozhen Xue, Fabian David Caba Heilbron, Ajinkya Kale, Ritiz Tambi, Juan Pablo Caceres, Ge Zhu, Oriol Nieto, Trung Bui, Naveen Marri, and Sachin Kelkar
Project Artistic Scenes
The metaverse has opened a whole new world of possibilities for 3D immersive content, but creating that content can take significant time and expertise. That’s why Project Artistic Scenes uses AI to automatically create entire 3D scenes in artistic styles from 2D artwork. For example, imagine a fully immersive 3D playground rendered as a watercolor painting, complete with realistic brushstrokes.
Presenter: Sai Bi
Collaborators: Fujun Luan, Zexiang Xu, Kalyan Sunkavalli, Nick Kolkin, and Eli Shechtman (university-based contributors: Kai Zhang and Noah Snavely)
Project All of Me
It’s easy to crop an image down, but much harder — and sometimes impossible — to un-crop if you need a larger size. Enter Project All of Me, a smart portrait editor that uses AI to generate the missing components of an image so it can be un-cropped. Project All of Me also provides recommendations, lets users edit out unwanted distractions, and makes it easy to modify the subject’s apparel.
Presenter: Qing Liu
Collaborators: Jianming Zhang, Zhe Lin, Scott Cohen, and Krishna Kumar Singh
Project Beyond the Seen
Project Beyond the Seen uses AI to generate full 360° panoramas from a single 2D image, pushing the limits of immersive virtual reality. Adobe’s AI technology can generate content behind, above, below, and to the sides of the image, so users can easily and quickly add objects, including AI-generated reflections and more.
Presenter: Yannick Hold-Geoffroy
Collaborators: Mohammad Karimi, J Eisenmann, Simon Niklaus, Jianming Zhang, Matheus Gadelha, and Kevin Matzen (university-based contributor: Jean-François Lalonde)
Project Made in the Shade
Project Made in the Shade uses AI for editing shadows in 3D images — a task that usually requires complex knowledge, advanced skills, and a lot of time. Since Adobe’s AI technology recognizes depth, lighting, and other aspects of a scene, users can move a person or object within a photo while casting realistic shadows, which is especially useful for compositing 3D images and making 3D motion graphics.
Presenter: Vojtěch Krs
Collaborators: Matheus Gadelha, Yannick Hold-Geoffroy, Giorgio Gori, Radomir Mech, and Nathan Carr
More Adobe Research Behind the Sneaks
Beyond the eight Adobe Research-led projects, Researchers also contributed to two more Sneaks.
Project Vector Edge helps designers visualize 2D designs in the 3D world. Adobe Research contributors include Siddhartha Chaudhuri, Simon Niklaus, Jianming Zhang, Matheus Gadelha, Kevin Blackburn-Matzen, Yannick Hold-Geoffroy, and Nathan Carr.
Project Magnetic Type lets users attach and fuse any shape to live digital text. Adobe Research contributors include Matthew Fisher.
Want even more glimpses into the future of creativity and technology? Check out the full Adobe MAX 2022 Sneaks here.
