The sound of a scene: Adobe Research brings video-guided AI soundscapes to Premiere Pro

Whether it’s the ambient sounds that establish a sense of place or carefully timed foreground effects, sound transforms a scene. But creating a soundscape often means sourcing, generating, and manually synchronizing individual sounds, a process that can be especially daunting and time-consuming for video creators.

That’s why Adobe Research and the Premiere team worked together to develop Generate Soundscape, a new AI-powered feature in Adobe Premiere Desktop that uses video understanding to analyze footage and generate layered, synchronized audio directly in the timeline.

Combining Adobe Research’s advances in video-guided sound generation with the Premiere team’s understanding of video editors’ workflows, Generate Soundscape makes it easier for creators to build immersive audio experiences, providing a starting point for sound design while keeping individual sounds editable and under their control.

The blank page problem

For video creators, one of the biggest challenges in sound design is knowing where to begin. Justin Salamon, Principal Scientist and head of the Sound Design AI team at Adobe Research, describes this as the "blank page problem." Even with tools that generate individual sound effects, editors still need to decide which sounds a scene requires, describe each one, and arrange them along the timeline.

"I think that for a lot of video creators, sound design can be somewhat intimidating," says Salamon. “Generate Soundscape helps creators reach a good starting point from which they can iterate and really make it their own."

To help users get started with their sound design, Generate Soundscape first analyzes what's happening on screen and then automatically proposes four complementary audio tracks: two for background ambience and two for foreground sound effects.

For example, a video of someone walking through a forest might call for the rustling of leaves and birdsong in the background, alongside the foreground of footsteps synchronized to the person's movements. Generate Soundscape can then build those layers together, giving creators an initial sound design they can refine.

Creating reliable, synchronized sound generation

A key step toward Generate Soundscape came from Adobe Research's work on video-guided sound generation, presented at CVPR 2025. The research explored how AI could generate high-quality sound effects that synchronize with actions on screen, helping make sound generation more responsive to what's happening in a video.

For Oriol Nieto, a Senior Research Engineer at Adobe Research and co-author of the paper, one of the biggest challenges was helping the model distinguish between different actions happening simultaneously. For example, if someone closes a door while another person moves across the frame, the model needs to recognize which action should trigger the sound.

“Before it reaches users, you need a good video model that understands what's happening,” explains Nieto.

Adobe Research Engineer Johannes Imort, who also worked on bringing Generate Soundscape into Premiere Desktop, explains how the team built on this research, using targeted training data and evaluations with human listeners, including professional sound designers, to improve the model's performance across different types of sounds.

“Research papers are a proof of concept, so making it work for any kind of sound means more work beyond that,” says Imort. “For example, listeners noticed the model was not yet working on footsteps. Getting this feedback was really helpful in identifying cases where we could improve the model.”

These advances helped power Sync to Video, a capability in Premiere Desktop that uses visual information to guide the timing of individual generated sound effects.

"With Sync to Video, you provide a prompt to describe the type of sound you want, and then when you hit Sync to Video, the timing and the energy of the sound come from the action on screen," explains Salamon.

From research prototype to product through co-development

Generating an individual sound effect synchronized to video was one step. The next was bringing multiple sounds together to create a complete soundscape.

That idea took shape in Project Sound Stager, a research prototype demonstrated by Nieto as an Adobe MAX Sneak in 2025 showcasing how AI can help creators generate and refine layered soundscapes for video.

The enthusiasm for Project Sound Stager helped spark a collaboration between Adobe Research and the Premiere team that led to the development of Generate Soundscape.

"We really like working closely with product teams at an early stage rather than just giving them a shopping list of features to choose from," says Salamon. "That way, we can co-develop our technology, getting feedback really early on."

During the development process, the product team shared important feedback about how to streamline the capabilities all in one place to match how creators actually like to work. The final feature integrates sound generation directly into the Premiere timeline, making it easy for creators to select the Generative Media Tool and drag a range selection across a portion of video to automatically generate corresponding audio tracks beneath the footage.

Giving creators control over sound

For Salamon and the team, the vision wasn’t simply to generate audio with less effort. They also wanted to give creators the freedom to experiment and spend more time making creative decisions.

"Our goal is to build AI tools that empower you to be creative, without replacing the creative process," says Salamon.

That philosophy is reflected in how Generate Soundscape handles its audio output. Rather than combining generated sounds into a single, uneditable track, the feature keeps each sound separate, allowing editors to adjust volume, trim duration, explore variations, or regenerate one element with a new prompt without affecting the rest of the soundscape. For example, a creator could replace the sound of footsteps while preserving the background ambience and other effects, without having to regenerate the entire soundscape.

“One of the things that I really love is to give control back to the user,” says Nieto. “I like having a human in the loop, and having lots of interaction.”

The next frontier in sound design

Generate Soundscape brings video-guided audio generation directly into the editing timeline, giving creators a new way to build and refine the sounds of a scene.

And Adobe Research is continuing to explore new ways for creators to interact with sound. At Adobe MAX 2026, the team demonstrated Project Sound Bender, a research prototype that explores how creators could reshape and manipulate audio interactively.

For creators ready to experiment with AI-powered sound design today, Generate Soundscape is available in Premiere Pro.

Wondering what else is happening in Adobe Research? Check out our latest news here.