Video Visualization and Interaction

Dan Goldman

Adobe Research

Brian Curless

University of Washington

David Salesin

Adobe Research, University of Washington

Steven Seitz

University of Washington

Existing approaches to interaction with digital video are complex, and some operations lack the immediacy of interactive feedback. Our research in video annotation, visualization, and interaction harnesses computer vision to aid users in understanding and communicating with digital video. We are developing new approaches for computing the motion of points and objects in a video clip, and interactive systems that utilize this data to visually annotate independently moving objects in the video. We have applied this interface to construct visualizations of a short video clip in a single static image, using the visual language of storyboards. The principal advantage of the storyboard representation over standard representations of video is that it requires only a moment to observe and comprehend but at the same time retains much of the detail of the source video. The layout of the storyboard can be optimized to place the elements in a configuration that maximizes the clarity of presentation. We also explore novel interaction techniques for random video frame access using the natural spatial dimensions of a storyboard representation or an individual video frame.

A layman’s introductory video to these technologies can be found here, and the official supplementary video for the paper is embedded below:

Project Publications

A framework for video annotation, visualization, and interaction

Goldman, D (Jun. 2, 2007)
PhD Thesis, University of Washington, 2007.

Schematic storyboarding for video visualization and editing

Goldman, D, Curless, B., Seitz, S., Salesin, D (Jul. 1, 2006)
ACM Transactions on Graphics (Proc. SIGGRAPH) , 25(3), 862–871, 2006.

Video object annotation, navigation and composition

Goldman, D, Gonterman, C., Curless, B., Salesin, D, Seitz, S. (Oct. 2, 2008)
In UIST '08: Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology , 3–12, 2008.