Publications

DrawTalking: Building Interactive Worlds by Sketching and Speaking

UIST 2024

Publication date: October 13, 2024

Karl Toby Rosenberg, Rubaiat Habib, Li-Yi Wei, Haijun Xia, Ken Perlin

We introduce DrawTalking, an approach to building and controlling interactive worlds by sketching and speaking while telling stories. It emphasizes user control and flexibility, and gives programming-like capability without requiring code. An early open-ended study with our prototype shows that the mechanics resonate and are applicable to many creative-exploratory use cases, with the potential to inspire and inform research in future natural interfaces for creative exploration and authoring.

Learn More

Research Area:  Adobe Research iconHuman Computer Interaction