LREC 2020

TutorialVQA: Question Answering Dataset for Tutorial Videos

Anthony Colas, Seokhwan Kim, Franck Dernoncourt, Siddhesh Gupte, Zhe Wang, Doo Soon Kim

Read Research Paper

Despite the number of currently available datasets on video-question answering, there still remains a need for a dataset involving
multi-step and non-factoid answers. Moreover, relying on video transcripts remains an under-explored topic. To adequately address
this, We propose a new question answering task on instructional videos, because of their verbose and narrative nature. While previous
studies on video question answering have focused on generating a short text as an answer, given a question and video clip, our task aims
to identify a span of a video segment as an answer which contains instructional details with various granularities. This work focuses
on screencast tutorial videos pertaining to an image editing program. We introduce a dataset, TutorialVQA, consisting of about 6,000
manually collected triples of (video, question, answer span). We also provide experimental results with several baseline algorithms
using the video transcripts. The results indicate that the task is challenging and call for the investigation of new algorithms.