In this paper, we present a multimodal dialogue system for Conversational Image Editing. We formulate our multimodal dialogue system as a Partially Observed Markov Decision Process (POMDP) and trained it with Deep Q-Network (DQN) and a user simulator. Our evaluation shows that the DQN policy outperforms a rule-based baseline policy, achieving a 90% success rate under high error rates. We also conducted a real user study and analyzed real user behavior.
AI & Machine Learning
Computer Vision, Imaging & Video
Human Computer Interaction
Intelligent Agents & Assistants
Natural Language Processing
A Multimodal Dialogue System for Conversational Image Editing
Tzu-Hsiang Lin, Trung Bui, Doo Soon Kim, Jean Oh
Read Research Paper