In this paper, we propose QACE, a new metric based on Question Answering for Caption
Evaluation. QACE generates questions on the
evaluated caption and checks its content by
asking the questions on either the reference
caption or the source image. We first develop
QACERef that compares the answers of the
evaluated caption to its reference, and report
competitive results with the state-of-the-art
metrics. To go further, we propose QACEImg,
which asks the questions directly on the image, instead of reference. A Visual-QA system
is necessary for QACEImg. Unfortunately, the
standard VQA models are framed as a classification among only a few thousand categories.
Instead, we propose Visual-T5, an abstractive
VQA system. The resulting metric, QACEImg
is multi-modal, reference-less, and explainable. Our experiments show that QACEImg
compares favorably w.r.t. other reference-less
metrics.