Advertisements are an integral part of internet economics and culture, and video ads are the most popular and arguably the most entertaining form of advertisements. With the recent growth in digital marketing, video ads have seen unprecedented growth and are growing in importance as an advertising means. Video ads are expensive to create and are not always effective. The effectiveness of a video ad is usually not known before its deployment, which is non-ideal for creators, advertisers, and ad platforms. In this paper, we outline an idea to provide feedback before an ad is placed on its effectiveness based on the video along with the historical data about the effectiveness of other video ads. We propose a multi-modal mixture based algorithm to predict the effectiveness automatically. Specifically, we exploit rich textual information often found with an advertisement as well as visual information to learn a finite mixture model. Our experiments on a publicly available dataset show that our approach can outperform other baseline approaches.