Charts are widely used representations of data in documents. Thus, it is critical to investigate into technologies that can understand charts and figures. One of the challenges is here is to collect wide variety of charts representing the real world data distribution, and then annotate them to enable learning framework. In this project, we are looking into building a rich corpus of annotated charts of different kinds. Currently, it has been accepted as a competition in ICDAR 2019, for which we will be posting the data for participants here.