SCI-CQA is a multimodal benchmark dataset for evaluating chart understanding, inspired by human exams. It contains 5,629 curated objective and open-ended questions paired with 2,894 chart images from scientific literature. The dataset was created by lyndons1 and last updated on April 28, 2025.
Use Cases
- Benchmarking chart question-answering models based on the 5,629 curated questions.
- Training multimodal vision-language models based on the 2,894 chart images and their associated questions.
- Evaluating model performance on scientific literature comprehension based on the dataset's source material.
- Developing models for open-ended reasoning about charts based on the included open-ended question format.
Strengths
- 5,629 carefully curated questions provide a substantial evaluation set.
- 2,894 chart images offer a visual corpus for multimodal tasks.
- Framework includes both objective and open-ended questions, enabling diverse evaluation.
Limitations
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count for the question-answer data is unknown, which may limit suitability assessment.
- Description metadata is limited; actual data quality requires manual inspection after download.
Provenance
- Source
- huggingface
- Collection Method
- Curated from scientific literature, as described in the associated paper.
- Time Range
- null
- Freshness
- Last updated 2025-04-28 06:40:25.
- Geography
- null