CharXiv: 2,323 High-Resolution Scientific Charts for Multimodal LLM Benchmarking

CharXiv is a diverse and challenging benchmark for chart understanding, fully curated by human experts. It includes 2,323 high-resolution charts manually sourced from arXiv preprints. The dataset was created by princeton-nlp and released in 2024.

Use Cases

• Benchmarking multimodal LLM performance on chart understanding based on the described high-resolution scientific charts.
• Training models to interpret data visualizations from academic papers based on the arXiv-sourced content.
• Studying gaps in model reasoning with complex, real-world charts as described in the NeurIPS 2024 paper.

Strengths

• 2,323 high-resolution charts provide a substantial testbed for evaluation.
• Charts are manually sourced from arXiv preprints, ensuring a realistic and diverse sample.
• Dataset is fully curated by human experts, as stated in the description.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Data may reflect source bias inherent to the arXiv preprint repository.

Provenance

Source

princeton-nlp

Collection Method

Charts were manually sourced from arXiv preprints.

Freshness

Last updated 2024-09-27 20:16:18; freshness should be verified.