Chartographer: Counterfactual Chart Variants for Visual Reasoning Evaluation

Chartographer is a dataset designed to test if vision-language models reason about charts visually rather than using shortcuts. Each chart-question family includes an original chart, a reconstructed version, and ten seed-controlled counterfactual variants. The dataset was created by author 1fanj and was last updated on Hugging Face in May 2026.

Use Cases

• Evaluating visual reasoning capabilities of vision-language models based on counterfactual chart variants.
• Benchmarking model robustness against spurious correlations using seed-controlled data families.
• Training models to focus on visual chart elements rather than textual or prior knowledge shortcuts.

Strengths

• Designed with a structured counterfactual generation pipeline to isolate visual reasoning.
• Contains ten controlled variants per chart-question family to test for specific reasoning failures.
• Includes both original and reconstructed charts for comparative analysis.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

Hugging Face (author: 1fanj)

Collection Method

Constructed via a pipeline generating original, reconstructed, and counterfactual chart variants.

Freshness

Last updated 2026-05-28 18:03:06; freshness should be verified.