MMB: Counterfactual Visual Question Answering Images and Questions

A counterfactual VQA dataset constructed using CLEVR blender assets to procedurally generate both negative and normal counterfactual images and questions. The dataset was created by author 'scholo' for the Multimodal Benchmark paper and was last updated on Hugging Face in February 2026. It contains original images, counterfactual variants, and corresponding questions.

Use Cases

• Benchmarking visual question answering models on counterfactual reasoning tasks based on procedurally generated image variants.
• Training models to distinguish between original and modified scenes based on the described counterfactual image pairs.
• Studying model robustness to visual perturbations using the generated negative counterfactual examples.
• Developing interpretability methods for multimodal systems using the structured question-answer pairs.

Strengths

• Dataset is specifically designed for counterfactual reasoning, a key area in AI evaluation.
• Data was procedurally generated using established CLEVR assets, which suggests controlled and consistent construction.
• Contains both original and counterfactual image variants, enabling paired comparisons.

Limitations

• Description metadata is limited; actual data quality, size, and structure require manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count, file formats, and license information are unknown, which may limit suitability assessment.

Provenance

Source

Hugging Face (user: scholo)

Collection Method

Procedurally generated using CLEVR blender assets.

Freshness

Last updated 2026-02-19 17:35:33; freshness should be verified.