DARE: A Multiple-Choice Visual Question Answering Benchmark for Robustness

DARE (Diverse Visual Question Answering with Robustness Evaluation) is a multiple-choice VQA benchmark created by cambridgeltl. It evaluates Vision-Language Model performance across five diverse categories and includes four robustness-oriented evaluations based on variations in prompts, answer options, output format, and the number of correct answers. The validation split contains images, questions, answer options, and correct answers.

Use Cases

• Benchmarking VLM robustness based on prompt variations mentioned in the description
• Evaluating model performance on multiple-choice questions with varying answer option subsets
• Testing VLM adaptability to different output formats as part of the robustness evaluation
• Assessing model accuracy on questions with a variable number of correct answers

Strengths

• Designed to evaluate performance on five diverse categories of visual questions
• Includes four distinct robustness-oriented evaluation dimensions
• Validation split contains images, questions, answer options, and correct answers

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download
• Row count is unknown, which may limit suitability assessment
• Column-level documentation is absent; field semantics must be inferred after download

Provenance

Source

cambridgeltl on Hugging Face

Collection Method

Carefully created and curated, as stated in the description

Freshness

Last updated 2025-02-11 10:32:38