VizWiz-VQA: Visual Question Answering Data for Multimodal Model Evaluation

VizWiz-VQA is a large-scale dataset for evaluating large multi-modality models. It is a formatted version used in the lmms-eval pipeline for one-click model evaluations. The dataset was created by lmms-lab and was last updated on March 8, 2024.

Use Cases

• Benchmarking visual question answering models based on the dataset's multimodal nature.
• Training models to answer questions about images, as suggested by the Visual Question Answering (VQA) domain.
• Evaluating model robustness on a large-scale evaluation suite as described in the dataset's purpose.

Strengths

• Designed for large-scale evaluation of multi-modality models.
• Integrated into the lmms-eval pipeline for streamlined one-click evaluations.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

lmms-lab on Hugging Face, based on the original VizWiz Grand Challenge dataset.

Collection Method

Formatted version of the original VizWiz-VQA dataset for use in an evaluation pipeline.

Freshness

Last updated 2024-03-08 05:11:16; freshness should be verified.