Visual Question Answering Dataset Requiring External Knowledge

OK-VQA contains 14,055 open-ended visual questions, each with 5 ground truth answers. The dataset is manually filtered to ensure all questions require outside knowledge, such as from Wikipedia, and has been processed to reduce bias from common answers.

Use Cases

• Train a visual question answering model on 14,055 open-ended questions requiring external knowledge
• Evaluate model performance using the 5 ground truth answers per question as a benchmark
• Analyze the distribution of required knowledge types across manually filtered questions

Strengths

• 14,055 open-ended questions provide a substantial testbed for VQA models
• 5 ground truth answers per question offer a robust evaluation metric
• Manual filtering ensures all questions require outside knowledge, creating a specific challenge

Limitations

• Dataset size of 14,055 questions may be limited for training large-scale models without augmentation
• Focus on questions requiring external knowledge may not represent general VQA tasks
• Potential for residual bias despite efforts to reduce questions with common answers

Provenance

Source

HuggingFaceM4

Collection Method

Manually filtered questions to require outside knowledge; reduced questions with common answers to mitigate bias.

Freshness

Last updated in June 2022.