PubMed Medical Image-Question Pairs with GPT-4V Annotations

PubMedVision is a large-scale medical visual question answering dataset built from image-text pairs extracted from PubMed. FreedomIntelligence enhanced the data quality using GPT-4V and added annotations for body parts and modality. The dataset was updated in February 2025.

Use Cases

• Train medical VQA models using the image-question-answer pairs.
• Fine-tune vision-language models on the annotated body_part and modality labels.
• Benchmark model performance on reformatted, high-quality medical image-text data.
• Explore cross-lingual medical VQA using the provided Chinese version of the dataset.

Strengths

• Dataset size categorized as between 1 million and 10 million entries.
• Annotations for body_part and modality were generated using the HuatuoGPT-Vision-7B model.
• Data quality was enhanced using GPT-4V.

Limitations

• Specific row counts, column details, and sample sizes are not provided.
• The reliance on model-generated annotations (GPT-4V, HuatuoGPT-Vision) may introduce label noise.
• Original data source (PubMed) may have inherent biases in publication focus and geographic representation.

Provenance

Source

PubMed, a biomedical literature database.

Collection Method

High-quality image-text pairs extracted from PubMed and reformatted using GPT-4V.

Time Range

null

Freshness

Last updated February 18,, 2025.

Geography

null