Colpali Train Set: 127,460 Query-Image Pairs for Visual Document Retrieval

127,460 query-image pairs for visual document retrieval comprise this training set released by vidore in 2024. It combines 63% academic data from sources like DocVQA with 37% synthetic PDF pages augmented by Claude-3 Sonnet pseudo-questions.

Use Cases

• Training visual document retrieval models using the query-image pairs
• Fine-tuning vision-language models for document question answering using Claude-3 Sonnet pseudo-questions
• Benchmarking zero-shot generalization from English queries to non-English document images

Strengths

• 127,460 total training examples
• 37% synthetic data generated by Claude-3 Sonnet
• Includes established academic benchmarks like DocVQA

Limitations

• English-only text design prevents direct training on multilingual query sets
• Synthetic labels may contain model-generated hallucinations or artifacts

Provenance

Source

vidore (Arxiv 2407.01449)

Collection Method

Hybrid of academic datasets and synthetic generation via Claude-3 Sonnet

Freshness

Last updated June 2025.