OCR Bench Britannica: Pairwise VLM-as-Judge Evaluation Results

Pairwise evaluation results for OCR models using a VLM-as-judge methodology on the Britannica OCR benchmark. Created by davanstrien and updated in March 2026, the data tracks performance metrics for five models across fewer than 1,000 records.

Use Cases

• Benchmarking OCR models using the ELO column
• Assessing model reliability through the 95% CI field
• Comparing relative performance using Win% and Wins counts

Strengths

• Includes 95% CI statistical confidence intervals for ELO ratings
• Provides granular pairwise Win/Loss/Tie counts
• Uses VLM-as-judge methodology for comparative ranking

Limitations

• Small sample size with fewer than 1,000 records
• Results are contingent on the specific Qwen35 judge model's biases
• Covers a narrow selection of five OCR models

Provenance

Source

davanstrien (Hugging Face)

Collection Method

VLM-as-judge pairwise evaluation

Freshness

Last updated March 2026.

Geography

United States