ClaimReview2024+: A Benchmark for Multimodal Fact-Checking Systems

MAI-Lab released the ClaimReview2024+ benchmark on July 10, 2025. It contains 300 real-world claims sourced from professional fact-checking articles via ClaimReview markup. The dataset is designed to evaluate systems that classify claims as supported, refuted, misleading, or not enough information.

Use Cases

• Benchmarking multimodal fact-checking models based on the four-class classification task.
• Evaluating model generalization on claims released after GPT-4o's knowledge cutoff to avoid data leakage.
• Training classifiers to detect misleading information based on real-world fact-checking articles.

Strengths

• Contains 300 real-world claims.
• Specifically constructed to avoid data leakage with GPT-4o's knowledge cutoff.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

MAI-Lab

Collection Method

Sourced via ClaimReview markup from professional fact-checking articles.

Freshness

Last updated 2025-07-10 16:00:43.