Label Errors: Corrected Test Sets for 10 Major ML Benchmarks

Cleanlab provides corrected test set labels for 10 major machine learning benchmarks including ImageNet, CIFAR, and IMDB, updated through late 2025. The collection identifies and rectifies mislabeled examples across image, text, and audio modalities to ensure more accurate model evaluation. It serves as a verification layer for researchers to measure true model performance against cleaned ground truth.

Use Cases

• Benchmarking model accuracy on ImageNet using corrected ground truth labels
• Evaluating label noise robustness in text classifiers for Amazon Reviews
• Auditing error rates in AudioSet test partitions to improve audio classification

Strengths

• Covers 10 major benchmarks across 3 distinct modalities
• Apache-2.0 licensed for open research and commercial use
• Expert-verified corrections of standard evaluation sets

Limitations

• Does not include the raw media files (images, audio, or text) which must be sourced from original providers
• Limited to test set corrections which may not represent the noise distribution in training sets

Provenance

Source

Cleanlab

Collection Method

Algorithmic detection of label errors followed by human-in-the-loop verification

Freshness

Last updated December 2025.

Geography

Global