CIFAR-10 Test Images with Human Label Distributions

CIFAR-10H extends the CIFAR-10 test set by adding human-annotated label distributions. It provides probability distributions, raw vote counts, and majority-choice labels for each image. The dataset was created by MKZuziak and uploaded to Hugging Face in April 2026.

Use Cases

• Analyze human label uncertainty by comparing expert_probs distributions across different image categories.
• Benchmark model confidence against human confidence using expert_counts vote counts for each class.
• Train models to predict human majority labels expert_argmax from image features.
• Study the relationship between image difficulty and the variance in expert_probs probability distributions.

Strengths

• Provides human uncertainty metrics expert_probs and expert_counts for the standard CIFAR-10 test images.
• Includes one-hot encoded labels expert_argmax derived from human majority votes.

Limitations

• Dataset size, row count, and sample size are unknown.
• Limited to the 10 classes and images of the original CIFAR-10 test set.
• Human annotation details, such as number of annotators or collection method, are not provided.

Provenance

Source

Extension of the CIFAR-10 test set.

Collection Method

Human annotation providing label distributions.

Time Range

null

Freshness

null

Geography

null