MetaShift Image Collection for Distribution Shift Evaluation

MetaShift is a dataset of datasets containing 12,868 sets of natural images across 410 classes. It was created by jameszou707 for evaluating machine learning model performance across diverse data distributions.

Use Cases

• Evaluate model robustness to distribution shifts across 12,868 image sets.
• Analyze training conflicts in multi-label image classification across 410 classes.
• Benchmark domain generalization methods using crowdsourced annotations from the US region.

Strengths

• Contains 12,868 distinct sets of images for analyzing data distributions.
• Covers 410 diverse image classes for multi-label classification tasks.
• Annotations were crowdsourced, providing a basis for real-world label variation.

Limitations

• Specific column structure, sample data, and file formats are unknown.
• The dataset's size and exact row count are not provided.
• Geographic coverage may be biased, as annotations are primarily from the US region.

Provenance

Source

huggingface

Collection Method

Collection of natural image sets, with annotations created via crowdsourcing.

Freshness

Last updated on 2024-09-10.

Geography

Primarily United States (US), based on region tag.