TiC-DataComp: Metadata for Time-Continual Learning Benchmark

Apple provides metadata for the TiC-DataComp benchmark, which evaluates time-continual learning for image-text models. The dataset contains timestamp groupings by year and month for DataComp-1B images, sourced from CommonCrawl. It also includes unique identifiers for TiC-DataCompNet and TiC-DataComp-Retrieval evaluations of CLIP models.

Use Cases

• Benchmarking continual learning performance of CLIP models based on timestamped image-text data.
• Analyzing temporal distribution shifts in large-scale web-crawled datasets based on year/month groupings.
• Evaluating retrieval and classification tasks in a time-continual setting using the provided evaluation UIDs.

Strengths

• Metadata is sourced from the widely-used CommonCrawl web corpus.
• Includes identifiers for two specific evaluation suites: TiC-DataCompNet and TiC-DataComp-Retrieval.
• Timestamp information is organized by year and month, enabling temporal analysis.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

Apple, via Hugging Face.

Collection Method

Timestamp information sourced from the original CommonCrawl data used for DataComp-1B.

Freshness

Last updated 2024-06-13 16:25:19; freshness should be verified.