Describe Anything: 100K-1M Localized Image and Video Captions

NVIDIA, UC Berkeley, and UCSF released this collection of 100,000 to 1,000,000 records in 2025 for training Describe Anything Models (DAM). The data consists of localized image and video captions stored in WebDataset tar files to support vision-language tasks.

Use Cases

• Training localized image captioning models to describe specific visual regions
• Developing video-to-text algorithms for temporal action description
• Fine-tuning Describe Anything Models (DAM) for multi-modal scene understanding

Strengths

• Scale of 100,000 to 1,000,000 records
• Dual-modality support for both image and video data
• Expert-backed research from NVIDIA, UC Berkeley, and UCSF

Limitations

• Geographic bias toward the US region as indicated by metadata tags
• Requires specialized WebDataset tools to handle tar-based storage format

Provenance

Source

NVIDIA, UC Berkeley, and UCSF

Collection Method

Annotated for Describe Anything Models (DAM) training

Freshness

Last updated April 2025.

Geography

United States